Unicode Bidirectional Algorithm

Hi there,
How about include Unicode Bidirectional Algorithm to LittlevGL? Since:

It is supossed to be used all over the world.
It supports unicode characters.
It supports different languages handling.

It seems hard to implement it, but doing a quick research there is a library that could do it for us --> GNU FriBidi.
I have never work with Unicode Bidirectional Algorithm, but maybe any of you have and could tell us about the possibility of including it for LittlevGL.

@beibean

Is FriBidi designed for embedded systems  (i.e. low memory requirements, small code size)?
It's LGPL licensed. We only accept MIT-licensed code into the project. I also don't think relying on incompatibly-licensed external libraries is wise.

@kisvegabor

As I said, I don't know much about this library and Unicode Bidirectional Algorithm in general.
It was just a suggestion to take into account, using FriBidi, other third party library or implementing by ourselves.
I really think this will endow LittlevGL with a good feature ;)

I'm planning to implement a simple and extendible text shaping engine for v6.0 later.
My idea is to assign some glyph modifier rules to the fonts.
I leave this issue open as memo.

I think it doesn't worth consider each unicode feature separate. Because attempt to add every new feature can change everything in API and implementation. It would be more consistent to inspect existing JS project, pick required features subset and design all at once.

FontKit - better glyphs combine, limited kerning tables support, no hinting support.
opentype.js - meduim combine, better kerning, hinting
text-rendering-tests - tests for different rendering engines

Fontkit uses unicode-properties to access character attributes. Compressed DB size is ~ 12Kb.

I was thinking about the complexity of this whole thing (text shaping, bidirectional support, kerning etc) and I realized it's a much bigger work (if done well) than it seemed at first look. Knowing that it should be the main feature of a new release later, instead of parot of dev-6.0.

Seriously, use freetype+sdl+harfbuzz on less cheaper hardware.
That is a pandora box one should never open especially knowing there are scripts requiring deferred shaping.

Seriously, use freetype+sdl+harfbuzz on less cheaper hardware.

At some point I will need RTL support, either through standard feature like this one or by some hack.
I don't care much about compliant unicode bidirectional support, I just need a proper way to display RTL characters, align text to the right and mix RTL characters with digits and other symbols which are LTR characters.
I don't think it makes sense to switch hardware and software framework to achieve this. It should be simple enough to do in LittlevGL on a cheap MCU, at least for my use case.

@amirgon then i'd suggest a pre-rendering of words via the SDL simulator ( with added harfbuzz+freetype that's easy) and storing that in flash, think of a gettext but with graphical hooks encoded on https://en.wikipedia.org/wiki/Private_Use_Areas

then i'd suggest a pre-rendering of words via the SDL simulator

@pmp-p This is good for text you can pre-render.
But I'm getting some of the text from a remote server (as text strings), and I must render it on the embedded device.

well if it's RTL only that should be ok, my remark is from past experience and was specifically about "text shaping, bidirectional support, kerning etc" because you need to think callbacks from the start for rendering text and that eats plenty of memory, also you can't get width/cesure until draw is completed => no batching.

Let's warm up this topic and collect goals and requirements. @amirgon, if I remember well, you live in an Arabic country so you know the details of  RTL.
Things not clear for me:

If you use RTL in "real life" your editor is still LTR?
How should the LTR "some word 123" string rendered in RTL?
How should the LTR "word1 > word2" string rendered in RTL?

The required features I see:

Text shaping:

Built-in support for very common character replacement (E.g. Arabic)
Possible callback to extend it

RTL support

Text level? A text can be RTL or LRT, no mixing
With control characters? E.g. RTL "right to left" LTR "left to right" RTL "rtl again"

Vertical writing?

I don't think it's required

What do you think?

Here's a good overview https://www.w3.org/TR/alreq

@amirgon, if I remember well, you live in an Arabic country so you know the details of RTL.

Almost! 🤣
I live in Israel so my main RTL language is Hebrew. I can answer these question in that context. AFAIK Arabic support is a bit harder because one unicode code-point could represent multiple shapes (there is more than one way to write a letter, depends on letters before/after it), and letter shapes are connected to each other. In Hebrew it's simpler - letter shapes are disconnected and each letter has a single form.

1. If you use RTL in "real life" your editor is still LTR?

When I edit text, if I enter latin characters or numbers, they are typed LTR. If I enter Hebrew characters, they are inserted RTL on the position the cursor is at.

2. How should the LTR "some word 123" string rendered in RTL?

איזושהי מילה 123

3. How should the LTR "word1 > word2" string rendered in RTL?

מילה1 > מילה2
(Notice the editor actually replaced > with < automatically while I was typing! I don't think you need to support something like that...)

RTL support

Text level? A text can be RTL or LRT, no mixing

Actually mixing is important because numbers are LTR and text is RTL.
For my purpose, the priority would be:

Aligning text to the right on label and text area etc.
Mixing RTL and LTR, at least for display purposes when a string is given.

For my purpose, the priority would be:
* Aligning text to the right on label and text area etc.

* Mixing RTL and LTR, at least for display purposes when a string is given.

i think it that is implemented then it should be possible to handle basic arabic with the MS  (very)  simplified arabic fixed font (also known as CP1256 codepage ).

Here's a good overview https://www.w3.org/TR/alreq

@pmp-p Thank you for this reference. It seems (even on a basic level) it's far more complicated than simply replacing some characters based on the context. The diacritics is quite difficult to handle because (according to the article) they need some hints from the font file itself. Are the diacritics parts of the "real life" text? Or are they special cases?
@amirgon Thank you, getting clearer now.

@kisvegabor yeah diacritics and ligatures are omnipresent that's why cp1256 is really not the best solution to render whole arabic words.

diacritics and ligatures are omnipresent

That's a big problem because we don't have the required data from the font file to properly place the diacritics. I'm a little bit puzzled at this point...
One thing is still not totally clear. When you type the last complex character shown in Figure 15 you hit the "base characters" and then the two diacritics (3 key hits in total) and the last character is continuously changing in the editor? And in the memory, there will be 3 characters?

Lot's of ?  :)  but i'm in the same position as you : as i said it was "past" experience unicode was not yet a thing (and i still can't read or write arabic ! ).
From what i remember of early pyqt arabic support i don't think it's composition ( all char are stored ) and end of word is changing while typing ( and that's why rendering is delayed via a callback ), erasing char is also very funky and need hammering the del key.

Uh, that's tough. It seems to add diacritics in run time won't work because the placement information is missing. The possible solutions I see:

Add that missing information to the fonts. (looks like a long and complicated development)
Somehow store the composed characters. (no standard, difficult to use)
Pre-render characters or texts. (not dynamic)

The RTL support itself, and the character replacement looks also trick but possible.
To properly join characters coordinates also needs to be returned which implies a more complicated text shaping engine.
@amirgon
You mentioned only to support RTL. How do see the issues mentioned above?

@amirgon
You mentioned only to support RTL. How do see the issues mentioned above?

@kisvegabor In Hebrew it's much simpler, as I mentioned above - no diacritics whatsoever, and each code point has a single shape, no special text shaping.
So the only complications over Latin script (which is currently supported) are

Aligning everything to the right
Handling RTL and LTR in the same string. This is important because numbers in Hebrew are written LTR as usual, and it's not uncommon to find an english word in a sentence here and there.

I don't think that's very hard. Here is what I had in mind:

Just aligning (with lv_ta_set_text_align for example) is not enough because we need the first character of the logical string to be the right-most character visually, so we would need to reverse the order of the characters visually when displaying an RTL string.
For each code point we would need to identify the Bidi information: "directionality" of each character and whether it's "strong" or "neutral" (punctuation, whitespace). Some general explanation about Bidirectional Text on Wikipedia.  btw there are a few more types such as "weak" and "explicit formatting", but I don't think there's a need to implement these, at least not at first stage.
Specifying Bidi information can be done by either adding it to the font data, or even by some callback that specifies Unicode ranges hard coded. For Hebrew script, there are only a few such ranges.
According to the Bidi information (character "directionality", "Strong/Neutral"), we can implement a very simplified Bidi algorithm that takes a logical string, identifies "runs" of characters with the same directionality, takes into account "strong/neutral" characters and creates the visual representation of the logical string.

This would allow mixing RTL and LTR characters for most cases, in Hebrew, which is good enough for me.
It might also be good enough for the very simplified Arabic script of CP1256 codepage, as @pmp-p suggested, but I'm not completely sure about that.

@amirgon Thanks for the clarification. I've missed these differences in Hebrew. If I understood well in Hebrew:

character replacement based on context is still required
joining characters (change letter space) is not required

I've done some research and found that Hebrew is spoken by few million people while Arabic is spoken by more than a hundred million people. Would it be useful enough to support Hebrew and Arabic in a limited way (with CP1256 or similar)?

@kisvegabor

character replacement based on context is still required

No, each code point has one shape, so there's no need to replace characters by context.
The only additional information you need for each character (or character range) is Bidi information (is this character a strong RTL, a strong LTR, or neutral).

joining characters (change letter space) is not required

Correct.

I've done some research and found that Hebrew is spoken by few million people while Arabic is spoken by more than a hundred million people. Would it be useful enough to support Hebrew and Arabic in a limited way (with CP1256 or similar)?

I don't know how useful a limited CP1256 Arabic script would be. We would need someone with more knowledge of Arabic script to comment on that. My knowledge is limited to Hebrew.
There are ~10 million native Hebrew speakers and ~420 million Arabic speakers, so I can see your motivation to promote Arabic script support in LittlevGL.
I'm using LittlevGL for my own project, where I would need Hebrew support and not Arabic at this point. This is where my motivation to promote Hebrew support in LittlevGL comes from.
In case Hebrew will not be formally supported in LittlevGL, I'll probably need to fork LittlevGL and hack Hebrew support in it. But first I'm suggesting the option to do it more cleanly on upstream lvgl even though the target audience is smaller.

I see.
To support Hebrew (and bidi in general) without text shaping we can store the original text and the bidi formatted text in the label. This way the bidi "translation" needs to be done only once in lv_label_set_text()
But word wrapping is still not clear:
The text "HEBREW ARABIC ENGLISH GERMAN" (let's say the "HEBREW ARABIC" words are RTL) should be written as "ENGLISH GERMAN CIBARA WERBEH". It should be wrapped as:
BEH
WER
ARA
CIB
ENG
LISH
GER
MAN

But the LittlevGL will wrap it as:
ENG
LISH
GER
MAN
CIB
ARA
WER
BEH

So word wrapping seems another challenging topic. What do you think about it? Do I see the issue well?

So word wrapping seems another challenging topic. What do you think about it? Do I see the issue well?

The behavior should be dependent on the bidi state of the label itself.
(or if you like, the default bidi state of the whole text in the label)
I'll try to demonstrate this on PowerPoint with a TextBox.
The text I typed is "עברית ערבית English German"
(Logically, the first two words are Hebrew and the last two words are English)
If the label is set as LTR:

And if it's set as RTL:

Thanks! Playing in PowerPoint is a good idea.
This point is interesting.

It rather begins a new line than put "Eng" in the same line with Hebrew characters.  It seems it's true only if the word itself is wrapped and whole words with different write direction can be in the same line.
I'm thinking about a reasonable approach to implement it.

If the words are long enough Libre Office Impress can mix LTR and RTL even is the word is wrapped:

I was thinking about it for a while and could come up with a reasonable approach but it has some limitations:

No text shaping for now, "only" RTL support
Support only LTR base direction
In case of line-wrapping, add a line break when the direction changes.
E.g.

ENGL
ISH
GERM
AN      < Line break on direction change
RBEH
A WE
IBAR
   C

What do you think??

@kisvegabor

No text shaping for now, "only" RTL support

Ok.

Support only LTR base direction

Actually I would need RTL base direction and alignment for my application, otherwise it will not be presented correctly (an additional English word would appear on the wrong side of the Hebrew sentence).
So choosing RTL/LTR base direction is important for me.
I guess this could be a "global" flag, to make the whole application LTR or RTL, if you think marking this per component is too expensive (although that's just 1 bit).

In case of line-wrapping, add a line break when the direction changes.

I could live with that, I think.

@amirgon
I see. In your particular application how important it is to handle line-wrapping? Basically almost all complications come from this. I see a way to implement it, but it would require another layer above "getting Unicode chars from UTF8". A lot of complicated parts of the library would be affected therefore probably some bugs would appear in various places.
I'm not against to add a proper way to handle line wrapping but I'd like to be sure if it's really required.

I'm not against to add a proper way to handle line wrapping but I'd like to be sure if it's really required.

@kisvegabor For the first phase of my application, I don't need line wrapping.
I think it's a good idea to support line wrapping in general, but we could split the work to phases and initially implement RTL support without line wrapping, as the first phase.
I do need, however, RTL base direction support (as I explained).

.. initially implement RTL support without line wrapping, as the first phase.

I afraid the two things are independent. A "simple" RTL support is quite straight forward: create a new text where the character's order is changed as required. It's a modular thing, basically an extra step in lv_label_set_text() like rtl_process(str_in, str_out).
On the other hand, to properly support line-wrap the text processing engine should be extended and reworked.
So it's rather a "quickly have something which is enough in lot of use-cases" approach.

I do need, however, RTL base direction support (as I explained).

Clear, it's ok.

It's a modular thing, basically an extra step in lv_label_set_text() like rtl_process(str_in, str_out).

@kisvegabor This I can do out of the scope of lvgl. Just translate one string to another before setting the text of a label.

Is there extra value in implementing a "simple" RTL support in lvgl vs. processing the string before setting the text in lvgl?
Did you think about of how to implement the base RTL/LTR? Globally? Per component?

Is there extra value in implementing a "simple" RTL support in lvgl vs. processing the string before setting the text in lvgl?

It really can be handled outside of lvgl as you described (even with a built-in lvgl function), but in this case, the user needs to take care with the extra buffer. E.g.
const char * txt = "some text";
char * txt_rtl = lv_mem_alloc(strlen(txt));
rtl_process(txt, txt_rtl);

/*Option1*/
lv_label_set_text(label, txt);
lv_mem_free(txt);

/*Option2*/
lv_label_set_text_static(label, txt);
...
lv_obj_del(label);
lv_mem_free(txt);

Another disadvantage could be that lv_label_get_text() will return a "messed-up" string. With internal handling both the original and the changed string could be stored.
On the other hand, this approach would give great flexibility. It'd be easy to pass ready-to-use strings to any object (e.g. lv_win_set_title()) without adding API functions for the direction and base direction.

Did you think about of how to implement the base RTL/LTR? Globally? Per component?

If we find a good way to handle the rtl translation outside of lvgl (like above) then we can have two functions for ltr and rtl base direction.
Else handling it per object could bloat the API. E.g.:

lv_win_set_tile
lv_win_set_tile_base_dir (might be replaced with global config)

What do you think, which approach would be better? Do you know other advantages/disadvantages I've missed?

@kisvegabor @amirgon Is RTL information normally stored inside the string itself (i.e. UTF-16/UTF-8 codes) or encoded separately?

@embeddedt The RTL information comes from the character code. A sequence of RTL character should be written Right to left.
Take look at this: https://stackoverflow.com/a/14824756/1999969

What do you think, which approach would be better? Do you know other advantages/disadvantages I've missed?

Do you plan to implement a full RTL support in the future? Perhaps with features that are needed for Arabic scripts as well?
If you do, maybe we can think of an API that would be forward compatible. For now it would only implement the simplistic string processing, but in the future the same API could be used for full RTL support.
If we can think of such an API in advance, it could prevent breaking the API in the future and waiting for a major version update.
For my specific purpose, memory allocation for text buffer is automatic when working with Micropython and I rarely use get_text functions, so handling it outside lvgl might be OK if only doing string processing.

@embeddedt The RTL information comes from the character code. A sequence of RTL character should be written Right to left.
Take look at this: https://stackoverflow.com/a/14824756/1999969

The RTL information mainly comes from the character code, as @kisvegabor mentioned, but:

Unicode has some special characters to explicitly set RTL and LTR for some special cases, so it can also come from the string itself. (We don't have to support this in my opinion).
The label itself has RTL "information" in the form of its base direction. The same string is displayed differently on two labels with different base directions.

Also, be aware that the isRTL algorithm on the link you provided is wrong.
For example, it does not take into account code points with "neutral" directionality. So an RTL string that terminates with a period would be considered LTR although its RTL.

Do you plan to implement a full RTL support in the future?

Let's consider to current no-line-wrap-support approach as an experiment to get more familiar with the BiDi in general. And we will see what to do with line-wrapping.

Perhaps with features that are needed for Arabic scripts as well?

The problem with supporting Arabic is the advanced text shaping which requires info from the TTF support. I'd say it's out of scope for now.

If you do, maybe we can think of an API that would be forward compatible. For now it would only implement the simplistic string processing, but in the future the same API could be used for full RTL support.

To ensure forward compatibility  RTL processing should be hidden in lv_label_set_text. If the  LV_TXT_RTL_SUPPORT is enabled in lv_conf.h, lv_label_set_text() can automatically do the required processing.

Unicode has some special characters to explicitly set RTL and LTR for some special cases, so it can also come from the string itself. (We don't have to support this in my opinion).

I've read some BiDi examples and find that in some cases algorithms are not able to correctly process the BiDi texts, therefore, special control characters are required. At first look, RTL or LTR isolation means only adding non-printable strong  RTL or RTL characters. (Not sure if it's really that simple though.)

The label itself has RTL "information" in the form of its base direction. The same string is displayed differently on two labels with different base directions.

I have an idea for that. See below.

For example, it does not take into account code points with "neutral" directionality. So an RTL string that terminates with a period would be considered LTR although its RTL.

I haven't seen such a rule so far. Can you link some docs to read about it?

I summarize what I know so far and what are my approaches.

There are strong (usually letters), weak (usually arithmetic symbols) and neutral (usually white spaces) characters: See this.
The text is split into RTL and LTR runs.
One run lasts until a strong char is not found with a different direction.
Runs of strong RTL characters should be displayed as RTL. Similar for LTR.
Neutral chars in strong RTL runs should be displayed as they are. (E.g. space between words). Similar for LTR.
"Symmetric"  weak characters (such as ()[]{}<>) should be changed their pair in RTL runs.
European numbers (e.g. 456) should be displayed as LTR in RTL runs too.
The meaning of LTR/RTL base direction is to determine the display order of runs.

Idea: we can use a control character as the first character of the text to determine the base direction. This way the whole BiDi processing can be hidden in lv_label_set_text.

To simply sharing examples I suggest considering uppercase ASCII letters as strong LTR and lower case letters as strong RTL.
Here I've listed some use cases according to my best understanding and described MEM (as in the memory, as they should be read), LTR (LTR base direction), RTL (RTL base direction).
Please verify or correct them.
MEM: "abc ABC"
LTR: "cba ABC"
RTL: "ABC cba"

MEM: "ABC abc"
LTR: "ABC cba"
RTL: "cba ABC"

MEM: "abc def ABC DEF"
LTR: "fed cba ABC DEF"
RTL: "ABC DEF fed cba"

MEM: "abc123 def  ABC123 EFG"
LTR: "fed 123cba ABC123 EFG"
RTL: "ABC123 EFG fed 123cba"

MEM: "abc (def) 123 < 456 ABC"
LTR: "456 > 123 (fed) cba ABC"
RTL: "ABC 456 > 123 (fed) cba "

MEM: "I'VE READ war and peace 25 TIMES"
LTR: "I'VE READ  peace dna raw  25 TIMES"
RTL: "TIMES 25 peace dna raw I'VE READ"

Also, be aware that the isRTL algorithm on the link you provided is wrong.
For example, it does not take into account code points with "neutral" directionality. So an RTL string that terminates with a period would be considered LTR although its RTL.

I haven't seen such a rule so far. Can you link some docs to read about it?

@kisvegabor
I just tried out the playground page provided on that SO post and noticed that when I type RTL string and add a period, which is a neutral character, the string is switched to LTR although it should not:

Neutral chars in strong RTL runs should be displayed as they are. (E.g. space between words). Similar for LTR.

I think this is wrong. Neutral chars should inherit the directionality of current run. This is especially important for punctuation symbols (,.;:!? etc.) so they are positioned correctly.
Take for example the exclamation mark - it must be positioned after a word. So
סימן! קריאה

inherits RTL because it's inside an RTL run, whereas -
Exclamation! mark

inherits LTR.
An interesting case is the start/ending of a string. Where would the exclamation mark go in such case?
This depends on the base direction of the component.
For example, when the component is LTR:
סימן קריאה!

This would seem wrong to a Hebrew speaker. To fix this, the base direction of the component itself should be set to RTL in order to position the exclamation mark on the correct (left) position.
After setting base direction to RTL  it would look like this:

The meaning of LTR/RTL base direction is to determine the display order of runs.

I would say it determines the "default" direction, and especially affects the characters at the beginning/end of the string. If characters at the beginning/end of the string are neutral, they will be considered as surrounded by the base direction from one side.

Idea: we can use a control character as the first character of the text to determine the base direction. This way the whole BiDi processing can be hidden in lv_label_set_text.

I don't like this idea.
It means that every string would need to be changed before calling set_text to add this control character.
This special character would only need to be added when setting text of RTL controls, so I would need to add it depending on which control I plan to set this text on.
I my opinion, it's better to separate the data (string logical content) and view (string visual display parameters).
I have a different suggestion: Can you spare one bit on lv_obj_t struct? (only when LV_TXT_RTL_SUPPORT is defined of course)
Let's say every object can be set to be either LTR or RTL. When creating a child object, this bit would be automatically set according to the parent, and can also be set by the user with a set function. So it's very flexible - you can set the whole screen as RTL before you add child objects, or you can set a page, or a single component if you want.

Regarding your examples - I've added Hebrew letters that correspond to your examples.
It really helps to validate your examples. I type each string according to your MEM and see what is displayed on the browser text box. With a right-click I can even change the base direction from LTR to RTL and see how it looks in each case.
On those cases I saw something different I marked the places ("WRONG") where I think it's wrong, and added ("CORRECT") what it should be instead. If I didn't specify "WRONG" or "CORRECT" then this is your original example and it's correct
"אבג ABC"
MEM: "abc ABC"
LTR: "cba ABC"
RTL: "ABC cba"

"ABC אבג"
MEM: "ABC abc"
LTR: "ABC cba"
RTL: "cba ABC"

"אבג דהו ABC DEF"
MEM: "abc def ABC DEF"
LTR: "fed cba ABC DEF"
RTL: "ABC DEF fed cba"

"אבג123 דהו ABC123 EFG"
MEM: "abc123 def  ABC123 EFG"
LTR: "fed 123cba ABC123 EFG"
RTL: "ABC123 EFG fed 123cba"

"אבג (דהו) 123 < 456 ABC"
--> WRONG MEM: "abc (def) 123 < 456 ABC"
--> CORRECT MEM: "abc )def( 123 > 456 ABC"
LTR: "456 > 123 (fed) cba ABC"
RTL: "ABC 456 > 123 (fed) cba "

"I'V READ מלחמה ושלום 25 TIMES"
MEM: "I'VE READ war and peace 25 TIMES"
--> WRONG: LTR: "I'VE READ  peace dna raw  25 TIMES"
--> CORRECT: LTR: "I'VE READ 25 ecaep dna raw TIMES"
--> WRONG: RTL: "TIMES 25 peace dna raw I'VE READ"
--> CORRECT: RTL: "TIMES 25 ecaeo dna raw I'VE READ"

@amirgon The last line of your post has TIMES 25 ecaeo dna raw I'VE READ instead of TIMES 25 ecaep dna raw I'VE READ (note the "p" in peace). Is that a mistake?

@amirgon The last line of your post has TIMES 25 ecaeo dna raw I'VE READ instead of TIMES 25 ecaep dna raw I'VE READ (note the "p" in peace). Is that a mistake?

Of course this is a mistake. Fixed above.

@amirgon Thank you for the corrections.

I just tried out the playground page provided on that SO post and noticed that when I type RTL string and add a period, which is a neutral character, the string is switched to LTR although it should not:

It seems like a bug to me.

I think this is wrong. Neutral chars should inherit the directionality of current run. This is especially important for punctuation symbols (,.;:!? etc.) so they are positioned correctly.

True!

An interesting case is the start/ending of a string. Where would the exclamation mark go in such case?
This depends on the base direction of the component.

Can't the staring weak/neutral chars inherit the direction of the first strong char? It makes more sense to me because they are related to it anyway. E.g with RTL base dir:
intention: "! exclamation"
"! " get base dir:  " !exclamation"
"! " get the dir of first strong char: "! exclamation"

In my opinion, it's better to separate the data (string logical content) and view (string visual display parameters).

I've found that in the practice you can't separate data from view even in CSS/HTML. You need to manually markup parts of the text because sometimes the BiDi algorithm does not give the expected result.
A typical issue with the algorithm is shown here. So it's inevitable to deal with markups in the text. In the most simple case hidden strong chars can be used to isolate/split runs.
Regarding the base direction, once control chars need to be used anyway, I don't find a control char for base dir so evil.
We can even autodetect base dir based on the first strong char which can be overwritten by a control character.
I found this short and straightforward write-up. From this the most essential rules are (we have already mentioned some of them):

A directions run lasts until a strong char with different direction is not found.
Neutrals between the same dir strong chars are treated as the dir of the strong chars.
Neutrals between two different dir strongs inherit the base dir.
Numbers (on any language) are always displayed as LTR.

An experiemnt about how a "space" behaves.
MEM: "ABC abc"
LTR base dir:
  - Run1 (LTR): "ABC " (space comes here)
  - Run2 (RTL): "abc" -> swap "cba"
  - Result: "ABC cba"

RTL base dir:
  - Run1 (LTR): "ABC"
  - Run2 (RTL): " abc" -> swap "cba "   (space comes here)
  - Result: "cba ABC"

A question regarding the alignment. Shall we automatically align texts with RTL base dir to the right?

I've added my first attempt to implement a "get runs" algorithm to feat-bidi branch.
Now it just prints the found runs. It can be tested like the following. (Still lowercase is considered as RTL):
 const char * s1 = "abc123 ABC123 aAbbBB";
    char s2[32];
    lv_txt_rtl_proc(s1, s2, LV_TXT_DIR_RTL);  //Last param is the base dir

@kisvegabor

Can't the staring weak/neutral chars inherit the direction of the first strong char?

I'm not an expert, but I don't think this is how it's supposed to work.
It's very easy to make empiric tests and see how other applications (PowerPoint, the web browser) behave.
Here, for example, I tried it on LibreOffice.
First paragraph was set as LTR, and the second as RTL:

As you can see, the period and the colon position is not dependent, as you suggest, on the direction of the first strong char, but on the base direction of the paragraph.
You'll get the same behavior on other applications, and even on HTML elements (where you set the base direction with the dir attribute to rtl or ltr)
On HTML5 you have the option to use dir="auto", but it's only recommended if the text direction is unknown, which is not the case, at least for my application and I think in most other cases.
By the way, HTML5 dir="auto" attribute works as you suggested:

HTML5 added an auto value to the dir attribute that can help here. The auto value tells the browser to look at the first strongly typed character in the element and work out from that what the base direction of the element should be. If it's a Hebrew (or Arabic, etc.) character, the element will get a direction of rtl. If it's, say, a Latin character, the direction will be ltr.

There are some corner cases where this may not give the desired outcome, but in the vast majority of cases it should produce the expected result.

I suggest providing a way to mark the base direction of an element, and not always default to this method of automatically detecting the direction.

I've found that in the practice you can't separate data from view even in CSS/HTML. You need to manually markup parts of the text because sometimes the BiDi algorithm does not give the expected result.

I think it's very rare case that a string is specially modified (by adding special RTL/LTR marks in it) in order to display a string correctly.
I think that in most cases, the solution to the isolation problem you present is to use two labels (or text boxes etc.) each one with the correct base direction.
I don't agree with your statement that "it's inevitable to deal with markups in the text."
Instead of markup in the text (the data), the layout should have different components (view) for isolated parts.

Regarding the base direction, once control chars need to be used anyway, I don't find a control char for base dir so evil.

I think that in practice - control chars are not used.
Instead, base dir is set on the component itself.
The reason is that most developers don't have much understanding about RTL/LTR issues, so they don't know how and where to insert the control chars.  Instead, they solve the problem on the view by selecting the correct base direction for each component.

We can even autodetect base dir based on the first strong char which can be overwritten by a control character.

The disadvantage of autodetecting the base dir is that it's not always correct.
A common example is a string that starts with a number. For example "2 Apples".
If you detect base direction automatically, you would detect this string as LTR, and the result would be "Apples 2", which is wrong.
In most cases we know, in advance, that the label is RTL and supposed to show Hebrew sentence. Auto-detecting this as LTR would yield the wrong result. Why prevent us from configuring this explicitly?

I found this short and straightforward write-up. From this the most essential rules are (we have already mentioned some of them):

A directions run lasts until a strong char with different direction is not found.
Neutrals between the same dir strong chars are treated as the dir of the strong chars.
Neutrals between two different dir strongs inherit the base dir.
Numbers (on any language) are always displayed as LTR.

These rules (and the write-up) look good to me!

A question regarding the alignment. Shall we automatically align texts with RTL base dir to the right?

That would be very helpful.
Probably every component with RTL base direction should be aligned to the right by default.

I've added my first attempt to implement a "get runs" algorithm to feat-bidi branch.

https://github.com/littlevgl/lvgl/blob/2c5c4abdcedfd527d0072c7b37ed9fde6cd8a369/src/lv_misc/lv_txt.c#L352
I assume that in the future RTL/LTR indication would be taken from the font data itself? Or did you think to use some hard-coded ranges for RTL unicode characters?
https://github.com/littlevgl/lvgl/blob/2c5c4abdcedfd527d0072c7b37ed9fde6cd8a369/src/lv_misc/lv_txt.c#L356-L364
I think that punctuation marks are neutral, not weak.
For the testing I've added the following:
static const char neutrals[] = " \t\n\r.,:";

What about marks that should be reversed? (< :arrow_right: > for example). Maybe they should be defined in a class of their own?

Now it just prints the found runs. It can be tested like the following. (Still lowercase is considered as RTL)

The easiest way for me to run it is with Micropython!
We can very easily compile this on the js port and have an online web sandbox for trying out your code!
For now, I'm running it on the unix port:
>>> lv.txt_rtl_proc("abc123 ABC123 aAbbBB", s, lv.TXT_DIR.LTR)
RTL: "abc123"
LTR: " ABC123 "
RTL: "a"
LTR: "A"
RTL: "bb"
LTR: "BB"
True

What about this? (I've added . to neutrals):
>>> lv.txt_rtl_proc("a.", s, lv.TXT_DIR.LTR)
RTL: "a."
True
>>> lv.txt_rtl_proc("a.", s, lv.TXT_DIR.RTL)
RTL: "a."
True

In this case I would expect the base direction to affect the position of the period.
Shouldn't the period be on a different run?

The easiest way for me to run it is with Micropython!
We can very easily compile this on the js port and have an online web sandbox for trying out your code!

Let me know whenever you want the JS port recompiled and I will rebuild and push it, if it saves you time.

Let me know whenever you want the JS port recompiled and I will rebuild and push it, if it saves you time.

Thanks @embeddedt.
I'm ok with the unix port for now, but perhaps @kisvegabor would want this at some point.

Meanwhile, I've added rtl_reverse and moved the bidi related things to lv_misc/lv_bidi.c/h. The API has been changed accordingly:
    const char * s1 = "abc123 ABC";
    char s2[32];
    lv_bidi_process(s1, s2, LV_BIDI_DIR_RTL);

RTL: "abc123 " -> " 123cba"
LTR: "ABC"

@kisvegabor

Add set_base_dir API object's which have texts.
Add an lv_obj_t attribute (as you suggested). It should have 3 possible values: RTL, LTR, NONE (means inherit from the parent).

Why not both? Why 3 possibilities?
How about always inheriting from parent, and in addition provide set_base_dir so the user could call  it with an object and change its the direction?
Practically, in most cases, users would want to set a single direction to the whole application, or to an entire page in a consistent way, so I guess set_base_dir would usually be used on screen or page objects, and very rarely on specific label.
Having only two options (LTR, RTL) saves a little because it would consume only one bit on lv_obj_t.
I think it would be a good idea to set the initial screen direction according to a configuration option on lv_conf.h, since it's the parent of all other objects and therefore would be the initial (default) direction of all objects.
For example, LV_LABEL_ALIGN_LTR means that every object that is created without a parent would be set to LTR.

We can introduce LV_LABEL_ALIGN_AUTO. It can be the default value which sets the alignment according to the base dir.

If you want to support automatic alignment, we really need 3 possibilities (LTR, RTL, AUTO).
But another option is to support this as a global configuration option on lv_conf.h:   If LV_LABEL_ALIGN_AUTO is defined, we can avoid adding the direction bit in lv_obj_t and always set it automatically.
I assume this could be a default for users who don't want to build an "RTL" layout. (When designing RTL application, it's not just the directionality of the text, it's also how components are arranged and aligned on screen one vs. the other)

There is no such data in the fonts right now. I meant to use hard-coded ranges. Do you see any problem with that?

No special problem, I think. There are relatively few ranges if we only consider Hebrew.
On the general case, however, there are many ranges (See ISO 15924 and also this, but I don't think we need to support the general case).
Maybe those ranges could be configurable on lv_conf.h?

I think that punctuation marks are neutral, not weak.

I've found this here:

The bidirectional types associated with numbers are called weak types, and characters of those types are called weak directional characters. With the exception of the directional formatting characters, the remaining bidirectional types and characters are called neutral.

Here I found this comment:

Characters are usually classified as 'weak' when they are associated with numbers. A small number of punctuation characters are initially classed as weak, but in a non-numeric context are treated like neutrals. In consequence, in this article we will refer to all punctuation as neutral characters.

So it seems that some punctuation marks are sometimes considered as weak and some other times as neutrals...
Regarding weak characters, weak characters are discussed here:

Because it is weakly typed, the number is seen as part of the preceding Arabic text, so the two Arabic words that surround the number are treated as part of the same directional run - even though the sequence of digits runs LTR on screen.

If I understand correctly, the sequence of numbers are considered LTR but the surrounding RTL characters are considered a single run, so their order is not switched.
Here is an example:
"אחת 123 שתיים"
"אחת abc שתיים"

So the order of אחת and שתיים depends on whether between them there's a weak run 123 or a strong run abc. But both weak and strong runs are LTR, so their internal order is reversed (as opposed to neutral, where the order is kept and everything would be the same RTL run).
An example of a number-related character is $.
For example:
אחת 100$ שתיים
אחת 100! שתיים

I think that what's happening here is that the $ behaves as weak character, and is presented LTR, while the ! is neutral and presented RTL. (in both cases, the logical "MEM" order is: 1 0 0 !/$

Meanwhile, I've added rtl_reverse and moved the bidi related things to lv_misc/lv_bidi.c/h.

Maybe it's time to generate a font with Hebrew characters... I never created a font yet.
A few questions:

Does the font generation tool work on Linux, or do I need a Window machine?
Can I load a font on runtime and provide a pointer to it (that is preferred for Micropython), or do I need to build the project with the font embedded in the program?

I think we are on the same page with base_dir and align but maybe I wasn't exact enough. So here is my approach
lv_obj_set_base_dir
Sets the base direction of the writing. Has no effect on the label's position.
Can have 4 values:

INHERIT: default value
LTR
RTL
AUTO: depends on the first strong char. I've realized it is useful when the user can change between the languages runtime and you don't know the base dir of the objects in advance, however it's usually clear from the text.

You can call lv_obj_set_base_dir(page/screen/label) and the clidren will inherit it.
lv_label_set_align
Important only with multiline text. E.g. ddlist's options are a long label with multiple lines.

LEFT
RIGHT
CENTER
AUTO: default. Turns to LEFT or RIGHT depending on base dir.

So the order of אחת and שתיים depends on whether between them there's a weak run 123 or a strong run abc. But both weak and strong runs are LTR, so their internal order is reversed (as opposed to neutral, where the order is kept and everything would be the same RTL run).

The last version should work like this.

I think that what's happening here is that the $ behaves as weak character, and is presented LTR, while the ! is neutral and presented RTL. (in both cases, the logical "MEM" order is: 1 0 0 !/$

I can extend the algorithm as once it's in an "internal weak run of numbers" consider $ as numbers. Maybe . and , should be treated similarly as well.

Does the font generation tool work on Linux, or do I need a Window machine?

Platform independent, but fortunately easier on Linux. However the easiest thing is to use the Online converter.

Can I load a font on runtime and provide a pointer to it (that is preferred for Micropython), or do I need to build the project with the font embedded in the program?

If you mean similarly to "bin" images it's not working for fonts. You need to generate C files and build with the project :frowning_face: It was already asked several times so probably it will be added once but needs some time to do.

@kisvegabor

I think we are on the same page with base_dir and align but maybe I wasn't exact enough. So here is my approach

Yes, your approach looks good! :+1:

lv_label_set_align
Important only with multiline text

Could you explain this?  How is multiline related? Why would I want to set align on multiline and not on single line?

If you mean similarly to "bin" images it's not working for fonts. You need to generate C files and build with the project ☹ It was already asked several times so probably it will be added once but needs some time to do.

That's unfortunate. It means that a Micropython would need to be built with specific fonts in it, instead of building it only once, and using the same firmware with whatever fonts we need.
It also means we cannot use the online lvgl simulator to visualize things related to fonts, RTL among them.
What is missing in order to support this? I don't really need lv_fs to load it, I can handle loading it myself, I just need a way to register a new font on runtime by providing a pointer to a binary font file.
Is it the creation of the binary font file (instead of C file) what is missing?

Could you explain this? How is multiline related? Why would I want to set align on multiline and not on single line?

Let's say there label with "Apple" text. The width is automatically set to e.g. 100 px according to the width of the text. As the width is the same as the text's width, the text can be aligned. However, a text like "Apple\nB" also has width = 100px but the lines can be aligned:
Left:
Apple
B

Center:
Apple
  B

Right:
Apple
    B

It's true if the label has LV_LABEL_LONG_MODE_EXPAND. E.g. in case of LV_LABEL_LONG_BREAK if you specify the width even single lines can be aligned.

What is missing in order to support this? I don't really need lv_fs to load it, I can handle loading it myself, I just need a way to register a new font on runtime by providing a pointer to a binary font file.
Is it the creation of the binary font file (instead of C file) what is missing?

The font converter (if run from the command line) can produce bin output (--format bin). The details of the binary format are described here.
The task is to write two interface functions to get glyph info and bitmap. Similarly to the interface functions of the C format.
Then an lv_font_t variable needs to be initialized with the binary data and the interface functions. E.g. lv_font_roboto_16.
The C format is essentially the same as the binary format just represented as structs instead of a blob.

@kisvegabor
I tried a basic "Hello World" test with a real Hebrew font, and I have a few problems.
First, I changed the RTL range on lv_bidi_letter_is_rtl to Hebrew script range:
 bool lv_bidi_letter_is_rtl(uint32_t letter)
 {
-    if(letter >= 'a' && letter <= 'z') return true;
+    if(letter >= 0x5d0 && letter <= 0x5ea) return true;

     return false;
 }

I created a new font:
node lv_font_conv.js --font /usr/share/fonts/truetype/culmus/FrankRuehlCLM-Medium.ttf -r 0x20-0x7F -r 0x5d0-0x5ea --size 16 --format lvgl --bpp 4 --no-compress -o ~/esp/projects/lv_mpy/lib/lv_bindings/lvgl/src/lv_font/lv_font_heb_16.c

And added it on lv_conf.h with
#define LV_FONT_CUSTOM_DECLARE LV_FONT_DECLARE(lv_font_heb_16)

Then I ran the following:
scr = lv.obj()
btn = lv.btn(scr)
btn.align(lv.scr_act(), lv.ALIGN.CENTER, 0, 0)
label = lv.label(btn)
s = lv.style_t()
lv.style_copy(s, label.get_style(label.STYLE.MAIN))
s.text.font = lv.font_heb_16
label.set_style(label.STYLE.MAIN, s)
text = u'\u05E9\u05DC\u05D5\u05DD\u0020\u05E2\u05D5\u05DC\u05DD\u0021'
rtl_text = u'.' * (len(text))
lv.bidi_process(text,rtl_text,lv.BIDI_DIR.RTL)
print("text: " + text +"\nrtl_text: " + rtl_text)
label.set_text(rtl_text)
lv.scr_load(scr)

And this is what I get:

And I get this on the console:
RTL: "�"
LTR: "�"
RTL: "לום עולם!" -> "!�ם�ל�ו�ע� �ם�ו�ל"
text: שלום עולם!
rtl_text: !�ם�ל�ו

The original text was "שלום עולם!" which translates to "Hello World!"

Why was the first character gone? (the ש)
I'm not sure yet if that's lvgl-bidi issue or Micropython issue.
How do I know the expected string length? Can I assume the length would be the same as the original? (that's true if all you do is reverse sub-strings)
I've noticed that if I enable compression (remove --no-compress) the font looks garbled.
Is compression supported in lvgl?
In not - do you plan supporting it in the future?

I've noticed that if I enable compression (remove --no-compress) the font looks garbled.
Is compression supported in lvgl?

Compression is supported in dev-6.1, but not in master.

@amirgon
I've updated the algorithm. Now it prints the strings correctly.
Note the I still don't compose the whole string from the runs. Therefore the output buffer of lv_bidi_process contains only the result from the last run.

@amirgon
I've updated the algorithm. Now it prints the strings correctly.
Note the I still don't compose the whole string from the runs. Therefore the output buffer of lv_bidi_process contains only the result from the last run.

@kisvegabor Please let me know when you compose the whole string, it would make is easier for me to check your algorithm.
Here are some problems I see now (all conversions are to RTL):

u'\u05d0\u05d7\u05ea' yields: RTL: "אחת" -> "ת"
Only one letter instead of the whole word
u'\u05d0\u05d7\u05ea \u0031' yields one run: RTL: "אחת 1" -> "1 ת חא"
There's an extra space in the word "אחת".
u'\u05d0\u05dc\u05e3 \u0061 \u05d1\u05d9\u05ea' yields:

RTL: "אלף " -> " ף לא"
LTR: "a"
RTL: " בית" -> "ת"

u\u0030\u002e\u0031 yields RTL: "0.1" -> "1.0"
I would expect it to remain "0.1" because "." in that context should behave like a weak character.

Have you used the latest version? If so please add the full printf-ed output. It should contain the input string too. It'd be easier for me to reproduce because I can't write u05e9 in C as it's already Unicode and not UTF-8.
I've added support to currencies:
lv_bidi_process("שלום $100 עולם!",  buf, LV_BIDI_DIR_RTL);

> Input str: "שלום $100 עולם!"
> RTL: "שלום $100 עולם!" -> "!םלוע $100 םולש"

lv_bidi_process("שלום 1.0 עולם!",  buf, LV_BIDI_DIR_RTL);

> Input str: "שלום 1.0 עולם!"
> RTL: "שלום 1.0 עולם!" -> "!םלוע 1.0 םולש"

It's interesting how the Terminal of Eclipse changes -> to <-. Somehow its autodetects the base dir.
Character replacement (e.g < to >) still missing too. I try to add it and composing today or at worse case Monday.

@amirgon
I've updated the code with

composing final string
swap arithmetical symbols (< -> >)

@amirgon
I've updated the code with

composing final string
swap arithmetical symbols (< -> >)

@kisvegabor I tried it out and it looks good!
I didn't find any problems so far.
Good job! :100:

I tried it out and it looks good!
I didn't find any problems so far.

Awesome, thank you for the feedback! :relaxed:
I'll continue with the integration to the library:

LV_LABEL_ALIGN_AUTO
LV_OBJ_BASE_DIR_LTR/RTL/INHERIT/AUTO

I've implemented LV_LABEL_ALIGN_AUTO and LV_BIDI_DIR_AUTO/INHERIT.
However, I'm a little bit puzzled. By default LV_LABEL_ALIGN_AUTO is set and LV_BIDI_BASE_DIR_DEF sets the default base dir in lv_conf.h (AUTO by default).
With labels all clear:
    lv_obj_t * label = lv_label_create(lv_scr_act(), NULL);
    lv_label_set_text(label, "hello שלום $200 > $100 עולם! world");
    lv_label_set_long_mode(label, LV_LABEL_LONG_SROLL);
    lv_label_set_body_draw(label, true);
    lv_label_set_style(label, LV_LABEL_STYLE_MAIN, &lv_style_plain_color);
    lv_obj_set_pos(label, 20, 20);
    lv_obj_set_width(label, 300);
    lv_obj_set_base_dir(lv_scr_act(), LV_BIDI_DIR_LTR);

    lv_obj_set_base_dir(lv_scr_act(), LV_BIDI_DIR_RTL);

It becomes more tricky with a ddlist:
    lv_obj_t * ddlist = lv_ddlist_create(lv_scr_act(), NULL);
    lv_ddlist_set_options(ddlist, "1 שלוםםםם\n2 שלום\n3 שלום");
    lv_obj_set_pos(ddlist, 20, 20);
    lv_ddlist_set_fix_width(ddlist, 150);
    lv_obj_set_base_dir(lv_scr_act(), LV_BIDI_DIR_RTL);

Two things:

I can't decide if numbers are on the correct end or not. Can you confirm it?
The text is forced to be multi-lined with \n but it is still treated as a single run. And as it's an RTL run the characters are reversed. So options' order is reversed too. Shall we somehow terminate on \ns?

@kisvegabor

I can't decide if numbers are on the correct end or not. Can you confirm it?

Yes, numbers should appear on the right on RTL text.
For example "1. שלום \n2. עולם" should yield:

If the same text is displayed on LTR, it would yield:

שלום
עולם

2. The text is forced to be multi-lined with \n but it is still treated as a single run. And as it's an RTL run the characters are reversed. So options' order is reversed too. Shall we somehow terminate on \ns?

Let's examine the Unicode Bidi Algorithm specification :
On section 2:

Three types of explicit directional formatting characters are used to modify the standard implicit Unicode Bidirectional Algorithm (UBA). In addition, there are implicit directional formatting characters, the right-to-left and left-to-right marks. The effects of all of these formatting characters are limited to the current paragraph; thus, they are terminated by a paragraph separator.

On section 3:

3 Basic Display Algorithm
The Unicode Bidirectional Algorithm (UBA) takes a stream of text as input and proceeds in four main phases:

Separation into paragraphs. The rest of the algorithm is applied separately to the text within each paragraph.
...

The algorithm reorders text only within a paragraph; characters in one paragraph have no effect on characters in a different paragraph. Paragraphs are divided by the Paragraph Separator or appropriate Newline Function (for guidelines on the handling of CR, LF, and CRLF, see Section 4.4, Directionality, and Section 5.8, Newline Guidelines of [Unicode]).

On section 3.3.1

P1. Split the text into separate paragraphs. A paragraph separator (type B) is kept with the previous paragraph. Within each paragraph, apply all the other rules of this algorithm.
P2. In each paragraph, find the first character of type L, AL, or R while skipping over any characters between an isolate initiator and its matching PDI or, if it has no matching PDI, the end of the paragraph.

Note that:

Because paragraph separators delimit text in this algorithm, the character found by this rule will generally be the first strong character after a paragraph separator or at the very beginning of the text.

On section 3.3 the first thing that is done is apply rule P1 to split the text into paragraphs, before any other rule.
So - it looks like the first step is to separate a string into paragraphs, where each paragraph is delimited by a newline character (CR, LF and some others).
Only after you do that, you can apply the rest of the algorithm, detect runs etc.

RTL
    lv_obj_set_base_dir(lv_scr_act(), LV_BIDI_DIR_LTR);

@kisvegabor You meant lv_obj_set_base_dir(lv_scr_act(), LV_BIDI_DIR_RTL);, right?

@kisvegabor What about the alignment of the objects themselves? (vs. their containing object)
In this case, the ddlist is aligned to the left on the screen, even though I've set scr base dir to RTL.
Do you think it's possible to change the default alignment in such case so that the object itself would be aligned to the right, instead of to the left?

@amirgon By default, the active screen is a plain lv_obj, which does not control the position of its children.
Are you referring to how an lv_cont would align its children (that object is used for pages as well)?

Are you referring to how an lv_cont would align its children (that object is used for pages as well)?

@embeddedt I was referring to default child object alignment in general, including both lv_cont and screen.
I thought there was a default alignment for every child object in a parent object.
What causes scr children to be aligned to the left, if plain scr does not control child position?

What about the default alignment of the objects themselves? (vs. their containing object)

By default, the children are created on the parent's (0;0) coordinate. It'd be possible to create them on (parent_widht - child_width; 0). The problem is if you set the width of the child it will change the x2 coordinate so it will "grow to the right". Therefore it will be out of the parent when width gets greater (or will cause a gap if becomes smaller). Although at first look, it seems possible to set x1 on width change to "grow grow the left".

@amirgon I've added the "grow to the left" and right default position. Can you test it?

@amirgon I've added the "grow to the left" and right default position. Can you test it?

@kisvegabor ddlist seems to grow correctly to the left when resizing on RTL (and to the right on LTR). I'm not sure how it's supposed to grow on the other options of LV_BIDI_DIR.
I've noticed one problem: When ddlist is in tabview and is aligned to the right with default RTL settings, it overflows a little and causes a scrollbar to appear (ddlist parent is a tab):

This does not happen when scr base dir is LTR and  it's aligned by default to the left:

Maybe related to some padding of the containing tab? If so, such padding need to be taken in account when aligning to the right.

However, other objects (table, tabview, text area, checkbox) don't respond well to RTL:

I would expect (when base dir is RTL):

Strings processed by Bidi and reversed (on Table and Tabs they are not)
Text in table cells aligned to the right by default
Tabs ordered from right to left (first tab should be on the right)
Text area cursor on the right
Checkbox on the right of the text, not on the left

Thanks for pointing them out.
I've fixed most of the issues:

Strings processed by Bidi and reversed (on Table and Tabs they are not)

On "tabs"  was an issue of btnm (and therefore kb) too. Now all these should work.

Text in table cells aligned to the right by default

Now set the default alignment according to the object's base dir.

Text area cursor on the right

Also fixed.
What is not fixed yet:

Checkbox on the right of the text, not on the left

It'd require to develop new layouts. Currently LV_LAYOUT_ROW_M is used which puts the objects from left to right. I'm planning to add new layouts later but it's out of the scope now.
Due to this, an issue occurs with lists too (the order of icon and text)

Tabs ordered from right to left (first tab should be on the right)

It can be fixed, just I haven't done it yet. :slightly_smiling_face:

I've fixed most of the issues:

@kisvegabor Thanks for working on these issues!
There are, however, problems with your fixes and some new problems I've found.

The elements on the table are indeed aligned to the right now.
But there are 2 problems:

The text itself is reversed as if the bidi algorithm did not run.
For example, the header should be "שם | מחיר" but it shows "ריחם | םש" which is reversed.
The columns should be numbered from the right, same as tabs, where the first column is the right-most one. In this case the שם should be the first (on the right) and מחיר second (on the left).

This problem was not fixed - the ddlist overflows and a scrollbar appears when the ddlist is on a tab. See my last post for more details.

This is the result of calling ta.add_text("1. שלום עולם!") .
There are two problems:

Bidi algorithm did not handle (reverse) the text as expected. Bidi should process the text incrementally with successive calls to add_text and add_char.
Cursor position should be on the left and not remain on the right.

There is a problem with long labels with breaks.
For example:

This is a page with a label:
page = lv.page(tab3)
page.set_size(150,200)
label = lv.label(page)
label.set_long_mode(lv.label.LONG.BREAK)
label.set_width(page.get_fit_width())
label.set_text("א ב ג ד ה ו ז ח ט י כ ל מ נ ס ע פ צ ק ר ש ת")

There are two problems:

The long text is broke "in reverse". The first line should be the last. The expected result should be:

א ב ג ד ה ו ז ח ט י כ ל
 מ נ ס ע פ צ ק ר ש ת

A letter ל is missing (actually you can see a few pixels of it), because it is drawn outside the page boundaries and truncated. It should appear after כ and before מ, exactly before/after the line break

When setting a long text string to a label, lvgl crashes due to stack smashing on process_paragraph function.
To reproduce, just enter a long text (814 characters in the crash example below)
*** stack smashing detected ***: /home/amirgon/esp/projects/lv_mpy/ports/unix/micropython terminated

Thread 1 "micropython" received signal SIGABRT, Aborted.
0x00007ffff6dfd428 in __GI_raise (sig=sig@entry=6)
    at ../sysdeps/unix/sysv/linux/raise.c:54
54	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff6dfd428 in __GI_raise (sig=sig@entry=6)
    at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff6dff02a in __GI_abort () at abort.c:89
#2  0x00007ffff6e3f7ea in __libc_message (do_abort=do_abort@entry=1,
    fmt=fmt@entry=0x7ffff6f5749f "*** %s ***: %s terminated\n")
    at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff6ee115c in __GI___fortify_fail (msg=<optimized out>,
    msg@entry=0x7ffff6f57481 "stack smashing detected")
    at fortify_fail.c:37
#4  0x00007ffff6ee1100 in __stack_chk_fail () at stack_chk_fail.c:28
#5  0x00000000004948f5 in process_paragraph (
    str_in=0x7ffff15c9000 "קרול הראשון, מלך רומניה (20 באפריל 1839 - 10 באוקטובר 1914), היה קצין גרמני ובנו השני של הנסיך הגרמני קרל אנטון. קרול נב"...,
    str_out=0x7ffff15c8a40 ".זכרמה תומצעמל ףרטצהל הינמור לעש ותדמע תא ולביק אלש ךכמ ותבזכאל רשוקמ ,הנושארה םלועה תמחלמ ץורפ רחאל ,ותומ .םינ"..., len=1427, base_dir=1 '\001')
    at ../../lib/lv_bindings/lvgl/src/lv_misc/lv_bidi.c:210

In my opinion, if doing all this effort, it's worth fixing the btnm and checkbox/list RTL layout issue you mentioned on your previous post.

Thanks for the feedback!

The text itself is reversed as if the bidi algorithm did not run.
For example, the header should be "שם | מחיר" but it shows "ריחם | םש" which is reversed.

I'll fix it soon.

The columns should be numbered from the right, same as tabs, where the first column is the right-most one. In this case the שם should be the first (on the right) and מחיר second (on the left).

As you already mentioned it affects the tabview's buttons order too. The elements of the button matrix and keyboards also should be shown in RTL?
How important it is to support it? If it's not critical, I'd skip it from this first iteration.

This is the result of calling ta.add_text("1. שלום עולם!") .

I'll look into it.

There is a problem with long labels with breaks.

It would be quite tricky to support it. As line break algorithms (and all text processing) needed to run the bidi process algorithm to know how/where to make line breaks. (Now I simply store the reversed text.) It seems too much work compared to the supposed benefit. It would be huge rework in all text related algorithms for only those users who need RTL and Text area with line break support.

When setting a long text string to a label, lvgl crashes due to stack smashing on process_paragraph function.

There was a  local buffer with 256 bytes to print debug info. I removed it, now it should work.

In my opinion, if doing all this effort, it's worth fixing the btnm and checkbox/list RTL layout issue you mentioned on your previous post.

I found a  good way to solve the layout issue. If an object has RTL base dir then "ROW" layouts work from Right to Left. It should solve the issues of the checkbox and list.

@kisvegabor -
What about the overflow problem of ddlist on a tab? I suspect it's the same problem as the missing character on text area.
There seems to be something wrong with X offset calculation on RTL, maybe padding that is not taken into account. I think you didn't refer to this yet on you replies.

The elements of the button matrix and keyboards also should be shown in RTL?

Button matrix in general - I think Yes.
As for keyboard - since it is defined once, its not very important. But probably the easiest thing is to keep its behavior same as button matrix.

How important it is to support it? If it's not critical, I'd skip it from this first iteration.

Well, it's not critical because it's always possible to access index N - i - 1  instead of i in case of N columns. It's just a little bit less intuitive/convenient.
In Microsoft Excel, for example, when you create a Hebrew spreadsheet, the first column would be on the right-most (instead of left-most you are used to on LTR), so this is the most natural RTL ordering.

It would be quite tricky to support it. As line break algorithms (and all text processing) needed to run the bidi process algorithm to know how/where to make line breaks. (Now I simply store the reversed text.) It seems too much work compared to the supposed benefit. It would be huge rework in all text related algorithms for only those users who need RTL and Text area with line break support.

This, on the other hand, is much more important.
Without this, breaking text lines is dysfunctional on RTL. So it's not a matter of convenience but of a missing/broken functionality.
I'm trying to understand the difficulties in implementing this.
What if we had another function lv_bidi_process_line that processes a single line (the first line of str_in, terminated by \n or \r)
Something like:
uint16_t lv_bidi_process_line(const char * str_in, char * str_out, lv_bidi_dir_t base_dir);

which returns the character position of the next line?
Then the Text Area object could call it incrementally for each line, instead of once for the entire text.

I found a good way to solve the layout issue. If an object has RTL base dir then "ROW" layouts work from Right to Left. It should solve the issues of the checkbox and list.

:+1: Very nice.

As for keyboard - since it is defined once, its not very important. But probably the easiest thing is to keep its behavior same as button matrix.

Considering that the keyboard is a button matrix, it's probably harder to make them differ than to keep them the same.

tabview overflow

What about the overflow problem of ddlist on a tab? I suspect it's the same problem as the missing character on text area.

Upps, I missed reacting to it. ddlist was a special case, now it should work.
table text bidi process
Updated.
table cell order
Not fixed yet
tabview/btnm button order
I updated it. Now the buttons and tabs should be reversed with bidi. I needed to do it becase the tabview uses LV_LAYOUT_ROW_TOP to align the tab pages which is reserved in RTL context. So the order of the buttons needed to match.
In lv_kb_create I explicitly set the basedir to LTR to avoid reversing the buttons.
text area
Please help to collect how the text area should behave. What I know so far:

align text to the right with RTL base dir (should work)
bidi process the text passed to lv_ta_set_text  (should work)
before inserting bidi process the text of lv_ta_add_text
move to cursor to the right if ...? the base dir is RTL? the last strong char is RTL?
should the cursor move left/right (arrow on the keyboard) reverse the step direction in an RTL run?
line break? see below

difficulties with line break
Let's say we have this text (lower case is RTL again):
ABCDEF abcdef ghijk GHIJK
After bidi process:
ABCDEF kjihg fedcba GHIJK
After line break:
ABCDEF ba|
jihg fedc|
k GHIJK  |

So the algorithm will see this. (I write it for myself too to better understand what should happen):

the first line ends on b
when proces_line is called it should see that we start from b
which is is an RTL run so add the chars from the left. But add the chars to the end of the buffer and when ready copy the text to the beginning.
the last char of this line will be j
play 2) again, but now the run ends with k.
still there is space in the line so add chars from the next LTR run

It seems tricky but doable.
The issues I see:

The algorithm seems really tricky, it would take quite a bit time to do implement
The line length is unknown so str_out should equal to strlen(str_in). Not a big issue though because affects only bidi
The line processing is used in several places. Updating and testing all of them might be also challenging.

So the "how should it work" is getting clearer. The question is the "is it really worth the effort now".
However, @amirgon If you contributed with the lv_bidi_process_line function I think I'd do the rest in a reasonable time.

@kisvegabor

move to cursor to the right if ...? the base dir is RTL? the last strong char is RTL?
should the cursor move left/right (arrow on the keyboard) reverse the step direction in an RTL run?

So here is my understanding of text insertion and cursor movements on text box:
When inserting bidi text, the insertion point and cursor movement depend on the direction of the current run. This is true for both RTL and LTR base direction.
For example (examples created from PowerPoint):
LTR:

When the base dir is LTR, the cursor "stays on the same spot" on RTL run, otherwise moves to the right.
RTL:

When the base dir is RTL, The cursor "stays on the sport" on LTR run, otherwise moves to the left.
So the cursor "stays on the same spot" when inserting text with direction opposite to base direction.
When current run ends, and direction switches back to the original base direction, the cursor "jumps" to the end of the string and text is inserted from that point for the new run.
You can see this happening on the LTR example when d is entered, and on the RTL when א is entered.
Explicit cursor movements (keyboard left/right keys) follow exactly the same route as the runs direction. When a run ends and switches back to the base direction, the cursor "jumps" to the beginning of the next run.
Here is an RTL base dir example:

The first cursor position on this example is the beginning of the string (rightmost), and all movements are caused by clicking the left ⬅ keyboard arrow.
On LTR runs, it causes the cursor to move right. On end of run - cursor jumps to the beginning of the next run.
If cursor movement rules are too difficult to implement - maybe we can skip this for the first stage of this feature.
However, I think that correct text insertion and wrapping (line breaks) are important.

Line breaks:

After line break:
ABCDEF ba|
jihg fedc|
k GHIJK  |

I think this is correct, for a text box with LTR base direction.
Please have a look at this discussion from SO linguistics site. Your example seems in line with their "Option D", which was selected as the most reasonable.
For text box with RTL base direction it would look different:
ba ABCDEF|
jihg fedc|
  GHIJK k|

However, @amirgon If you contributed with the lv_bidi_process_line function I think I'd do the rest in a reasonable time.

I can give it a try.
A few questions -

Do you mean a function that would replace lv_txt_get_next_line completely, and take bidi into account? Or a function in addition to lv_txt_get_next_line?
Is it enough that the function returns the index of the letter where wrapping/newline happens?  Then the text box would call lv_bidi_process for every line?
Or does the function need to convert the string itself?

However, @amirgon If you contributed with the lv_bidi_process_line function I think I'd do the rest in a reasonable time.

I can give it a try.

Giving it some more thought, there may be a simple way to achieve line wrapping with a very small effort and minimal changes to lvgl code, without the need for process_line function at all.
It's based on the following assumptions (which I think are true):

Text with wrapped lines looks exactly the same as text with newlines at the end of each line. In other words, each wrapped line can be converted individually by lv_bidi_process.
Line width and the wrapping (newline) point can be calculated on the original string (before calling lv_bidi_process)

In that case - why not add newlines to the string before calling lv_bidi_process?
Something like this:
Every time the label visual width (max_w) changes:

Call lv_txt_get_next_line to find the first character of the next line
Insert newline (\n) before this position, and go back to 1. until the end of the string
Call lv_bidi_process on the new string (which now contains the additional newline characters at the end of each line)

We would probably need to build a new string, instead of inserting newlines.
The size of the new string would be bigger than the original. We can either try to estimate it according to the width, the original string length and font, or simply allocate some memory and re-allocate it if it overflows (the same way std::vector grows on C++)
This suggestion is less efficient and less "clean" than calling lv_bidi_process incrementally, but may be simpler to implement, and re-uses lv_txt_get_next_line to wrap lines.
Another option is not to call lv_bidi_process when setting the text, and instead keep the original string in the label object. Then, when drawing the text, call  lv_bidi_process individually for each line (according to lv_txt_get_next_line).
That would be cleaner, but I'm not sure how hard it would be to change lvgl to work that way.
@kisvegabor What do you think?

simply allocate some memory and re-allocate it if it overflows (the same way std::vector grows on C++)

This increases the chances of fragmenting the heap, in my opinion.
Currently (from a quick test) we only allocate/reallocate memory if the user performs an action that requires it (e.g. creating an object, changing the text on a label - which is a single reallocation, appending to a text area, drawing a shadow that is larger than the previous one). This makes heap usage a lot safer, because since there is not much freeing and reallocating, the chances of fragmentation are practically zero.
From what I can tell, your suggestion would make all text rendering operations require allocation of memory each time the text is drawn, and possibly more reallocations during processing. That's a pretty high price to pay for simplicity, in my opinion.

@amirgon
Cursor movement
Thanks for the hints!
Line wrap
Based on your ideas I got another one (maybe a little bit simpler)
The idea is that if we run lv_txt_get_next_line on the original (not bidi processed text) we got the line breaks at the correct location.  Theoretically, we can mix the characters as we want and will get the same line width. Kerning can be an issue though.
All we need to do is to call lv_bidi_process on these lines and store the result in a new string. All these can be done in lv_label_set_text once.
It's important to store the result as it will be shown because some functions (e.g. lv_label_get_letter_pos) should see the characters in "visual order".
So finally we'd have 2 text stored in the label

the original
the bidi processed

However, I see an issue. If a character is inserted the original text (memory order) should be retrieved to process it again.
So this approach is also not perfect but has issues only with text inserting. What do think? Do you see a good solution for this?

we'd have 2 text stored in the label

the original
the bidi processed

This means that RTL text will now use double the memory of LTR text, right?

If I understood you correctly, this means that RTL text will now use double the memory of LTR text, right?

Yes. It seems we need to choose between memory usage (precompute text) or speed (runtime compute text)

Do either of you know roughly what percentage of the languages someone would be likely to use with LittlevGL require RTL capabilities?

Yes. It seems we need to choose between memory usage (precompute text) or speed (runtime compute text)

@embeddedt Only if RTL is enabled. This will not impact lvgl usage when RTL is disabled.

Do either of you know roughly what percentage of the languages someone would be likely to use with LittlevGL require RTL capabilities?

@embeddedt Is your question is specific to LittlevGL? Do you have information about LittlevGL usage in general? I think it would be very hard to get any number without knowing about all the projects LittlevGL is used on, and how many devices on each project are deployed on each country. For commercial companies this would be confidential information.

All we need to do is to call lv_bidi_process on these lines and store the result in a new string. All these can be done in lv_label_set_text once.

@kisvegabor I don't understand how this can be done once. When label object width changes, we have more lines and more "newline" characters.
When you call lv_bidi_process you must have all the newlines on the right location.
How do you get newlines on the correct location without knowing the object width?

Only if RTL is enabled. This will not impact lvgl usage when RTL is disabled.

True, but any multilingual device would have to have RTL enabled at some point, even if its a runtime setting.

I think it would be very hard to get any number without knowing about all the projects LittlevGL is used on, and how many devices on each project are deployed on each country.

Also true. My main concern is that this could have a significant impact on the memory requirements depending on how text-based an application is.

@kisvegabor - I have a working example of bidi with multi-line text wrap.
It displays the bidi multi-line text correctly, as far as I checked.
See c6b7cf130bebd205b0d355622bb8755d541e3d85
This example does not have the problem of double memory usage - the only text that is saved in the object is the original text.
Instead, I'm calling lv_bidi_process_paragraph for every text line that is processed by lv_draw_label:
https://github.com/littlevgl/lvgl/blob/c6b7cf130bebd205b0d355622bb8755d541e3d85/src/lv_draw/lv_draw_label.c#L172
So in this case I chose runtime penalty (runtime compute text) over memory, but I think it's not very significant. The text processing is relatively simple and most work is done on the font glyph rendering anyway, so I doubt it adds significant runtime compared to the whole text rending runtime cost.
I had to change the lv_bidi to limit lv_bidi_process_paragraph to a maximum number of characters, not only to newline characters, and in order to expose lv_bidi_process_paragraph on the header.
One advantage of this approach is that bidi handling is centralized on lv_draw_label, and there is no need to call lv_bidi_process for every object type that has some label internally.
It is not ready for PR yet because of the following issues:

Need to pass base_dir to lv_draw_label. The problem is that it is called from a lot of places. Perhaps it could receive the lv_obj_t of the caller and extract the base_dir from there?
It uses a global buffer to convert the text. Temporarily I'm just using a 1000 bytes global variable:
https://github.com/littlevgl/lvgl/blob/c6b7cf130bebd205b0d355622bb8755d541e3d85/src/lv_draw/lv_draw_label.c#L171
Instead, we should either allocate it dynamically and grow it as necessary, or use some macro on lv_conf.h for maximum size.
Still need to add ifdefs to compile it conditionally according to LV_USE_BIDI
I'm not sure I'm handling all the corner cases correctly. For example, when max_len does not align well with the new index returned by lv_txt_encoded_next

I have a working example of bidi with multi-line text wrap.

Looks awesome! Congratulations.
We have a temporal buffer during drawing  (e.g. for shadow) which grows as required and freed after drawing the screen. We can use it here too.
There are three important places in lv_label.c where lv_bidi_process_paragraph still needs to be called:

lv_label_get_letter_on
lv_label_get_letter_pos
lv_label_is_char_under_pos

These functions usually find a line with  lv_txt_get_next_line related to a y coordinate and find a char in that line lv_txt_encoded_next related to an x coordinate. I believe we only need to call lv_bidi_process_paragraph when working on the x coordinate.
These functions are used mainly by the text area to get the cursor position. BTW, the way stepping the cursor left/right should work correctly if lv_label_get_letter_pos gives back the coordinates according to the current run.
I see only one point where incompatibility is introduced. Is it possible to keep this condition as in the original version?
In summary, I like this approach. If @amirgon were updated the lv_label_get_letter_ function and test with the text area and the condition in lv_draw_label would be changed I could finalize the code in 1-2 days next week.
I saw that TouchGFX has RTL support. But I haven't ever heard about real bidi support in embedded GUIs. It seems we will be the first with that! :rocket:

Congrats !

I haven't ever heard about real bidi support in embedded GUIs

Well at least definitively not any with small memory footprint !

From what I can tell, your suggestion would make all text rendering operations require allocation of memory each time the text is drawn, and possibly more reallocations during processing. That's a pretty high price to pay for simplicity, in my opinion.

@embeddedt Correct me if I'm wrong, but the draw buffer (lv_draw_get_buf) if freed on every frame in lv_disp_refr_task:
https://github.com/littlevgl/lvgl/blob/9a7b7f8511ced4359990a5365e1f937010f8e7e2/src/lv_core/lv_refr.c#L220

Doesn't it cause memory fragmentation?
What is the motivation of freeing it on every frame and reallocating it on the next frame?
(you could, for example, never free it implicitly)

@kisvegabor That's the buffer you referred to in "temporal buffer during drawing", right? If we are considering using the draw buffer for bidi processing, I would like to avoid memory fragmentation.

The buffer is freed because if you have an object with a large shadow it needs a large buffer but if delete that object a smaller buffer is enough. So it's freed to keep its size suitable for the existing (really rendered) objects.
Basically, if you don't change anything just render the screen over and over always the same part of the memory should be allocated and freed. So after drawing you'll and up the same memory as before rendering.

Basically, if you don't change anything just render the screen over and over always the same part of the memory should be allocated and freed.

This works, until you add another object to the screen/any other allocation happens. After that that chunk of memory is not available to be allocated and freed.

The buffer is freed because if you have an object with a large shadow it needs a large buffer but if delete that object a smaller buffer is enough. So it's freed to keep its size suitable for the existing (really rendered) objects.

@kisvegabor In such case, why not call lv_draw_free_buf only when deleting an object? Why call it on every frame refresh?

@embeddedt

Although, provided that people delete their screens when they're not visible, that (hopefully) isn't a big deal.

Deleting a not visible screen won't trigger redrawing but the memory chunk will change so the reallocations of draw_buf will use different parts of memory. However, as the memory is freed after drawing the whole memory chunk will look the same as before drawing so it doesn't affect fragmentation.
@amirgon

In such case, why not call lv_draw_free_buf only when deleting an object? Why call it on every frame refresh?

For example, shadow width can change with style change too. E.g. a button has some shadow when released but no shadow when pressed.

In summary, I like this approach. If @amirgon were updated the lv_label_get_letter_ function and test with the text area and the condition in lv_draw_label would be changed I could finalize the code in 1-2 days next week.

@kisvegabor I gave it some thought. The change in lv_label_get_letter_ is not that simple.
It's not enough to call lv_bidi_process_paragraph, because in these cases we don't only need bidi text processing, we need to translate cursor position from "logical" to "visual" position.
Moreover, the behavior changes whether we are inserting text, or just moving across text with the keyboard. When adding/inserting bidi text, the cursor stays in the same position, but when moving across text the cursor first jumps to the beginning of the "reveresed" string and then moves backwards.
This is demonstrated on my previous post with animated gifs.
So this is definitely possible, but requires more effort to make it work right. It would probably require changes in lv_bidi.c to translate between visual and logical cursor positions, in both "insert" and "movement" modes, on both LTR and RTL texts.
The other changes/fixes are quite simple and already done on f7c2512c76ca3edebc5241465638e59c273e6f67
These include

Add bidi_dir parameter to lv_draw_label
Use draw buffer for bidi processing
#if LV_USE_BIDI on lv_draw_label which solves incompatibility

With these changes we would have everything needed for displaying bidi text, but not for editing bidi text.
So I'm opening a PR (#1231) for this, without implementing the changes in lv_label_get_letter_.
Text editing is still an open issue which we would need to discuss.

@kisvegabor One more suggestion -
A slider is usually LTR (low values on the left, slides from left to right).
How difficult is it to make lv_set_base_diraffect the slider? (mirror it)

@kisvegabor Found another small bug:
The string "PEM[1]" is displayed as [PEM[1 on RTL. It's displayed correctly on LTR.
Maybe Related to bracket pair when base direction is different than run direction?

A slider is usually LTR (low values on the left, slides from left to right).
How difficult is it to make lv_set_base_diraffect the slider? (mirror it)

It should be possible. I suppose the same applies to bar and sw too.

The string "PEM[1]" is displayed as [PEM[1 on RTL. It's displayed correctly on LTR.
Maybe Related to bracket pair when base direction is different than run direction?

According to the bidi algorithm, it is the expected behavior because the first naturals are processed with base dir. Here is a test in PowerPoint.

With . it works as expected (visually not correctly) but it seems with [] there is an additional logic to keep the brackets together. Shall we implement this? Or in these tricky causes is it acceptable to use LV_BIDI_LRO/RLO (defined in lv_bidi.h) to force the algorithm to works we expect?

I gave it some thought. The change in lv_label_get_letter_ is not that simple.

I'm no 100% sure about it so correct me if I'm wrong. I think lv_label_get_letter_ is really that simple but it really not solves the issue of cursor movement. Can we have a lv_bidi_get_pos_dir(txt, char_index) function? Calling it after insertion we can decide what to do with the cursor. What do you think?

So I'm opening a PR (#1231) for this, without implementing the changes in lv_label_get_letter_.
Text editing is still an open issue which we would need to discuss.

I haven't tested it yet, but looks good. Thank you!

A slider is usually LTR (low values on the left, slides from left to right).
How difficult is it to make lv_set_base_diraffect the slider? (mirror it)

It should be possible. I suppose the same applies to bar and sw too.

Great!

but it seems with [] there is an additional logic to keep the brackets together. Shall we implement this? Or in these tricky causes is it acceptable to use LV_BIDI_LRO/RLO (defined in lv_bidi.h) to force the algorithm to works we expect?

@kisvegabor You are correct that this is not handled by the core UBA algorithm.
However, there is already an extension that handles parentheses and brackets correctly: Bidirectional parenthesis algorithm.  Most programs (PowerPoint, web browsers etc.) implement it.
Should we implement it? That really depends on how "complete" you what the Bidi support to be.
For a complete Bidi support we would need it at some point.
An initial Bidi version of lvgl can probably be published without it, with an "Errata" that specifies that this in not implemented yet.
btw, I found this problem after trying to display some long random texts (from Wikipedia) in a Text Area (to check the multi-line Bidi). This is the first "wrong" thing that "pops up" in the eye of a person who is looking at these texts.
So it's not a "rare" case, and usually texts don't contain LRO/RLO by default because they expect the algorithm to handle it.

Can we have a lv_bidi_get_pos_dir(txt, char_index) function? Calling it after insertion we can decide what to do with the cursor. What do you think?

If you mean lv_bidi_get_pos_dir(txt, char_index) only returns the direction of the character, then it's not enough.
For cursor movements, and also for inserting text (for knowing where to insert a character when the user clicks a Text Area), I think we may need two functions: visual to logical char position conversion, and logical to visual, in order to convert a character position on "memory" (logical) into character position on the converted (visual) string, and vice versa.
After we have these functions, lv_bidi_get_pos_dir may help knowing whether to move the cursor after text insertion.
But I think lv_bidi_get_pos_dir by itself is not enough.

However, there is already an extension that handles parentheses and brackets correctly:

I've checked the link and  I think the gist is the following:

4.1.1. Proposed rule
*N0. Paired punctuation marks take the embedding direction if the enclosed text contains a strong type of the same direction. Else, if the enclosed text contains a strong type of the opposite direction and at least one external neighbor also has that direction the paired punctuation marks take the direction
opposite the embedding direction.

I'd exaggerate if said I fully understood it. :sweat_smile:

Paired punctuation marks take the embedding direction if the enclosed text contains a strong type of the same direction.

"... of the same direction" of what? the base dir?
Anyway, I agree it'd be great to add it because really it's the expected behavior. However, even CSS needs control chars to hande parentheses.
I think line wrap has a higher priority so I suggest solving it first.

If you mean lv_bidi_get_pos_dir(txt, char_index) only returns the direction of the character, then it's not enough.

I was rethinking it and you are right. Here is an example to be sure we are talking about the same thing. (LTR base dir, lower case is RTL)
ABCD dcba EFGH ghfe

If the user clicks on a lv_label_get_letter_on(label, &point) should return pos=5 (logical position) to know where to insert a text. So we need a  function like lv_bidi_get_logical_pos(text, 9). (9 is the visual position of a).
However lv_label_get_letter_pos(label, 5, &point_res) should return the visual coordinates of 'a' to know where to display the cursor. Here we need lv_bidi_get_visual_pos(text, 5).
Do you agree with that?

I think line wrap has a higher priority so I suggest solving it first.

We actually have 3 issues:

Line wrapping. I think this is solved on my PR #1231, but probably needs some review and testing.
Editing support (visual-logical cursor locations)
Bidirectional parenthesis algorithm support

I'd exaggerate if said I fully understood it. 😅

Looking again at the original Unicode Bidirectional Algorithm (UBA), section 3.3.5 (Resolving Neutral and Isolate Formatting Types), I think that correct brackets handling is already described there:

Bracket pairs within an isolating run sequence are processed as units so that both the opening and the closing paired bracket in a pair resolve to the same direction.
...
N0. Process bracket pairs in an isolating run sequence sequentially in the logical order of the text positions of the opening paired brackets using the logic given below. ...

Identify the bracket pairs in the current isolating run sequence according to BD16.
For each bracket-pair element in the list of pairs of text positions

a. Inspect the bidirectional types of the characters enclosed within the bracket pair.
b. If any strong type (either L or R) matching the embedding direction is found, set the type for both brackets in the pair to match the embedding direction.
c. Otherwise, if there is a strong type it must be opposite the embedding direction. Therefore, test for an established context with a preceding strong type by checking backwards before the opening paired bracket until the first strong type (L, R, or sos) is found.

@kisvegabor What do you think?

"... of the same direction" of what? the base dir?

I think that "embedded direction" in that context is the direction of current run (current "embedded" text). Looking at UBA section 3.1.1 "Basics":

BD3. The default direction of the current embedding level (for the character in question) is called the embedding direction. It is L if the embedding level is even, and R if the embedding level is odd.
For example, in a particular piece of text, level 0 is plain English text. Level 1 is plain Arabic text, possibly embedded within English level 0 text. Level 2 is English text, possibly embedded within Arabic level 1 text, and so on. Unless their direction is overridden, English text and numbers will always be an even level; Arabic text (excluding numbers) will always be an odd level. The exact meaning of the embedding level will become clear when the reordering algorithm is discussed, but the following provides an example of how the algorithm works.
BD4. The paragraph embedding level is the embedding level that determines the default bidirectional orientation of the text in that paragraph.
BD5. The direction of the paragraph embedding level is called the paragraph direction.
In some contexts the paragraph direction is also known as the base direction.

So "base dir" is the paragraph embedding direction, but "embedding direction" in general is related to the direction of the current embedding level (odd or even).
I think you don't track and handle embedding levels on your code, right?
In such case the closest thing is the current run direction.

Anyway, I agree it'd be great to add it because really it's the expected behavior. However, even CSS needs control chars to hande parentheses.

Actually, modern browsers handle this without control chars with unicode-bidi:embed which allows bi-directional text in an element.
The posts you refer to in the link are quite old (2011, 2012).
Today, all modern browsers support unicode-bidi:embed and don't require control chars.
I agree that it will be very good to implement this.

If the user clicks on a lv_label_get_letter_on(label, &point) should return pos=5 (logical position) to know where to insert a text. So we need a function like lv_bidi_get_logical_pos(text, 9). (9 is the visual position of a).
However lv_label_get_letter_pos(label, 5, &point_res) should return the visual coordinates of 'a' to know where to display the cursor. Here we need lv_bidi_get_visual_pos(text, 5).
Do you agree with that?

Yes. Exactly what I meant.
In addition to that, we would need to fix cursor movement when inserting text with a direction opposite to base dir.

Line wrapping. I think this is solved on my PR #1231, but probably needs some review and testing.

It seems working and merged.

Bidirectional parenthesis algorithm support

Thank you for the links. I think I got the idea. The gist is to check the text inside the parentheses and:

if there is at last on strong char which has the same dir as the base dir than consider the parenthesis as strong chars with base dir direction.
else start to read the text backward from the opening parenthesis until finding a strong char and apply its direction on both parentheses.

I think nested embedding directions can be achieved by control chars (such as RLE, LRE). I really haven't implemented them. IMO at least for the first iteration, we should be fine without them too. So in our case embedding direction mean base dir.
I have an idea of how to implement it and will do it soon. The standard says 63 levels deep nesting support but I think we should be fine with ~4 too. (I'll make it defineable)

Editing support (visual-logical cursor locations)

Do you already have an idea about its implementation?

Do you already have an idea about its implementation?

@kisvegabor I have an idea, I'm interested in hearing your opinion about it.
We want functions to convert from logical to visual position, and vice versa. But we don't want to replicate/re-implement the bidi algorithm. It's best to keep a single implementation that could both process a Bidi string and perform the visual-logical position conversion.
What if we processed the string with the Bidi algorithm, but instead of writing the original characters we would write numbers that represent the logical position of each character, instead of the character itself?
Here is an example:
Suppose the original (logical) string is
logical_s="ABdcbaC"

After Bidi processing, the new (visual) string is:
visual_s="ABabcdC"

But instead, we would output:
pos_conv_s="0154326"

Now -

If we would like to know the visual position of a, which is on logical position 5, we would check index 5: pos_conv_s[5] yields 2 which is the visual position of a
If we would like to know the logical position of a, we would have to search pos_conv_s for character with value 2 which is the visual position of a -  and it's found on index 5 which is the logical position of a.

Since each character is a Unicode character, we have a lot of code-points and enough bits to represent any reasonable character position.
Even if the text is long, remember we always process it line by line, so a processed string is always a single line that is displayed visually.

Do you mean outputting only numbers or would it be a different mode of operation?
I definitely vote for a different mode because it'd be time-consuming to always get the Nth letter with lv_txt_encoded_next.
It seems "only" this part
needs to be updated to finally write numbers instead of letters.

Do you mean outputting only numbers or would it be a different mode of operation?

Yes, a mode of operation.
I thought of changing the prototype of lv_bidi_process_paragraph and add a new boolean argument:
void lv_bidi_process_paragraph(... , bool pos_conv)

Then add two more functions that would call lv_bidi_process_paragraph with pos_conv enabled and do the conversion:
uint32_t lv_bidi_get_logical_pos(const char * str_in, uint32_t len, uint32_t visual_pos);
uint32_t lv_bidi_get_visual_pos(const char * str_in, uint32_t len, uint32_t logical_pos);

len is important because we don't necessarily convert a whole string, just one wrapped line in case of multi-line text.
These functions would need to allocate str_out before calling lv_bidi_process_paragraph.

It seems "only" this part needs to be updated to finally write numbers instead of letters.

We would need to change any code that changes str_out:
https://github.com/littlevgl/lvgl/blob/d5aabef42d1dd195a525ecdf8824aff215aa6d2e/src/lv_misc/lv_bidi.c#L159-L167
https://github.com/littlevgl/lvgl/blob/d5aabef42d1dd195a525ecdf8824aff215aa6d2e/src/lv_misc/lv_bidi.c#L173-L181
When pos_conv is enabled, Instead of memcpy and rtl_reverse we would need to call a very simple function to fill up numbers.
Something like:
void fill_numbers(char * str_out, uint32_t len, uint32_t initial_value, bool reverse);

Yes, a mode of operation.
I thought of changing the prototype of lv_bidi_process_paragraph and add a new boolean argument...
Then add two more functions that would call lv_bidi_process_paragraph with pos_conv enabled and do the conversion...

Great!

These functions would need to allocate str_out before calling lv_bidi_process_paragraph.

It might be not trivial.

When pos_conv is enabled, Instead of memcpy and rtl_reverse we would need to call a very simple function to fill up numbers.

It's a little bit more complicated because rtl_revers handles the weak LTR runs in RTL context too

void fill_numbers(char * str_out, uint32_t len, uint32_t initial_value, bool reverse);

What about using an uint16_t num_out[] buffer to store the numbers? It'd be more convenient to handle than UTF-8 encoded numbers.

These functions would need to allocate str_out before calling lv_bidi_process_paragraph.

It might be not trivial.

Why? What's wrong with using lv_draw_get_buf for that?

It's a little bit more complicated because rtl_revers handles the weak LTR runs in RTL context too

I don't think so. It is called separately for each run.
Could you explain?

What about using an uint16_t num_out[] buffer to store the numbers? It'd be more convenient to handle than UTF-8 encoded numbers.

But then the strings would need to be cast from char* to uint16*.
Also, how would you know how much memory to allocate for str_out, if str_in is UTF8 string and str_out is uint16_t[]?

It's a little bit more complicated because rtl_revers handles the weak LTR runs in RTL context too

I don't think so. It is called separately for each run.
Could you explain?

Got it now.
Then the reverse function would also need to operate in two modes.

Why? What's wrong with using lv_draw_get_buf for that?

Probably lv_label_get_lettert_on/pos are not called in a drawing context but during indev handling when draw_buf is not allocated.
However, lv_label_get_lettert_on/pos probably will trigger a redrawing so draw_buf will be freed in a few milliseconds anyway.

I've added bracket handing and it worked correctly with the examples in UBA 3.3.5.
I've set LV_BIDI_BRACKLET_DEPTH to 4 but we can modify it if you feel it's not enough. (8 bytes / depth)

I've added bracket handing and it worked correctly with the examples in UBA 3.3.5.

@kisvegabor It looks good!
It seems to work well for balanced brackets, but how do you handle unbalanced brackets? (I noticed that the examples on UBA 3.3.5 are balanced)
In general, text does not have to be written with balanced brackets. Unicode defines some rules how to handle non balanced brackets.
The Bidi Brackets for Dummies explains this (and is also really fun to read!).
On Chapter 5. Brackets: Parsing and Multiple Brackets:
So what is the first rule of Unicode Bracket Club?

— Um..., don't talk about brackets?

No. The first rule of Unicode Bracket Club is: Matching bracket pairs for UBA is not expression evaluation.

— Huh?

Well, yes. Most people's notion of how multiple parenthesis matching works comes from their high school algebra class. Remember those equations:

((a + b) * (b + 9a)) + ((2a – 1) * (2b + 17)) = –1

You had to go through and evaluate all the expressions and combine the inner expressions by applying the operators to form other expressions, and so on.

And what happened if you were missing a balancing parenthesis?

— Uh, the algebra teacher hit us with a ruler.

Right, and the equations didn't work, because they had what is called an expression error. If you use parentheses (or brackets for matrices, and so forth), they have to balance, or the expression has a syntax error, and it cannot be evaluated.

But in Unicode Bracket Club, if a "balancing parenthesis" is missing, meh! It just changes what "bracket" matches what other "bracket" in the line. And a missing match isn't an error — it's just "interesting".

So if you're all ready with your BNF for brackets and have an LL-grammar ready and want to get busy with an LALR parsing strategy for bidirectional brackets, you're probably in the wrong class. Maybe Compilers for Dummies is what you wanted.

(And this is actually taken from a formal Unicode Technical Note! (tn39) )
Actually, I'm not sure if you handle non balanced brackets correctly or not, or if it's even important enough to spend time doing this correctly.
Just wanted to bring this up for completeness, and throw here the link to Bidi Brackets for Dummies.

This text is really kind of funny. :smile:
Now if the algorithm in lv_bidi finds an opening bracket it looks for its closing pair and decides the bracket's direction. If no closing bracket has found then the opening bracket will be treated as neutral. Orphean closing brackets are also treated as neutral.
IMO it's even not trivial what is the expected behavior for a novice user for a text like apple[WERBEH) {2]] with RTL basedir.
Therefore I'm not sure it's worth the effort to implement the unbalanced bracket handling. In addition, it fits less in the current algorithm because it required more "free move" in the text and more buffering.

@amirgon Will you have time to deal with the visual/logical order converter things?

Will you have time to deal with the visual/logical order converter things?

@kisvegabor I'll try to find some time for this.

@kisvegabor I noticed something strange with text selection (lv_ta_set_text_sel(ta, true)), and I don't think it's related to Bidi.
Did you try selecting text in a multi-line text area? It doesn't work right.
I also noticed that there is no function for getting the selected text, or the selected text position and length.
Is this feature in use?

Text selection also requires the proper handling of logical/visual positions. It works similarly to the cursor: you can set a start and an end position to draw a background letter between them.
In dev-6.1 with the demo it works well for me:

I also noticed that there is no function for getting the selected text, or the selected text position and length.

This functionality exists in the underlying label (because, AFAIK, I implemented it) but the corresponding functions have not been brought through to the text area.

@kisvegabor I spent some time and implemented lv_bidi_get_logical_pos and lv_bidi_get_visual_pos.
I think they work as expected.
For example, the string:
Hello - שלום!...

which is displayed (on RTL base dir) as:

yields these positions
[11, 12, 13, 14, 15, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

(Which means, for example, that the first logical character H is displayed on visual position 11).
However, I'm having difficulty integrating this into lv_label.c functions (lv_label_get_letter_pos, lv_label_get_letter_on etc.).
It seems that we need to both convert the line to Bidi and do the logical-visual conversion, but so far I can't get it to work right.
Could you have a look?  (https://github.com/amirgon/lvgl/tree/feat-bidi-pos_conv)

I think they work as expected.

Can you send a PR?

I think they work as expected.

Can you send a PR?

#1253

Thank you!
I've tested lv_label_get_letter_on like this and it worked well for me.
static void label_event_cb(lv_obj_t* label, lv_event_t event)
{
    if(event == LV_EVENT_CLICKED) {
        lv_point_t p;
        lv_indev_get_point(lv_indev_get_act(), &p);
        p.x -= label->coords.x1;
        p.y -= label->coords.y1;
        uint16_t id = lv_label_get_letter_on(label, &p);
        printf("id: %d\n", id);
    }
}

...

    lv_obj_set_base_dir(lv_scr_act(), LV_BIDI_DIR_RTL);

    lv_obj_t * label = lv_label_create(lv_scr_act(), NULL);
    lv_label_set_text(label, "abcdefgשדג");
    lv_obj_set_click(label, true);
    lv_obj_set_event_cb(label, label_event_cb);

However,  lv_label_get_letter_pos gave wrong result. I've pushed a minor fix but it doesn't solve the issue. Maybe there is another letter index/byte index mismatch too?
    lv_obj_set_base_dir(lv_scr_act(), LV_BIDI_DIR_RTL);

    const char * txt = "abcשדג";
    lv_obj_t * label = lv_label_create(lv_scr_act(), NULL);
    lv_label_set_text(label, txt);
    uint8_t i;
    for(i = 0; i < lv_txt_get_encoded_length(txt); i++) {
        lv_point_t p;
        printf("log:%d, ", i);
        lv_label_get_letter_pos(label, i, &p);
        printf("x:%d, y:%d\n", i, p.x, p.y);
    }

vp: visual_position in letter index
vpb: visaul position in byte index
log:0, vp:3, vpb:6, x:0, y:23
log:1, vp:4, vpb:7, x:1, y:30
log:2, vp:5, vpb:8, x:2, y:37
log:3, vp:2, vpb:4, x:3, y:13
log:4, vp:0, vpb:0, x:4, y:0
log:5, vp:65535, vpb:5, x:5, y:23

As I've tested further I'm not sure my fix is required. Does lv_bidi_get_visual_pos return letter or byte index?

Both lv_bidi_get_visual_pos and lv_bidi_get_logical_pos currently receive and return character position, not byte position.
I don't think a "visual" byte position makes any sense, and a "logical" byte position just seemed like a waste of ram and cycles, so it seems to me that the most natural way to convert visual-logical positions is not by byte offsets, but by character offest.

Byte position is mainly required to point to a character in the string. E.g. line_start which you have already seen in a lot of places.
I've pushed a fix related to byte/character indexes. I've texted it with lv_test_kb_1() and changed some characters on the keyboard to Hebrew.
It's almost perfect. Can you take look at it? As you have more experience with bidi hopefully you will see what would be the expected behavior and where to search for the bug.

It's almost perfect. Can you take look at it? As you have more experience with bidi hopefully you will see what would be the expected behavior and where to search for the bug.

@kisvegabor It certainly looks better.
Indeed, I didn't see all the places where char index should be converted to byte index and vice versa.
Here are some problems I noticed after your recent commits:

It doesn't work well on multiline text. When clicking the second line to set the cursor position, the cursor remains on the first line and moves incorrectly.
The problem seems to be in lv_label_get_letter_on and not in the cursor positioning, because when I set the cursor position with lv_ta_set_cursor_pos it is moving to the correct line.
Text selection (lv_ta_set_text_sel(ta, True)) is not working well. It's not selecting ("coloring") the same characters being selected (dragged). Perhaps additional functions need special Bidi handling?
When adding a character to an RTL string, the character is inserted on the left side of the character under cursor instead of on the right side. It feels like text is inserted with a "shift" of one character.
For example, suppose "cba" is the visual representation of an RTL string and the cursor is blinking between "a" and "b" (cb|a).
After calling lv_ta_add_text(ta, '.'), the result is cba. instead of the expected cb.a
I suspect the cursor itself is not displayed on the correct side of the character it points at.
When calling lv_ta_set_cursor_pos(ta, 0) on RTL text, the cursor is set after the first character, instead of before it. This seems to be related to the previous problem of adding text.
Here is an example:

It could be that it's not enough to know the visual/logical position. We may also need to know the direction of current run for a specified character position in a string, so we would know on which side of the glyph to put the cursor.
On the "pos_conv" array we keep uint16_t for each position. We could use the MSB of the 16-bit position to record whether current position is RTL or LTR, I think the remaining 15 bits are enough for specifying the position itself.
What do you think?

It doesn't work well on multiline text. When clicking the second line to set the cursor position, the cursor remains on the first line and moves incorrectly.
The problem seems to be in lv_label_get_letter_on and not in the cursor positioning, because when I set the cursor position with lv_ta_set_cursor_pos it is moving to the correct line.

I've fixed it and found an issue
https://github.com/littlevgl/lvgl/issues/1256) with text line processing (not related to bidi)

When calling lv_ta_set_cursor_pos(ta, 0) on RTL text, the cursor is set after the first character, instead of before it. This seems to be related to the previous problem of adding text.

The cursor is shown of the left side of the current logical character. So in the image, the cursor is shown on the left side of the 0th character.  It's really bad in RTL context. It seems the cursor position should be shifted by a char_width to the right.

On the "pos_conv" array we keep uint16_t for each position. We could use the MSB of the 16-bit position to record whether current position is RTL or LTR, I think the remaining 15 bits are enough for specifying the position itself.

It seems a little bit overkill to gather so much information only to know that if the latter is in an RTL or LTR dir. However, there are complex cases where it's difficult to decide the direction (e.g. because of brackets) so I'm fine with that.

Text selection (lv_ta_set_text_sel(ta, True)) is not working well. It's not selecting ("coloring") the same characters being selected (dragged). Perhaps additional functions need special Bidi handling?

Let's postpone it. Hopefully, it will be solved automatically with the other things.

I've fixed it and found an issue
#1256) with text line processing (not related to bidi)

The cursor is still jumping to the wrong position and wrong lines on multiline text.
When clicking the second line and forth, the position is incorrect (larger than the clicked position, even past the end of the string).
Is this behavior related to #1256?

It seems a little bit overkill to gather so much information only to know that if the latter is in an RTL or LTR dir. However, there are complex cases where it's difficult to decide the direction (e.g. because of brackets) so I'm fine with that.

It was simple to add this. I don't think it adds much complexity.
Fixed on 0130f3e5f2
On this fix the pos_conv array also hold the an RTL indication for each character (whether the character is part of a reversed string).
With this fix it looks like the characters are added on the correct location where the cursor is.
Instead of changing the cursor position (it's still on the left of the character) I incremented the visual/logical pos in case of RTL, I'm not sure this is the right way to do it though.
It's still not good enough because when I click at the end of the line the cursor is displayed on the next line instead of the end of the line (see what happens when I click the left-most dot):

Perhaps it's better to change the way the cursor is displayed when moved on RTL character instead of changing the visual/logical pos.
What do you think?

Let's postpone it. Hopefully, it will be solved automatically with the other things.

So far - It wasn't.... :disappointed:
Selection is going to the opposite direction - when you drag left it selects right and vice versa. I don't see how it's related to the other problems (beside the fact that multiple lines are "selected").

I click at the end of the line the cursor is displayed on the next line instead of the end of the line

I think that's the same problem as #1256, so hopefully that can be solved easily.

The cursor is still jumping to the wrong position and wrong lines on multiline text.
When clicking the second line and forth, the position is incorrect (larger than the clicked position, even past the end of the string).

I've pushed a fix for it. byte_id and char_id was mixed in lv_label_get_letter_on. It's really tricky do deal with them but I haven't found a better way so far... :frowning_face:

Is this behavior related to #1256?

I don't think so, it should affect only the line endings.

It was simple to add this. I don't think it adds much complexity.

Great!

With this fix it looks like the characters are added on the correct location where the cursor is.
Instead of changing the cursor position (it's still on the left of the character) I incremented the visual/logical pos in case of RTL, I'm not sure this is the right way to do it though.

It seems fine. I was concerned about what happens if you explicitly set the cursor position with lv_ta_Set_cusrsor_pos(ta, 1);  But it worked well in my test.

Selection is going to the opposite direction - when you drag left it selects right and vice versa. I don't see how it's related to the other problems (beside the fact that multiple lines are "selected").

I fixed part of it. Now not all lines are selected but something is not working yet. It workes more or less with LTR base dir but does nothing with RTL for me.
It seems it select less character if Hebrew chars are also selected. Probably it's also a char_id/byte_id issue.

I've pushed some fixes for text selection.
It works quite well in LTR:

But it still wrong with RTL:

I've found that the problem is here. lv_bidi_get_visual_pos returned 25 for logical_pos = 2.
@amirgon  Could you take a look?

I've found that the problem is here. lv_bidi_get_visual_pos returned 25 for logical_pos = 2.
@amirgon Could you take a look?

@kisvegabor I think the problem is that logical-visual position only makes sense on the X axis and not on the Y axis.
(After writing an RTL line we don't go "up" one line, we go "down" one line just like LTR)
On this code you pass the whole string txt and not a single visual line:
https://github.com/littlevgl/lvgl/blob/6eaf8dd6d3eefc87455ad856999295bbe92b812d/src/lv_draw/lv_draw_label.c#L117
https://github.com/littlevgl/lvgl/blob/6eaf8dd6d3eefc87455ad856999295bbe92b812d/src/lv_draw/lv_draw_label.c#L122
The lv_bidi_get_visual_pos conversion function can only process a single visual line. It cannot process the entire text because it does not get enough information to know where each visual line starts (that depends on the label width, font size etc.)
So the solution is to first find the logical line_start and line_end of both sel_start and sel_end and pass only txt + line_start, line_end - line_start instead of txt, strlen(txt) to lv_bidi_get_visual_pos.
We would also need to add the logical line_start to the result of lv_bidi_get_visual_pos, but that's ok because an entire line length is the same whether it's visual or logical.
What do you think?

Thanks for the clarification. After evaluating some options I ended up converting all line_start+i to logical char position and check if it between sel_start and sel_end.
It still not perfect with the first character of the line (might be related to #1256) and on the edge of RTL/LTR dir changes.
I'll check the fix of #1256 soon and try text selection again.

I've merged the lien ending fix of @BrianPugh and applied some other fixes and now it works quite well for me.

As a reference here is how it works in Eclipse:

If we don't find critical issues I suggest merging this feature to dev-6.1 to get some feedback. I'll enable LV_USE_BIDI until the release.

@kisvegabor after your fixes it looks better, but there are still some issues.
If you want, you can merge it to dev-6.1 and we'll continue the fixes there, I don't think there is a reason to keep updating the PR separately.
We previously discussed slider/bar/sw RTL direction:

A slider is usually LTR (low values on the left, slides from left to right).
How difficult is it to make lv_set_base_diraffect the slider? (mirror it)

It should be possible. I suppose the same applies to bar and sw too.

How difficult is it to implement it?

Text selection works better, but there are still a few problems:

On a text area with RTL base dir and RTL text, I just can't select the first character on a line without selecting characters from the previous line.
Also, when trying to select "backwards" (to the right towards the beginning of the RTL line) when the cursor reaches the first character, the entire text is suddenly scrolled and selected all the way down:

When selecting LTR text in RTL base-dir text area, the cursor and the selection end position are not the same.
Perhaps there is another place that should be converted from byte-pos to character-pos?

Something is reversed in the selection on text with both RTL and LTR runs.
Look at this for example:

Several wrong things happen here:

First character is not selected (as mentioned earlier)
The text Base64 selection: When the character 4 is dragged by the cursor, ase64  is selected, and selection is reduced when dragging the cursor further until only the space is selected, then the entire Base64  is selected...
The text קריאותם is RTL run in RTL run, should not be different from other RTL strings, but selection there also behaves in a "reveresed" way.

My wild guess is that there is a relation between the Base64 and the קריאותם strings.
Their position is exactly mirrored on the line (indices i and len-i correspondingly) and at some point the selection should be calculated with len - i instead of i to fix this.

If you want, you can merge it to dev-6.1 and we'll continue the fixes there, I don't think there is a reason to keep updating the PR separately.

Done.

We previously discussed slider/bar/sw RTL direction:

I've reworked the bar, slider, and switch in v7.0. So internally they work differently and it's difficult to transfer the updates from v6.1 to v7.0.
If it's not so important then I'd rather add it v7.0 to not make the same work twice.

On a text area with RTL base dir and RTL text, I just can't select the first character on a line without selecting characters from the previous line.
Also, when trying to select "backwards" (to the right towards the beginning of the RTL line) when the cursor reaches the first character, the entire text is suddenly scrolled and selected all the way down:

Might it be related to the logical_pos+1 in lv_label_get_letter_on?

Several wrong things happen here:

That's really interesting. Please copy here the text you have tested with.

If it's not so important then I'd rather add it v7.0 to not make the same work twice.

Sure, having these on v7.0 is good enough.

Might it be related to the logical_pos+1 in lv_label_get_letter_on?

Yes it might be related.
Do you have an idea how to draw the cursor on the other side of the glyph, according to is_rtl character indication returned from lv_bidi_get_visual_pos?

That's really interesting. Please copy here the text you have tested with.

I've added the entire script I'm using on https://gist.github.com/amirgon/8f2f1ac2fea32f6657f9c00b3da40340#file-test_heb_font-py-L115-L150.
The text where I noticed the problems is on txt1 variable.
If you want to run the test_heb_font.py script, you can use:
ports/unix/micropython -i test_heb_font.py

I've played with the text you sent and I've also seen the strange selection. I tried how it works in Eclipse and I was able to make this funky selection there too:

So the perfect selection might be another can of worms which I afraid to open. :)

Do you have an idea how to draw the cursor on the other side of the glyph, according to is_rtl character indication returned from lv_bidi_get_visual_pos?

IMO it can be only a visual shift added in refr_cursor_area. I.e. adding/substracting (I'm not sure which one :sweat_smile: ) the logical character's width to/from the current position.
Looking into this function I've just realized that we haven't considered different cursor types (such as LV_CURSOR_BLOCK).  Now it is shifted by on char in RTL dirs:

I think we have done a great job so far even if there are minor issues. IMO this feature in its current stage can serve the need of the vast majority of the users. So I'm open to minor tweaks/fixes but I think it is not reasonable to dig much deeper. What do you think?

So the perfect selection might be another can of worms which I afraid to open. :)

No I'm not talking about a "perfect selection". If we even get close to Eclipse (or HTML text box) behavior it would be excellent!
But I think that we are not there yet.
There are some real bugs you will never find in Eclipse or any other editor of your selection.
I've showed and explained these bugs on the previous post, but here is a summary of them:

Can't select the first character
Text suddenly scrolls all the way down when selecting backwards towards the beginning of the line (rightmost)
Cursor and selection area edge are, at some cases, away from each other
Correctly support different cursor types, as you correctly pointed out.
When RTL in RTL is selected (not LTR in RTL!) sometimes a reversed selection happens. If all your text is RTL, selection should not suddenly behave differently for some random word. But this is exactly what happens (screenshot from the gif above):

This "reversed" behavior should happen only when you are selecting a mix of RTL and LTR runs. But if you select RTL text only, it does not make sense that one random word in the middle would suddenly be selected in reverse.

All of these problems are not "weird" or "unusual" behavior. They are obvious bugs to the RTL reader and you will not find them on any other production-class editor/text-box out there.

I think we have done a great job so far even if there are minor issues. IMO this feature in its current stage can serve the need of the vast majority of the users. So I'm open to minor tweaks/fixes but I think it is not reasonable to dig much deeper. What do you think?

We did a lot of progress and addressed a lot of issues, no doubt about that!
I don't think the job is done, because of the issues above. I don't think that solving these bugs above can be considered as "digging deeper".
On the other hand, I understand your desire to close this and go on. You've spent a lot of time on this already.
So my suggestion is - let's merge it as is to v6.1. and v7.0, knowingly there are open bugs that were not resolved.
But let's open a new issue that summarizes all these open issues and keep it open until they are resolved. We could add the requirement for slider/bar/sw RTL direction support there as well.
What do you think?

So my suggestion is - let's merge it as is to v6.1. and v7.0, knowingly there are open bugs that were not resolved.
But let's open a new issue that summarizes all these open issues and keep it open until they are resolved. We could add the requirement for slider/bar/sw RTL direction support there as well.

That sounds good! We can close it to indicate the progress and record the remaining things to do in a clear way in a new issue. Would you open the new issue?
I've merged feat-bidi to dev-6.1 and dev-6.1 to dev-7.0.
In dev-7.0 draw_buf works differently. There are multiple draw buffers and lv_draw_buf_get returns a suitable one. In addition, the used buffer needs to be explicitly released by lv_draw_buf_release. So using draw buf in lv_bidi_get_logical/visual_pos is not working so nicely in v7.0 because we need to keep track of the allocated buffer.
I've fixed it by changing lv_graw_get_buf to lv_mem_alloc/free. I used lv_mem to not mix independent modules (drawing to general text processing). Its performance impact should be minimal.

That sounds good! We can close it to indicate the progress and record the remaining things to do in a clear way in a new issue. Would you open the new issue?

Sure! Opened #1274.

I've fixed it by changing lv_graw_get_buf to lv_mem_alloc/free. I used lv_mem to not mix independent modules (drawing to general text processing). Its performance impact should be minimal.

Isn't it better to have a static RAM area that can be used as a "scratchpad" instead of allocating/de-allocating it on every Bidi function call?
Even if it doesn't cause defragmentation in this case, malloc/free are notorious for their inefficiencies and performance impact on non trivial heap implementations, when blocks can be re-used.

Sure! Opened #1274.

Thanks!

Isn't it better to have a static RAM area that can be used as a "scratchpad" instead of allocating/de-allocating it on every Bidi function call?

What should be size of this static RAM area? I mean if it's too large then it consumes memory for nothing, if too small it won't be enough for longer texts.
Maybe a simple "buffer manager" can be created directly for bidi: reallocate is the buffer is too small and sometimes free. Similarly to disp_buf in v6.1. What do you think?

What should be size of this static RAM area?

Probably related to the maximum number of characters in a line.