Diagram text display according to a given expression

The aim is to implement something similar to Herve's Specialization plug-in where an alternate expression can be specified for displaying text on a diagram node or connection. So we must credit @herve91 with the original idea and implementation for this. :-)

There will be a text box in the Properties tab where the expression is entered
The expression is stored internally as a string in a "feature"
If the expression is present it is evaluated and converted to a string that is displayed on the node or connection label instead of the name

Examples of expressions:
${name}
${documentation}
${name}\n${documentation}
Hello World

Starting code is in branch "text-render".

Obviously we want this implemented on these objects:

ArchiMate Connections
ArchiMate Elements

But which of the following visual objects do we also want this on?

Other Connections
Groups
Notes
Sketch objects
Canvas objects

I've added a Property section text control to enter the expression. But where should this go? As it is part of the display options it should really go in the "Appearance" tab (like the "Display Name" tick-box for a Connection) but it seems unintuitive there.
For testing I've just stuck it at the top of the "Main" tab:

Current expression options are (as well as \n for newline):
${name}
${documentation}

But when typing these in they seem to me long-winded, and wonder if we could use as short codes:
${n}
${d}

I'll test soon. But in the meantime, some comments...

But which of the following visual objects do we also want this on?
I have my doubts about Notes and Sketch/Canvas objects since they are free text.

I'd say most if not all of them. It makes sens on notes because this would allow some nice features like displaying the documentation of the element connected to the note. And now that most visual objects have properties, there might be some use-cases related to mixing object's text and properties.

But where should this go? As it is part of the display options it should really go in the "Appearance" tab (like the "Display Name" tick-box for a Connection) but it seems unintuitive there.

I think it has to go in the Main tab, but below Name field. I wonder whether we need a checkbox to enable this feature ([ ] Use custom label), or if it simply has to come with a default value ${name} (in this case, maybe we need a button at the end to reset its value), or maybe even simply make sure that an empty label is replaced by the object's name.

But when typing these in they seem to me long-winded, and wonder if we could use as short codes

I think we have to support Hervé's options as they are, but add some shorter options too. Maybe at some point we could mark historical options as deprecated. But personally I won't go for too short option neither, because it might become hard to read ($[name} seems ok to me, ${doc} would be just fine).

I think it has to go in the Main tab, but below Name field.

That's a pain because Name and Documentation fields are one section (AbstractNameDocumentationSection). It can be done but all the existing Property Sections will have to be refactored to split Name and Documentation.

I wonder whether we need a checkbox to enable this feature ([ ] Use custom label), or if it simply has to come with a default value ${name} (in this case, maybe we need a button at the end to reset its value), or maybe even simply make sure that an empty label is replaced by the object's name.

If the field is empty the default text is shown (name or note/sticky content)

I think we have to support Hervé's options as they are

It won't make any difference since Herve stores the expression in a Property and we store it in a Feature.

For now lets support all objects in an ArchiMate View and think about Sketch and Canvas later.

this would allow some nice features like displaying the documentation of the element connected to the note.

I wonder if we could do this by referencing another object by its ID? Something like:
${documentation:1234-5678-9876}
or:
${ref:1234-5678-9876${documentation}}
Which returns the documentation that is in the object with the ID "1234-5678-9876"

I think we have to support Hervé's options as they are

Because I'm using properties, my implementation is linked to the elements and relationships themselves so the same label applies in all the views. Your implementation, which allows have different labels in each view, is much better ;-)

That's a pain because Name and Documentation fields are one section

So let's move label in Appearance tab and keep Main and Properties for everything which is purely related to model elements and not visual objects.

If the field is empty the default text is shown (name or note/sticky content)

Good

It won't make any difference since Herve stores the expression in a Property and we store it in a Feature.

I was thinking about people who could create a script to switch from props to features, or simply copy and past them. But I agree this is not a big issue if we are slightly incompatible.

For now lets support all objects in an ArchiMate View and think about Sketch and Canvas later.

Agree. I wasn't saying to do everything now.

Problems with this approach:

The user would need to get the ID of the other object

Yes, that's the issue and why I'm planning to navigate connections (but not relationships)

So let's move label in Appearance tab and keep Main and Properties for everything which is purely related to model elements and not visual objects.

I've created a sub-tab "Display" off of the Main tab as an experiment, We can, if we like, have a multi-line text box instead of a single line text box so user doesn't have to use \n for new lines.

I've just made a change to TextControlRenderer and added:
public static final boolean MULTI_LINE = true;
So we can try out single-line or multi-line text controls.

"text-render" git branch has been cherry-picked, rebased and re-pushed to tidy it up.

Added some expressions for Properties and a Note linked to another object (shows the linked object's documentation)

I've refactored things so that each text expression renderer can register itself to handle text expressions and return the resulting display string. Each renderer implements ITextControlRenderer and registers itself with TextControlRenderer.  So we have one for name, documentation, properties, relations etc. This is so plug-ins can contribute.

I am really looking forward to having this feature.
When it comes to expressions it would be great to have also something like ${element-type} which resolved to ArchiMate Element type, e.g. Business process, etc. The rationale is: to be able very easily recreate the style used in "Mastering ArchiMate" with its explicit indication of type (instead of relying only on a combination of icon and color). It would be even better if one is able to define it for selected elements in one go as something like "default display label" :)

@WaruiOkami Yes, that's possible, and I've just committed a "TypeRenderer".
Example - given a Business Process with the name "Create Contract" and this expression:
${name} (${type})
This would render as on the View as:
Create Contract (Business Process)

I've created a sub-tab "Display" off of the Main tab as an experiment, We can, if we like, have a multi-line text box instead of a single line text box so user doesn't have to use \n for new lines.

After playing with this for some time now I much prefer the multi-line text box over the single line one

Here's an example of my experiments.
Given the following name and documentation of and element:

With these Properties:

And this Text Expression:

We get this:

That is looking great :)
If I may suggest something: if you are looking for a place for text expression - have you considered making it part of the "appearance" tab? For me it seems logical (i.e. this was the place I looked for this feature before) to put it together with font, text alignment and such things - after all, all of this defines how the element looks.

Yes, I am considering that. But if so, it should be a sub-tab of the Appearance tab (with tab name something like "Display" or "Text Expression") as there are too many UI controls on the Appearance tab and this really needs some space for a multi-line text control.

I first thought it should be on the Appearance tab, but I now think it should be separated because we could extend this mecanism to the Model Tree, not on a per concept basis, but with a setting on Folders to change the way concepts inside this folder are listed.
A typical example would be to define a display expression '${name} (${viewpoint})' for the Views folder.

...we could extend this mecanism to the Model Tree, not on a per concept basis, but with a setting on Folders to change the way concepts inside this folder are listed.
A typical example would be to define a display expression '${name} (${viewpoint})' for the Views folder.

First implementation of this is committed.

Hi,
I've played a bit with this neaw feature and it is awesome! I really like the way it has been implemented (I've been able to extend some part of it quickly).
Things I'm working on atm:

I've extended PropertiesRenderer.java to add support for ${properties:separator:propkey} which basically allow to get the values from one specific property key and concatenate them using a choosen separator. This is useful for (e.g.) data modelling:
Given this properties on a business object

Setting the label to ${name}\n\n${properties:\n:Attribute} gives you

I'm adding a ${wordwrap:maxlength:text} macro to allow some kind of wordwrapping. I think this might be helpful for relationships when description is a bit too long.
I will certainly change a bit the order of execution so that the result of some macro can be used as input for the next ones. This should allow some kind of freedom and power (Herve uses this to get the lang from a model property and then the right text to show from a concept property).

Some remarks:

It seems that some sequences of characters are removed from the text. I wanted to allow entering \n (verbatim, not a newline) as a separator in my new ${properties:separator:propkey} macro, but the sequence \n is not in the text passed as argument to ITextRenderer and no real newline is generated neither (and this is not related to TextRenderer#renderNewLines). Of course as we use a multiline I can enter a real newline, but this is not always easy to read.
Label set on folders are really great but we have to set them on each folder. Would it be possible to define some inheritance so that setting a label on a top level folder would apply anywhere below, unless another label is defined (in this case ${name} would allow to reset to default behavior).
(dream_mode on) is there any way to alter the tree structure itself? If yes, we could create "virtual" tree through label: ${type}/${name} would create a virtual folder for each concept type used and classify concepts accordingly.
As plugin can expose new macros, one day we could have the modelrepository plugin expose some ${lastcommitid} or ${lastcommitdate} providing access to the latest commit having impacted the concept ;-)

Sounds great!
Make sure you got the latest "text-render" branch because I've been deleting it and re-adding it on top of the latest master branch a lot to keep things up to date (I think I even did that yesterday). I won't do that now that you are working on it.

I'm adding a ${wordwrap:maxlength:text} macro to allow some kind of wordwrapping. I think this might be helpful for relationships when description is a bit too long.

How would this work in the Connection Figure? The figure's label can only handle "\n" characters.

It seems that some sequences of characters are removed from the text.

I thought this would be because of TextRenderer#renderNewLines()? Must be some other reason? Perhaps use a special char sequence instead of \n? Something like ${CR} or similar?

Label set on folders are really great but we have to set them on each folder. Would it be possible to define some inheritance so that setting a label on a top level folder would apply anywhere below, unless another label is defined (in this case ${name} would allow to reset to default behaviour).

Should be possible.
I look forward to seeing your new stuff. What I've done so far was just a starting point. Feel free to change things. The only thing I feel strongly about is having the multi-text control for the expression as it gives a lot of room.

(dream_mode on)...

(dream_mode off) - probably not possible, or at least not easy, given the way the Tree works.

As plugin can expose new macros

Yes, but we have to be careful about the order of calling each one ("I will certainly change a bit the order of execution so that the result of some macro can be used as input for the next ones.")

Make sure you got the latest "text-render" branch...

Ok, I'll check before commiting my changes

How would this work in the Connection Figure?

That's really simple: the idea is to simply insert newlines after each word reaching the "maxlength" of each line.
So if a concept has for name The quick brown fox jumps over the lazy dog, then ${wordwrap:15:${name}} would render as
The quick brown
fox jumps over
the lazy dog

Perhaps use a special char sequence instead of \n? Something like ${CR} or similar?

That's a good idea, but if I implement it I'll go for ${nl} (for newline) ;-)

I look forward to seeing your new stuff. What I've done so far was just a starting point. Feel free to change things. The only thing I feel strongly about is having the multi-text control for the expression as it gives a lot of room.

I really like the multi-line control ;-)
I'll try to commit something before end of week. Would be good to to publish an "Early Access 3" next week :-)

but if I implement it I'll go for ${nl} (for newline) ;-)

I thought of that but not sure if nl is sometimes used relating to language internationalisation.

I'm not sure if this was noted on the thread, but one of the advantages of Herve's plugin is that the label replacement can be controlled by element, but also by view. So the same element can have one name on one view, and another name on another view. This helps when switching from an over view where element names might not be very important to a more detailed view where those same elements require further detail.

one of the advantages of Herve's plugin is that the label replacement can be controlled by element, but also by view.

Well, in fact that's the opposite: Herve's plugin stores the label macro in an element property, so the same label is used everywhere. The only thing you can do is set a view to not use labels, but you can't have 3 different views showing the same element with 3 different labels.
This native implementation in Archi takes care of this: label is similar to color, font...: it is on a per view basis.

I'd like to request that the HTML report can also support this naming mechanism

That will work by design because HTML report use images created using the view rendering mecanism

Which Herve's plugin doesn't.

That's strange, but that's perhaps because Herve uses some very specific tricks for his plugin to work.

it is on a per view basis.
Yay!

That will work by design
Double Yay!

Double Yay!

Have another Yay!

Also, I want to again recognise and respect @herve91 for the work he has done on this in his Specialization plug-in.
He was at a disadvantage when he first developed his plug-in because there are no easy "hooks" to set the text (or icons) in Archi's code so he had to resort to analysing the code and come up with workarounds. That was not an easy task, and I know from personal experience that Eclipse/Archi code is very complicated. A major undertaking!
We have the advantage now of building on Herve's work to try and produce a "native" solution. This is not meant in any way to ride over Herve's work. We are building on it! Thanks!

As we are releasing the initial version of this with Archi 4.7 EA3 let's document what we have so far:
So far, Text Expressions apply to Folders in the Model Tree and graphical objects in ArchiMate Views. They can be edited in the "Display" tab in the Properties window. Text Expressions for a Folder apply to the Folder's child objects.
${name} Name
${documentation} Documentation
${content} Text content of a Note
${type} Type of object
${viewpoint} Viewpoint name (can only be set on an ArchiMate View Folder in the Model Tree and applies to its child View nodes)
${strength} Applies only to ArchiMate Influence connections and displays its Strength, if set
${linkeddoc} Applies only to Notes. Display the Documentation of the first connected ArchiMate concept in the View
${properties} Display all Property key/values in a list like "key: value"
${propertiesvalues} Display all Properties' values in a list
${property:key} Display the value of the Property whose key is "key"
${property:key} Display the value of the Property whose key is "key"
${property:model:key} Display the value of the Property in the root model whose key is "key"
${property:view:key} Display the value of the Property in the parent View whose key is "key"
@jbsarrodie JB, can you provide a summary of your "filtered properties with separator" so I can add it to this list?

Note: while updating the list, I wonder is this should be named ${properties:separator:key} or ${property:separator:key} ? (after all, this simply extend ${property:key} for the case where the same key is used multiple times)

I think ${property:separator:key} to be the same. I can change it.

I think ${property:separator:key} to be the same. I can change it.

@jbsarrodie  It doesn't work if we use property because renderPropertyValue() will have already replaced it by the time it gets to renderPropertiesValuesCustomList(). So we need to either have order of precedence or use the original "properties" or something else.

or use the original "properties" or something else. And properties is already used to list all values. So all expression tags need to be different.

There's no conflict with ${properties} because regexps are mutually exclusives, while I agree there is some conflict with ${property:key} because we cannot exclude that key might contain : (actually, all properties used by Jasper contain it).
So It might be best to simply keep it as it is (ie. ${properties:separator:key})

There's no conflict with ${properties}

You're right, my bad.

So It might be best to simply keep it as it is (ie. ${properties:separator:key})

Yep

I can fix "\n" not rendering as a newline, too, so it can be used as a separator.
Are we OK with the feature name "formatExpression" - this is how it is stored in XML
We agree to use multi-line? I can get rid of single-line code, then

Embedded expression is something I think @herve91 supports.

Add a property in the model, lang = en
Add properties to an object like en =  English Name, fr =  Nom Francais
Add an expression to the object ${property:${model:property:lang}}

Which evaluates to "The value of property whose key is the value of the model's property whose key is lang"

I can fix "\n" not rendering as a newline, too, so it can be used as a separator.

Would be good

Are we OK with the feature name "formatExpression" - this is how it is stored in XML

Why not use "labelExpression"  as this is about label ?

We agree to use multi-line? I can get rid of single-line code, then

Of course yes

OK with "Display" on the tab? OK with it under Main tab?

I would suggest to rename it to "Label" and move it after analysis because Main, Properties and Analysis are related to the underlying concept so we want to keep them together, then start adding "Label" which will also be available for folder, and then view only specific tabs like Appearance and Figure.
BTW, there's something strange that I noticed on Windows and Linux: the string "Display" has some space before, is not aligned with other tabs' label and has a flat background. Same is true for Figure tab:

BTW, there's something strange that I noticed on Windows and Linux: the string "Display" has some space before, is not aligned with other tabs' label and has a flat background. Same is true for Figure tab:

These are sub-tabs. "Figure" is a sub-tab of Appearance. This way they can be grouped.

Why not use "labelExpression" as this is about label ?

We are using it also for folders in the tree. I guess that is a label. I think I was trying to aim for something neutral in case we have a case where it is not used for a label, although I don't know what that case would be.

These are sub-tabs. "Figure" is a sub-tab of Appearance. This way they can be grouped.

Do we really need this, I find it counter intuitive and annoying because we can not hide/show such group. Would it break something if they were normal tabs ?

We are using it also for folders in the tree. I guess that is a label. I think I was trying to aim for something neutral in case we have a case where it is not used for a label, although I don't know what that case would be.

I think this will always be a label, and if someday we have something else, I guess we'll be able to add another feature.

Would it break something if they were normal tabs ?

Nope.
We can have this:

Or even this:

At first I would had voted for the second option, but now that we can also set a label for folders' content I think it is better to keep the "Label" tab separated.

At first I would had voted for the second option, but now that we can also set a label for folders' content I think it is better to keep the "Label" tab separated.

Good point. I also think it looks crowded.
I've committed the changes if you want to test.

I've committed the changes if you want to test.

Just tested and everything is fine. My only comment (but this will be for EA4) is that when we use  ${linkeddoc}, the note content is not refreshed when the linked element documentation changed. But that's a minor point for the moment.

the note content is not refreshed when the linked element documentation changed.

This will not be easy to solve.
To remain lightweight, every EdiPart only listens to eCore notifications from its contained IDiagramModelComponent and its Properties and Features (see LightweightEContentAdapter).
In order to respond to another objects' eCore notifications it would also have to know which object is connected via a connection, track when those connections and linked objects were added and removed, and register another adapter to listen to its changes. and de-register the adapter if the other object is deleted. This kind of breaks the whole EditPart listener paradigm and would be difficult to implement.

But let's release EA3 today and think about that issue. The Workaround is to either close and re-open the View or delete and undelete the Note or the connected object.
@jbsarrodie OK for EA3 now?

OK for EA3 now?

Of course yes !

Reminder to me to look at this:

Label set on folders are really great but we have to set them on each folder. Would it be possible to define some inheritance so that setting a label on a top level folder would apply anywhere below, unless another label is defined (in this case ${name} would allow to reset to default behavior).

Label set on folders are really great but we have to set them on each folder. Would it be possible to define some inheritance so that setting a label on a top level folder would apply anywhere below, unless another label is defined (in this case ${name} would allow to reset to default behavior).

I've implemented this now.
Caveat - Model Tree label rendering is already an expensive operation if there are many tree nodes expanded. This will add to the expense. If users are sensible about how many nodes are expanded it should be OK.

the note content is not refreshed when the linked element documentation changed.

This is implemented now. It also updates if the note's connection is added/removed/reconnected.
In order to do this the Note's EditPart has to register as a listener for the whole model, and this can be expensive. As long as we don't have too many special cases like this we should be OK.

Great feature in EA3. I'd like to offer some feedback.
When trying to use it, one issue is that there is no visual clue to separate the name from other info like documentation. I'd like to have some kind of formatting instruction in the expression language, so I can put for instance the name in bold +2 font size, documentation normal and properties in italic.
I would suggest a subset of html or markdown, as both would be easy (?) to translate for a Rich Text Format like display box in Eclipse and other output: HTML Report, Markdown report, etc
In the worst case, a text filter removing the special tagging would allow to export the label as simple text.

@rchevallier I'm afraid supporting different fonts isn't possible in this implementation without a considerable re-write of the the diagram figures. A diagram figure contains one text box figure and that can only be set to one font. To support multiple fonts would require a different text box for each font.

Thanks for your quick reply. Not familiar with Eclipse framework, I suppose the EMF doesn't support using the org.eclipse.swt.custom.StyledText widget, which seems to allow such mix. :-(

It's not EMF, it's Draw2D figures. No you can't embed StyledText.

Just tried this new version - looking good.
Q1) As a heavy user of the specialisation plugin, I'll need to migrate the contents of the label property into the new Label Expression field. Is there (/will there be) a jArchi way to reference the expression?
Q2) I see that the specialisation plug-in works alongside, so I can still replace icons. Are you looking to add that into the main tool.

Is there (/will there be) a jArchi way to reference the expression?

Not immediately. The specialisation plug-in stores things in Properties which can be accessed by jArchi. Archi stores the label expression in a "feature" XML tag which currently can't be accessed by jArchi because we haven't written an API to do that yet. It's on the roadmap.

I see that the specialisation plug-in works alongside, so I can still replace icons. Are you looking to add that into the main tool.

We are looking at using a different approach for icons, perhaps with Font Awesome or similar. We prefer not to use bitmap images as they don't scale well in the UI or when exporting in SVG. Again, this is something on the roadmap.

Is there (/will there be) a jArchi way to reference the expression?

Update: Yes there will be a setter/setter for the label expression. I just implemented it. This will be released at the same time as Archi 4.7 (or you can compile the jArchi code in branch "dev")

Is it (or will it be possible) to reference the name of the view and the name of the view's folder as a label expression?
Very useful for creating banners / headings at the top of the view.  tia. Guy

Is it (or will it be possible) to reference the name of the view and the name of the view's folder as a label expression?
Very useful for creating banners / headings at the top of the view. tia. Guy

Yes, at least I am planning to contribute it as I often need some kind of banner too myself. The usal things needed here are: view name, view documentation and view properties.
@Phillipus I can see two ways of implementing this:

Either by extending ${property:view:key} to support non property keys like name, documentation, viewpoint and folder,
Or, add a more generic way to specify that some macro has to run in a different context. Something like ${view:any_other_macro}, or ${model:any_other_macro}. This could be implemented by passing the right object to underlying macro evaluation code. Only issue I can see is in the parsing which must be able to extract any_other_macro while this will obviously contain '}'.

Is it (or will it be possible) to reference the name of the view

We could have a prefix like these ones:
${model:name}
${view:name}
${folder:name}
I've done some POC code and it works.
@jbsarrodie What do you think?
A caveat - if a diagram object's label expression depends on the state of another object (view name, parent folder, model name, etc) then every object will have to register as an overall model listener and update its label depending on the event (rename, move View in the models tree, delete, etc)

Only issue I can see is in the parsing which must be able to extract any_other_macro while this will obviously contain '}'.

My regex Fu is not string enough for this.

My regex Fu is not strong enough for this.

My Google Fu is strong enough for this: https://stackoverflow.com/questions/17759004/how-to-match-string-within-parentheses-nested-in-java/17759264#17759264
This approach should work as we'll never allow infinite macro nesting.

I think Herve does something like this but uses a special regex library for named groups (I think).

I don't remember his approch, but this is something I did without regexp at all in one of my first C exercices when I was 18 (ouch). You basically define a counter (starts at 0) and read every characters. When you encouter an opening bracket, you add 1 to the counter, and when you encounter a closing bracket you substract 1 to the counter. When the counter comes back to 0, you've just found the last closing bracket so you know that everything before was part of the internal macro.

I've got the prefix: method working for name, documentation and properties (I've changed the position of the prefix on properties to the start, like name - ${model:property:key})
As for nested expressions this is something I mentioned here - https://github.com/archimatetool/archi/issues/576#issuecomment-604900208 - but if we chose this method I would definitely need a Regex Master to help out.

I've got the prefix: method working for name, documentation and properties (I've changed the position of the prefix on properties to the start, like name - ${model:property:key})

Good. This is the easiest way to implement and obviously it's up to the ITextRenderer to support it or not. If we go this way (and I think it makes sens), then I would suggest another naming scheme: $model{something} for model level expressions, $view{something} for view level expression, and of course ${something} for concept level expressions. I think it will be easier to read/understand in the long term and I think regex will be easier to parse. What do you think?

We really need to get the value of one expression so we can feed it into the next expression, for example to get the value of a property in the model to use as the key for another local property (https://github.com/archimatetool/archi/issues/576#issuecomment-604900208)

We really need to get the value of one expression so we can feed it into the next expression, for example to get the value of a property in the model to use as the key for another local property (#576 (comment))

There are several ways of doing so, one of the easiest (I think) is to add a loop in TextRenderer#render() which continue until there is no more $...{something} matching. So safe guard must be added though, or else setting both name and label to ${name} will lead to infinite loop.

I've committed the changes needed for our first option - prefixes of  model:, view:, and folder:. We don't have to use this method, but it's worth having it as a fallback.

Regarding the folder: reference:
Which folder for item 1 below?

Select an ArchiMate concept node or connection in a View. The folder can be the container for the ArchiMate concept or the folder container for the current View.
Select a non-ArchiMate node or connection (Note, Group, Line). The folder can only be the container for the current View.

So maybe there is in fact two folders: mfolder for concept's folder in model tree, and vfolder for view's folder.

So maybe there is in fact two folders: mfolder for concept's folder in model tree, and vfolder for view's folder.

That's what I thought, I was just hoping that no-one else would say it so I didn't have to do two... ;-)

mfolder and vfolder now implemented.

That's what I thought, I was just hoping that no-one else would say it so I didn't have to do two... ;-)

TBH I don't really like this idea neither, but that's the easiest way to do a quick POC. What I don't like is that this might end up implementing jArchi in label expression... We have to draw a line in the sand...

What I don't like is that this might end up implementing jArchi in label expression... We have to draw a line in the sand...

Not sure what you mean...?

Not sure what you mean...?

What I don't like is that this might end up implementing jArchi in label expression...

We don't want to implement any possible way to query the model from a label expression, or else we'll end up duplicate jArchi code.

We have to draw a line in the sand...

We have to fix a limit about what can be done or not (ie. manage expectation). ANd also a reference to a book we both read.

Surely the only thing jArchi needs to do is get/set the feature containing the label expression?

Surely the only thing jArchi needs to do is get/set the feature containing the label expression?

I mean: we first implemented expressions to query name, doc and properties for current concept, Then we added a way to query same information from model and view containing this concept, now we discuss doing so for folder, then model folder vs view folder. Next step will be navigating relationships (TBH I even have a use case for this). So where should we stop ?

So where should we stop ?

I think we have enough for now with model, view and folders.
I think, though, that users will compare our implementation with Herve's and, well, you know... ;-)

My concern with a label referencing other objects and their name, documentation, properties etc (model, view, folder, connected notes) is the cost of updating the objects in a View when a change in other objects occurs.
Now, an EditPart/Figure registers itself as a listener only to its own model object. So a rename action or fill color action will refresh the figure in the view.
With the label renderer also able to display properties and documentation it has to listen and react to those events too. That's not too bad because the event only concerns itself.
But with references to model:, view:, mfolder:, vfolder (and the connected note to show documentation) there are many events that could affect the label display - properties changes, name change, documentation changes, move object to a new folder, undo/redo etc.
In order for an EditPart/Figure to refresh its label on such an (unrelated object's) event it would need to register itself as a listener for the whole model. And once it receives notification of any event then decide if that event will require a label refresh. The logic required to determine whether the notifier is indeed referenced in the label expression might actually be slower than just updating the label in response to any model change. Either way, it's a very expensive process and will greatly impact performance.
Consider the case of several Views open at the same time with hundreds of graphical objects each one doing this. Performance will take a big hit.
The alternative is to make it clear that changes to other objects might require a manual refresh of the View - i.e closing and opening it. I am strongly against having a "Refresh" menu item - something that the user shouldn't really have to do and a case of the "tail wagging the dog".
So the options are:

Have very expensive listeners refreshing graphical objects on every model change.
User manually refreshes by closing/opening affected Views.
Have a "Refresh" menu item for a View :-(

So the options are:

Have very expensive listeners refreshing graphical objects on every model change.
User manually refreshes by closing/opening affected Views.
Have a "Refresh" menu item for a View :-(

Refresh a view when it gets focus back (after having switched to the model tree or another view) ???

Refresh a view when it gets focus back (after having switched to the model tree or another view) ???

That would be like a hidden refresh that user wouldn't have to care (too much) about

Refresh a view when it gets focus back (after having switched to the model tree or another view) ???

Nice idea but that might also be quite expensive. But I'll investigate...

Refresh a view when it gets focus back (after having switched to the model tree or another view) ???

@jbsarrodie This is working quite nicely, good idea. This is what I've got:

An AbstractDiagramEditor is already a model listener and has a setFocus() method so it's already equipped to handle this situation.
When a model change occurs, set a changed flag
When setFocus() is called and changed is true - iterate through all EditParts/Figures and if the IDiagramModelComponent has a label render feature set the refresh its visuals (set changed to false)

A stress test of 1200 objects in a View that have the label feature shows that it takes, at least on my average Windows box, about 60 ms to refresh the figures.

An AbstractDiagramEditor is already a model listener

Note - I already tested refreshing each EditPart/Figure in response to every model change event and that's a no go, very sluggish.

An AbstractDiagramEditor is already a model listener

Note - I already tested refreshing each EditPart/Figure in response to every model change event and that's a no go, very sluggish.

In this test, was it done only on the view having the focus or on all opened views ?

In this test, was it done only on the view having the focus or on all opened views ?

On the view with the focus. The problem is not the time it takes to refresh the figures in response to one model change event but in the case where there are multiple model events (for example, deleting several objects, moving several objects and other cases where EMF sends out multiple events)

In this test, was it done only on the view having the focus or on all opened views ?

On the view with the focus. The problem is not the time it takes to refresh the figures in response to one model change event but in the case where there are multiple model events (for example, deleting several objects, moving several objects and other cases where EMF sends out multiple events)

Ok, so that's a no go for this idea. But I'm happy to read it works well when the view gets the focus back ;-) I think that's a good compromise: for expressions we implement as part of Archi ourselves, we might take care of listenening to events, but for some edge cases and for contributed plugins, this mecanism should be enough.
Just one comment though: we should also make sure that exporting the view to an image (from the menu, the CTRL+SHIFT+C or jArchi) also refreshes the view before actually doing the export.

The refresh problem occurs only if the label expression references another object with model:, view:, xfolder:, or a connected Note with ${linkeddoc} and something changes those objects' state.
What we really don't want to do is listen to every model event and react to it on the off-chance that it affects an object and requires a UI update. And computing whether an event concerns the object's label expression is too complicated and time-consuming (in fact, would be quicker just to update the object anyway).

What we really don't want to do is listen to every model event and react to it on the off-chance that it affects an object and requires a UI update. And computing whether an event concerns the object's label expression is too complicated and time-consuming (in fact, would be quicker just to update the object anyway).

I agree. So this means that you'll roll back the changes you've done first to update notes making use of ${linkeddoc} if I understand well ?

So this means that you'll roll back the changes you've done first to update notes making use of ${linkeddoc} if I understand well ?

Partly. That was a bit of hack - "If the Note has connections AND the event was Documentation changed AND the Note had a ${linkeddoc} Feature" but that still didn't check if it was a Documentation change on the connected object.
A lightweight solution for a Note is to refresh the text if a connection changes and the setFocus() for anything else.
I'm still experimenting, but I hate having expensive event listeners (the code in the Models Tree to handle model events is an example of what a nightmare this can be).
Having said that, we do a pretty good job in Archi of being lightweight in response to model events. Other tools I've seen have a "fuck it" philosophy and a refresh button. ;-)

A lightweight solution for a Note is to refresh the text if a connection changes and the setFocus() for anything else.

Would it be possible (in general) to update the label of an object (is an expression is defined) each time something in this object is changed (including any visual property), or each time its get focus back ? Together with the "view gets focus" trick, we should handle most cases.

Would it be possible (in general) to update the label of an object (is an expression is defined) each time something in this object is changed (including any visual property),

That's already implemented.

I'm experimenting with another trick.
The problem with reacting to all model changes is when there are multiple event notifications - let's say you delete 100 objects in a view you end up with 100+ change events.
So....

Listen to all model events
When the first event arrives, set a Timer object to do the EditParts update but after a delay of, say, 20ms in the Timer.
If another event arrives (in that 20ms delay), cancel the first Timer and start another one with the same 20ms
Goto (3)
Should be that only the last Timer event is called

This is the logic behind the keypress delay in the Search box, when refreshing the Models Tree in response to multiple key presses.

I'm experimenting with another trick.

Too slow if there are several Views open at the same time. Not good.

Just one comment though: we should also make sure that exporting the view to an image (from the menu, the CTRL+SHIFT+C or jArchi) also refreshes the view before actually doing the export.

The problem with that is those image export methods don't know anything about the AbstractDiagramEditor , they only access the underlying GraphicalViewer which doesn't contain the code to update the editor...

The problem with that is those image export methods don't know anything about the AbstractDiagramEditor , they only access the underlying GraphicalViewer which doesn't contain the code to update the editor...

But could they trigger an event caught by the current view to refresh itself (ok, now that I wrote it I find it strange...)

Actually, jArchi and the reports won't be affected because they create their own GraphicalViewer to create the View and recreate it each time from the current model.
So the only actions affected are actions for an editor - copy to clipboard and export as image. And these actions could be passed the AbstractDiagramEditor as a parameter instead of the GraphicalViewer so it's possible.

@jbsarrodie All of these workarounds are less than satisfactory, so let's test the performance of the ideal solution. I've committed into the text-render branch changed whereby each EditPart listens to the model and updates its own text if it has a label expression. Is it possible for you to do some stress testing?

Have many Views open
Have very many diagram objects with a label expression, such as ${model:name} ${view:name} etc (copy and paste of objects might be the way to do this)
Carry on - adding, deleting, renaming, moving things around in folders, deleting many objects, moving many objects, undo/redo etc etc...

I can see a slowdown with these conditions:

Create a View with a few objects in that have label expressions.
Copy and Paste these objects in the same View so that there are many objects with label expressions
In the Models Tree, select the View and Duplicate it (Ctrl-D)

As each View is duplicated opened it get slower as compared with normally.

Is it possible for you to do some stress testing?

Have many Views open
Have very many diagram objects with a label expression, such as ${model:name} ${view:name} etc (copy and paste of objects might be the way to do this)
Carry on - adding, deleting, renaming, moving things around in folders, deleting many objects, moving many objects, undo/redo etc etc...

I tested with more than 50 views opened (and these were big views).
I updated all object using this script:
$('archimate-diagram-model').each(function(v) {
	$(v).find('element').each(function(o) {
		o.labelExpression = '${name}\n(${type})\n${model:property:test}';
	});
});

And did some tests. This reacts well for single changes, but becomes damn slow each time you operate on lots of elements at the same time, like duplicating views (approx 5 time slower) or running the same script again (approx 10 time slower). So not acceptable IMHO.

So not acceptable IMHO.

I agree. My tests on duplicating views show a real slow-down. And it's a heavyweight approach.
The best option so far is the refresh when focus is gained (and a model event has previously) happened.
Rolling back to that version...

More research on auto-refresh of the Editor.
I can implement a lightweight listener for the Model and an object's parent View, so that would take care of cases where the expression uses model: and view:.
But I can't have a lightweight listener for mfolder: and vfolder: in the case where the object is moved to a new folder because it needs to listen to Add/Remove events on all folders.
Now, if we only supported model: and view: we would be OK... ;-)

Now, if we only supported model: and view: we would be OK... ;-)

Yes... but I have plan for navigating relationships. My use cases are:

I'd like to use the name or some properties from the concept which is the source of a composition relationship targeting current concept (ie. its parent in ArchiMate term)
I'd like to use the name or some properties from the application component which contains, is assigned to or realizes the current concept (ie. the application component of current interface, function, process or service)

So whatever we do we need at least some (not perfect) fallback (like refreshing the labels when the view gets the focus).

OK, if the rules are going to get more complicated then we would have to listen to the whole model. So the setFocus method is still preferred.

For reference, this explains why adding a model listener (an EContentAdapter) to each EditPart is a bad idea:
https://eclipsesource.com/blogs/2013/05/06/emf-dos-and-donts-8/

@jbsarrodie I've done a lot more experimentation on updating the Editors in response to all model changes, and today I took a new approach.
An AbstractDiagramEditor has a method commandStackChanged() which is called once when a model change occurs (in order to update menu items), rather than several times when an eCore change notification occurs. By hooking into this, then iterating thru all EditParts, and only refreshing the figures who have a label rendering expression this is quite fast. Please test when you get a chance!

 :-)
I bet you spent a lot of time on it...gave up....but couldn't let go! And came back at it...again...and again....I WILL NOT BE BEATEN!!...until...finally...it works.
I would never do that. ;-)

Advance warning - the text-render branch needs a clean-up so this means it will be temporarily deleted, squashed, cherry-picked, rebased, etc, etc and re-pushed. If you have any changes on top of the current branch please store them.

Advance warning - the text-render branch needs a clean-up so this means it will be temporarily deleted, squashed, cherry-picked, rebased, etc, etc and re-pushed. If you have any changes on top of the current branch please store them.

I've already saved than in a stash ;-)

I've already saved than in a stash ;-)

Thank-you for your social distancing measures, normal service can now be resumed. :-)

@jbsarrodie What do we need to do now in order to get this done? I'd like to get an EA4 out with our recent changes (inherited folders, refresh Views, mfolder, vfolder and other stuff).

I still have to work on ${wrap:max:text} and ${truncate:max:text}, but this can wait for EA5 (if you plan an EA5).

Yes, EA5 and even EA6, EA7...and then Beta. I'd like to keep keep the updates available to get more testing from users (especially things like refresh objects)

I still have to work on ${wrap:max:text} and ${truncate:max:text}

I guess those ones will have to work on the text last in the order of processing. We could ensure that they do that by calling them after we have processed registered ITextRenders

I guess those ones will have to work on the text last in the order of processing.

Yes. Maybe we could define something similar to the "stages" we have in Archi CLI? Each ITextRender could "hook" to a specific stage to make sure things don't get messy.
Something like:

Global Context: Expressions which are not related to the view (ie. model and folder)
Local Context: Expressions related to the view
Object Context: Expressions related to the object itself (other than name and description) or other objects related to the current one.
Name & Description: Expressions relying only to name and descriptions
Formatting: Expressions which doen't add up information but change the way it is rendred.

The table of expressions got lost somewhere in this mega-thread. So I've put them in the wiki:
https://github.com/archimatetool/archi/wiki/Label-Expressions

So far we have only two types of renderers that need to be in a given order:

All renderers that deal with replacing macros with the values of other things
Formatters that need to format the result of (1)

So far we have only two types of renderers that need to be in a given order:

All renderers that deal with replacing macros with the values of other things
Formatters that need to format the result of (1)

Yes, but "1." should be splitted depending on their scope. If we look at "Herve's LANG use-case", we want to get the value of a model property (Global Context) first because it become an input for an object property (Object Context). That's what my proposition for "stages" tries to address.

This will probably require some additional internal object handling if we are using the value of one object as the input of another, rather than the current simple find and replace string.

This will probably require some additional internal object handling if we are using the value of one object as the input of another, rather than the current simple find and replace string.

That's not exactly what I'm saying. My point is that this keep being only find and replace string, but we have to do it in the right order, or else we'll end up having loops (and I don't like this idea). So I'm simply trying to figure out a logical order to do these "find and replace string".
I'm assuming that:

80% of use-cases will be to create a label which is simply based on current object
15% of use-cases will be to create a label which is based on the current objet and model/folder information, but without dependencies between them.
4% of use-cases will be to create a label which is based on some current object's property which name is defined by the value of a model/folder property (e.g. use the value of a property named Label_LANG, with LANG being the value of the model property Lang). So this can be managed by replacing expression from the wider context (model/folder), then local context (view's properties), then the object itself.
1% of use-cases will want to mix everything in any order. I don't want to support these edge-cases.

So replacing expression in this order...

Global Context: Expressions which are not related to the view (ie. model and folder)
Local Context: Expressions related to the view
Object Context: Expressions related to the object itself (other than name and description) or other objects related to the current one.
Name & Description: Expressions relying only to name and descriptions
Formatting: Expressions which doen't add up information but change the way it is rendred.

...addesses this

OK, we need some example uses cases and then I can experiment with ordering.

Early Access 4 is now available - https://www.archimatetool.com/beta/

I wonder if ${linkeddoc} could become more generalised to ${linked:documentation} and ${linked:name} and ${linked:property:key} but using the same rules as before - "linked" is the first item linked to a Note. Or is this over-kill?

I wonder if ${linkeddoc} could become more generalised to ${linked:documentation} and ${linked:name} and ${linked:property:key} but using the same rules as before - "linked" is the first item linked to a Note via a connection. Or is this over-kill?

I've implemented it and it works nicely. Useful or not?

I wonder if ${linkeddoc} could become more generalised to ${linked:documentation} and ${linked:name} and ${linked:property:key} but using the same rules as before - "linked" is the first item linked to a Note. Or is this over-kill?

This might be useful and I think it is very similar to one of my previous comments:

If we go this way (and I think it makes sens), then I would suggest another naming scheme: $model{something} for model level expressions, $view{something} for view level expression, and of course ${something} for concept level expressions. I think it will be easier to read/understand in the long term and I think regex will be easier to parse.

Seems to me we are speaking about $linked{something}.
And if you remember, I also wrote:

I have plan for navigating relationships. My use cases are:

I'd like to use the name or some properties from the concept which is the source of a composition relationship targeting current concept (ie. its parent in ArchiMate term)
I'd like to use the name or some properties from the application component which contains, is assigned to or realizes the current concept (ie. the application component of current interface, function, process or service)

Seems to me we are speaking about $relationship{something} (with relationship being the type of a structural relationship, "structural" because this is usually used for 1:N relations): something applies to the source element of the first relationship of type relationship targetting current concept.
One advantage (IMHO) of this approach over ${model:something} or ${linked:something}, is that it is easier to read and understand because the part between $ and { becomes the scope, and the part between {and } becomes the expression to evaluate (based on the scope) and never change.

OK, I'll commit it as is (and it also fixes a couple of other issues)  using the linked: prefix for now. And then we can work on the $linked{something} syntax..

I would suggest another naming scheme: $model{something} for model level expressions, $view{something} for view level expression, and of course ${something} for concept level expressions.

That's now implemented. @jbsarrodie Can you test it?

This is a great work you have done.
Just had it for a quick tour only thing that I cannot work out right now is the ${linkeddoc} where should that one be applied and how?
I'll sure can find use for this feature and combined with jarchi it's kind of making this a darn great tool compared to the commercial alternatives.

Just had it for a quick tour only thing that I cannot work out right now is the ${linkeddoc} where should that one be applied and how?

Add ${linkeddoc} to a Note's label expression
Connect the Note to an Element that has some documentation

Note - in Archi 4.7 EA 5 (next version) this will change to $linked{documentation}

@jbsarrodie I find documentation too long to type. I can make the regex handle both documentation and doc. What do you think?

I find documentation too long to type. I can make the regex handle both documentation and doc. What do you think?

I agree that it's a bit long. But OTHO that's how it is called also in jArchi and we never found it too long in this context, and properties is not much smaller, so maybe it is best to keep it explicit (thus documentation)

I will certainly keep documentation but we can also have doc as a short version as well. We can keep it a secret if you like. ;-)

Thanks guys, forgot about the property of the 'plain' connection as being able to connect any two items.
+1 for doc as a abbreviation for documentation :-)

Would it be that I can access the label from jarchi?

Would it be that I can access the label from jarchi?

There will be an API:
object.labelExpression = "${some_expression}"
and
var expression = object.labelExpression
and also some APi to get the result of the expression (not done yet)

Sounds great, would open for using jarchi to apply both property and label setting in one go based on the user input from a dialog.
Thanks for your great work.

There will be an API:
object.labelExpression = "${some_expression}"
and
var expression = object.labelExpression
and also some APi to get the result of the expression (not done yet)

I think we should also add an object.label property to get the result of the expression in one go.

I think we should also add an object.label property to get the result of the expression in one go

Sure. Or name it something like object.labelValue. So that it basically does what the Figures do to retrieve the result of the expression. The other two methods are still needed to get/set the actual expression.

Seems to me we are speaking about $relationship{something} (with relationship being the type of a structural relationship, "structural" because this is usually used for 1:N relations): something applies to the source element of the first relationship of type relationship targetting current concept.

So could we combine $relationship{something} with $linked{something} to generalise a rule?

Use the name of the relationship or connection (like composition, realization or connection) - example - $composition{name}, $connection{documentation}
Refer to the source object of the first matching relationship/connection

Why source object of connection? What about target? Maybe specify this in the first part of the name like connection_source? Maybe not?

So could we combine $relationship{something} with $linked{something} to generalise a rule?

I've committed a change.

Removed $linked{something}
Now we have $composition{something}, $realization{something}, etc (all relationship names lower case) and $connection{something} for a plain connection
The source object of the connection is the supplier

This fulfils the job of using the connected source object but that is not clear from using the relationship name when really we mean $composition_source_object{something}

Why source object of connection? What about target? Maybe specify this in the first part of the name like connection_source? Maybe not?

My first though was that this is obvious for structural relationships (usually one source element and multiple targets), but this is not always the case.
Maybe we could use $s_relationshipname{expression} to get the source of the relationship and $t_relationshipname{expression} to get the target ?
Or maybe it should be $source{relationshipname, expression} and $target{relationshipname, expression} ?

I quite like $source{relationshipname, expression} - I'll see how easy that is to implement.
Do you think that users will use the source/target connection object much? What are the use cases? (just curious)

Do you think that users will use the source/target connection object much? What are the use cases? (just curious)

One example: when modelling applications, we often create lots of application functions and services, but some of these functions and services have a generic name (manage users, generate reports...). In order to easily find and distinguish then people (including Gerben) add a prefix which is a short version of the application's name, leading to things like "[SAP] Manage Users", "[SAP] Generate Reports", "[AppX] Manage Users", "[AppY] Generate Reports"...
I don't really like having hard coded things in names, thus using a label expression seems better and could lead to [$source{assignment,${property:Short Name}}] ${name}

$source:relationshipname{expression} and $target:relationshipname{expression} is easier to implement.

I've gone for relationshipname:source{expression}, relationshipname:target{expression}

I've updated the wiki https://github.com/archimatetool/archi/wiki/Label-Expressions and released EA5 with the new syntax.

I've figured out a way to implement nested expressions for properties so that a property key can be derived from the value of another property value.
An example:

The model has a property lang with value en
An object has two properties en -> English and fr -> Francais
Set the object's label expression to ${property:$model{property:lang}}

The label will display "English"
If the model property lang is set to fr then the object will display "Francais"
The expression ${property:$model{property:lang}} decomposes to:
$model{property:lang} = en
And so becomes ${property:en} which becomes "English"

I've figured out a way to implement nested expressions for properties so that a property key can be derived from the value of another property.

Stages ?
This "simply" requires to replace $model before $views before other expressions... no ?

It just parses the nested expression first.

Added $source{expression} and $target{expression} for connections.

Stages ?
This "simply" requires to replace $model before $views before other expressions... no ?

After some more experiments (and another commit that ensure that the renderers are registered in a set order) I see what you mean.
Yes, if we process all other renderers before the PropertiesRenderer then a nested expression (${name}, ${documentation}, ${type} etc) will be rendered first before being processed by PropertiesRenderer.
For example:
${property:$view{name}}
NameRenderer will evaluate $view{name} first to result and then PropertiesRenderer will evaluate ${property:result}
In this example:
${property:$model{property:lang}}
PropertiesRenderer will evaluate the nested $model{property:lang} by calling itself (recursive).

You can even do this:

An object has these properties:

en -> Yes
fr -> Oui
lang -> en

The expression in the object is this:

${property:${property:lang}}

So it will display "Yes".
If you change en to fr it will display "Oui"
Bon, non?

@jbsarrodie I've discovered a trick (side-effect?) of embedding property values. I can make it so that we can have re-usable macros.

Define a property in the model - macro1 -> ${name}\n${documentation}
Reference this property elsewhere in various objects to display that object's name and documentation - $model{property:macro1}
Edit the model property once to have that macro update wherever it is used

One could set up various macros for different objects.
Useful?

...the only problem with that is this:
java.lang.StackOverflowError
If there's a circular reference.

I've discovered a trick (side-effect?) of embedding property values. I can make it so that we can have re-usable macros.

In fact I thought about it too and that's quite useful.
We can also do things like:

set a Default label property on the model with a value such as ${name} (${type})
set a Default label property on views with a value such as $model{property:Default label}
Set label expression on all concepts to $view{property:Default label}

This should allow users to globally manage labels and provide per view exceptions (in fact could be viewpoint) or per concept.
We could even imagine defining model level properties with a name based on possible stereotypes (e.g. Network Flow Label property containing ${property:Protocol} / ${property:Port}), and then setting a standardised label expressions on objects (e.g. $model{property:${property:Stereotype} Label} on a Flow relationships having a property Stereotype containing Network Flow)

In short, that's the basis for very powerful customization features. And we'll be able to leverage this even more when we'll have rework the way viewpoints work and added a real Stereotype attribute.

set a Default label property on the model with a value such as ${name} (${type})
set a Default label property on views with a value such as $model{property:Default label}
Set label expression on all concepts to $view{property:Default label}

Yes, that works. (I've just committed the code to do that.)
This is indirection at more than one level: A -> B -> C. In this case PropertiesRenderer recurses twice (for B and C).
My problem is avoiding a circular recursion: A -> B -> A
I have set a counter in the PropertiesRenderer to allow only n levels of recursion. But I can't figure out a way to actually test for circular recursion. Any ideas on that?

But I can't figure out a way to actually test for circular recursion. Any ideas on that?

I have an alternative way to handle this:
Each time PropertiesRenderer retrieves a value from a Property, it stores that Property in a list. The next time a Property is retrieved if it is in that list then we have a circular recursion. At this point we could return a text value of something like this:

Or display nothing at all? Or some other behaviour?

At this point we could return a text value of something like this:

This seems the best option IMHO. It is simple and quite easy to understand for the user. Maybe use *** Recursion Error in Label Expression *** to be a bit more precise.

OK, I've committed that now.
This means that we can have property value references with no limit now (although realistically they shouldn't be too deep).
Note - as we are using text replacement, a property reference has to use absolute, not relative, expressions. For example here are some properties set in the model:

And referenced from another object as $model{property:p1} evaluates as "Got there!"

There's a bug in this. It doesn't work on this:
${property:key} ${property:key}

@jbsarrodie The regex in PropertiesRenderer is wrong. In the above case, group 2 (the property key) is evaluating to:
key} ${property:key
The regex is "\\$" + allPrefixesGroup + "\\{property:(.*)\\}"
Group 2 is (.*) so should be something else. Not sure what, though...

Group 2 is (.*) so should be something else. Not sure what, though...

Try ([^}]*)

Try ([^}]*)

Almost,
But it doesn't work with this one - ${property:$model{property:key}}. That renders as }
🆘

But it doesn't work with this one - ${property:$model{property:key}}. That renders as }

That's normal because this is recursive. This can only work if $model{property:key} has been evaluated first, thus the idea of stages. But this is not a silver bullet and has some limitations (especially if we want to use a model or view level expression using the return of an object level expression...

Hmmm...will need some help on this then. Ideally it would be great to have a working regex so that the recursion works. Parsing and looking for nested expressions might require a more substantial re-write.

Everything works except for PropertiesRenderer since I changed the regex in an attempt to support nested expressions.
A Properties expressions works if it is on its own line

This currently works:
${property:p1}
${property:p2}

This currently does not work:
${property:p1} ${property:p2}

Ideally it would be good to devise a regex for PropertiesRenderer that could handle this. I've failed to do so.
Therefore it might be possible to pre-process the expression string and look for ${property... expressions and render each one first. The aim is to simulate each property expression being on a new line.

I'll look at it later today...

This currently does not work:

It works for me. Can you share a really small model for which it doesn't work for you?

Have you got the latest code?

Have you got the latest code?

I thought, but in fact no. I can reproduce the bug now. I'll look at it tonight.

I'll look at it tonight.

I think I have an idea to solve this (I'm making good progress). I'll continue working on it tomorrow.

Just a note regarding using different fonts in a figure.
It is possible but would not be easy to implement and might be expensive in terms of time to render.
The main parent Draw2D box we use is a FlowPage. This can contain more than one child which can be of type BlockFlow. This in turn can contain more than one TextFlow. Each child can have its own font.
In our implementation we use just one of each, therefore we have only one font. See AbstractTextControlContainerFigure#createTextFlowControl()
To achieve multi-font would involve dynamically creating multiple BlockFlow and TextFlow child figures and removing them as required when the rendered text changes. This would be expensive, and I think quite complicated to implement.
What we might consider for a future update is not to have inline font markup in the label expression but have 2 or three text areas in the figure and allow a font for each.
Note - this would be hard, at least for me. But...we're open source, so this would be a perfect opportunity for all of those open source contributors clamouring to help! ;-)
See https://github.com/eclipse/gef-legacy/blob/master/org.eclipse.draw2d.examples/src/org/eclipse/draw2d/examples/text/TextFlowExample.java

I think I have an idea to solve this (I'm making good progress). I'll continue working on it tomorrow.

@Phillipus I've commited what seems a solution to this issue. I've tested with really ugly expressions and it works, but more tests are needed.

Thanks, man. I'll check it properly tomorrow ( 😴  )
Quick question - what's the reason to move visitedLabels' into TextRenderer`? It only really affects the nested Properties renderer.

@jbsarrodie A quick test shows your solution is certainly working better than what we had before! I'll carry on checking tomorrow...

Quick question - what's the reason to move visitedLabels into TextRenderer? It only really affects the nested Properties renderer. I'm sure you have a good reason, just curious. ;-)

This seems the best way to manage recursion in a very generic way. So if someone adds new renderers which also takes arguments, it will still allow any combinations. At least that's my idea.

BTW: to understand lookahead regexp: http://www.regular-expressions.info/lookaround.html

I've tested with really ugly expressions and it works, but more tests are needed.

I've added a bunch of Unit Tests that test all of these renderers. So far the new PropertiesRenderer is passing all tests. We can add more tests as required if you can think of any additional ones.

@jbsarrodie I like your idea here:
Set<String> visitedLabels = new HashSet<String>();

do {
   visitedLabels.add(result);

   for(ITextRenderer r : renderers) {
      result = r.render(object, result);
   }
} while(!visitedLabels.contains(result));

@jbsarrodie I like your idea here:

That's your idea, I stole it ;-)

That's your idea, I stole it ;-)

And improved it. Your idea is to iterate through the renderers and store the latest rendering result until done. This is more generic rather than just for properties.

...although it should be called resultSet not visitedLabels.

I think it can be optimised. This seems to work:
            for(ITextRenderer r : renderers) {
                result = r.render(object, result);
                if(!result.contains("$")) {
                    break;
                }
            }

This will stop processing if no more expressions. I might have missed something here, though.

This will stop processing if no more expressions. I might have missed something here, though.

Won't work if a name or doc contains '$'...

Won't work if a name or doc contains '$'...

In that case it will continue to iterate thru the renderers. Aim is to optimise for most cases. For example with ${name} the text is rendered to The Name in the first iteration of the loop. Then  all renderers are called twice.

I've just discovered a problem.
If an object's expression is ${name} and the object's name includes the expression ${name} it gets stuck in an infinite loop.
I know this is unlikely, but it could happen if, like me, you're curious to know what would happen...

In that case it will continue to iterate thru the renderers. Aim is to optimise for most cases. For example with ${name} the text is rendered to The Name in the first iteration of the loop. Then all renderers are called twice.

I thought it was instead of do/while loop. If it's innaddition then yes, this works.

If an object's expression is ${name} and the object's name includes the expression ${name} it gets stuck in an infinite loop.

Why, this should not happen because now we can't iterate twice on the same result... Unless there is also some recursion or loop inside the NameRenderer ?

If the object's expression is ${name} and the object's name is ${name} Something then the text is rendered in each do/while iteration as:
${name} Something
${name} Something Something
${name} Something Something Something
${name} Something Something Something Something
etc...

We might need to parse the text content first and remove any macros.

Or escape then so that they can be used (makes sens in a documentation about ${documentation}) but are not converted.

would you have an example for a note linked , I can't find it ?
I would like the note to show the documentation of the linked object

would you have an example for a note linked , I can't find it ?
I would like the note to show the documentation of the linked object

Add this expression to the Note:
$connection:target{documentation} or $connection:source{documentation} depending on the way the connection is connected to the other object (from or to it).
I've updated the wiki - https://github.com/archimatetool/archi/wiki/Label-Expressions

does not seem to work in EA3

does not seem to work in EA3

No, it was implemented in EA5
https://www.archimatetool.com/beta/

Works in EA5 ! thx

I still haven't solved the problem of infinite recursion. Here's a summary of the main points:

If each renderer does its rendering just once there's no problem
Because we want to use an expression as the value of a property (to reference another property) we have introduced recursion
The effect of this is that it's possible to put any expression in an object's name, documentation, property value etc and these will also be evaluated
This can lead to infinite recursion
We have tried a technique of keeping a list of each evaluation so we know when to break the loop. However, this doesn't work as explained in my comment above

So how do we solve this? Some suggestions:

Only recurse on properties? (As it was before)
Pre-parse the text/documentation fields and remove expressions? But we don't want to do that on Properties

So how do we solve this? Some suggestions:

Only recurse on properties? (As it was before)
Pre-parse the name/documentation fields and remove expressions? But we don't want to do that on Properties

3. Add a counter and allow only n loops at max when running text rendereds (I suggest n=10)

Add a counter and allow only n loops at max when running text renderers (I suggest n=10)

OK, that kind of works but still leads to 10 iterations of ${name} Something Something Something Something... in the example above.
Question - should we be allowing label expressions like ${name} and ${documentation} in the text fields?

OK, that kind of works but still leads to 10 iterations of ${name} Something Something Something Something... in the example above.

Not if we change the label to *** Too many recursion *** when reaching the limit.

Question - should we be allowing label expressions like ${name} and ${documentation} in the text fields?

I'd say yes because it could make sens in some occasion to use this kind of placeholder

Add a counter and allow only n loops at max when running text renderers (I suggest n=10)

Not if we change the label to *** Too many recursion *** when reaching the limit.

I've committed changes that checks for this. It works, but my intuition tells me there must be a better way to do this... ;-)

I've updated the wiki - https://github.com/archimatetool/archi/wiki/Label-Expressions
And released EA6 - https://www.archimatetool.com/beta/

While using Archi 4.7 EA5, I noticed that when a lebel expression is set on a model tree folder, objects still get sorted by their name and not their label. Is it manageable to always sort by what is actually visible (label or name depending on context) ?

While using Archi 4.7 EA5, I noticed that when a label expression is set on a model tree folder, objects still get sorted by their name and not their label. Is it manageable to always sort by what is actually visible (label or name depending on context) ?

Yes. However, the tree will need to be manually refreshed to show the new sorting if the folder's expression references another object and a value of that object changes. Because the tree is slow when updating, we cannot update it on every single model change.

The Tree is registered as a model change listener and the listener code is heavily optimised to only update the tree nodes that it has to - on name change, delete, add, etc. If one object changes only that one object's tree node is updated, or its parent folder if the name changes, or a relationship if one of its ends changes. Because calling the refresh() method on the tree is so expensive and potentially slow if there are thousands of tree nodes, we can manage this.
However, with label expressions applied to a folder and affecting its children, the model notification is now a bit more expensive in computing whether the parent or ancestor folder of a node has an expression. Also, the Tree listener now has to react to documentation and properties changes on the folder's child objects in case of using ${documentation} and ${property:key}
So far, so (relatively) good. There is an added overhead, but if the user doesn't use too many label expressions in the Tree this is manageable.
However, if the Tree folder expression now references other objects like $model{name} or $model{property.key} or some other objects then when the value of these other objects changes (rename the model, for example) because the Tree listener is not updating all nodes in response to this notification the nodes affected by the label expression will not update. To do that, one would have to update every Tree node label in response to every model notification in case it was affected. And that would make the Tree slow down to a crawl.
This was the same problem we had with Views, except that was do-able because it was quicker just to update all View objects. But the Tree is a slug.

What does "manually" refreshed mean?

Close and re-open the Tree. Add a sibling node and undo. Rename a node and undo. Delete something and Undo.

Just tested the https://github.com/archimatetool/archi/wiki/Label-Expressions feature in 4.7.0
Build<PHONE_NUMBER>30. Looks great! Found one issue with $view{properties}. It does not expand to all the properties of the view (with their key in front of them). I would expect the $view{properties} to show that, similar to ${properties} for all the properaties of the object. But that doesn't seem to work (it does not expand). If I manually add all the keys in the Label Expression it works, e.g.
key1: $view{property:key1}
key2: $view{property:key2}

That seems to be broken. In fact these are not working with any prefix:
${properties}
${propertiesvalues}
${properties:separator:key}

@jbsarrodie Could this be a side-effect of the new regex expression?

Actually, I just checked and prefixes on these has not been implemented:
${properties}
${propertiesvalues}
${properties:separator:key}

I'll see how hard/easy this is and take it from there,

I've implemented prefix support for these now. This will be in the next beta release.

I've just implemented word wrapping. Seems to be working fine, but as usual should be tested a bit more.
This now raises a point: connection text is always left aligned. This was not an issu until now because it was impossible to have multiple lines. But now it becomes possible and having a connection text left aligned is a bit strange:

I would suggest to either force it to center aligned, or to add ability to set text alignment (with center as the default).

I would suggest to either force it to center aligned, or to add ability to set text alignment (with center as the default).

Unfortunately, that's not possible without a re-write of AbstractDiagramConnectionFigure and sub-classes. It uses a org.eclipse.draw2d.Label which doesn't support text alignment. We'd have to replace it with some other text figure, re-write the dependencies of code that expects to see a Label, and also re-write the code for clipTextLabel()

I've done some experiments.
A TextFlow figure (this is also used in an element figure) as a label for a connection works. However, several parts of the code elsewhere will need to change because it expects to see a Label figure. The clipping hack works.
However, to support user setting the text alignment on a connection label IDiagramConnectionFigure would have to implement ITextAlignment which means changing the eCore model, regenerating and then changing the ecore model version to warn about non-backward compatibility.

A TextFlow figure (this is also used in an element figure) as a label for a connection works. However, several parts of the code elsewhere will need to change because it expects to see a Label figure. The clipping hack works.

Good !

However, to support user setting the text alignment on a connection label IDiagramConnectionFigure would have to implement ITextAlignment which means changing the eCore model, regenerating and then changing the ecore model version to warn about non-backward compatibility.

I think forcing center alignment when label is in middle position is more than enough. There's no need to make it configurable

As this is a significant change we'll have to test it - there's test branch connection-label-test - hard coded to centre alignment.

I think forcing center alignment when label is in middle position is more than enough. There's no need to make it configurable

OK, if you'd like to try the latest commits in connection-label-test branch.

I've just implemented word wrapping.

I will write some unit tests for it.

OK, if you'd like to try the latest commits in connection-label-test branch.

I did. That's quite good and it doesn't break anything in my models. That's ok for me.

@jbsarrodie I've started testing WordWrapRenderer (and committed some new changes to the branch) but the first problem is that it is prefixing all text with a new line. So ${wordwrap:6:Hello World} is rendering as \nHello\nWorld instead of Hello\nWorld

Possible fix:
if(!trimmedLine.toString().isEmpty()) {
    allLines.append(trimmedLine).append(LINEBREAK);
}

I've committed that possible fix and some more unit tests.

@jbsarrodie I've started testing WordWrapRenderer (and committed some new changes to the branch) but the first problem is that it is prefixing all text with a new line. So ${wordwrap:6:Hello World} is rendering as \nHello\nWorld instead of Hello\nWorld

This doesn't happen with my test strings. I'll look at it...

Possible fix:
if(trimmedLine.length() > 0) {
    allLines.append(trimmedLine).append(LINEBREAK);
}

It should be possible to have a better fix. I'll revise the logic.

I've committed a fix which also solves other edge cases.

I've committed a fix which also solves other edge cases.

Thanks, all unit tests are passing. 👍
Heads up - I need to do some rebasing/cherry picking/merging on this branch now. Some of this stuff needs to go in master and the text-render branch needs tidying up, deleted from GitHub and then re-pushed.

Heads up - I need to do some rebasing/cherry picking/merging on this branch now. Some of this stuff needs to go in master and the text-render branch needs tidying up, deleted from GitHub and then re-pushed.

 ;-)

Done

However, to support user setting the text alignment on a connection label IDiagramConnectionFigure would have to implement ITextAlignment which means changing the eCore model, regenerating and then changing the ecore model version to warn about non-backward compatibility.

Just one question: shouldn't such things be done through Features instead of changing eCore model now ?

Just one question: shouldn't such things be done through Features instead of changing eCore model now ?

Yes, but a lot of code depends on querying whether a class implements ITextAlignment so there would have to be two checks - if class implements ITextAlignment or hasFeature.

After have worked with Archi 4.7 EA/Beta with some colleagues and clients, one good remark has been formulated...
The main goal of using labels in the model tree (defined on a per folder basis) is to focus on data (ie concepts' properties) instead of naming convention. For example, I have often seen people (including me) using this kind of conventions for naming views: [STATUS] View_name (Viewpoint), while now it becomes possible to use a label such as [${property:Status}] ${name} (${viewpoint}), which makes it possible to use things like "Status" through script (to filter an export or to provide some KPI).
The issue is that when doing so, Properties' Analysis tab only reflect the name, not the model tree label.
So this led my colleagues (and me) to formulate this wish: have the Analysis tab use the model tree label instead of the real name (for concepts and views). Is it manageable ?

So this led my colleagues (and me) to formulate this wish: have the Analysis tab use the model tree label instead of the real name (for concepts and views). Is it manageable ?

It's possible if the code to render the tree label is refactored out into a common location (TextRender class). However, should this be an option? It might take someone by surprise, or simply that a user prefers not to see label rendering in that place.

However, should this be an option?

I had the same question. This could be an option (enabled by default).

So this led my colleagues (and me) to formulate this wish: have the Analysis tab use the model tree label instead of the real name (for concepts and views). Is it manageable ?

Done.

Added a new parent prefix.
Example - $parent{name}
"parent" = diagram object's parent object or View, or parent folder if an object in the tree.

All now merged to master branch. The text-render branch is deleted. Any changes/fixes will go in master branch.

Archi 4.7 is released. Let's close this monster issue. New issues can be opened if required.


docs: clarify function of DLL mode

#992

ref https://github.com/vuejs/vue-cli/issues/992#issuecomment-373958362

Closing as we are planning to upgrade to webpack 4, and due to the perf improvements in webpack 4, we will remove built-in DLL support as it's providing less value and no longer worth the extra complexity/maintenance cost.


sshchecker: init at 1.0

Motivation for this change
Dedicated SSH brute-forcing tool.
https://github.com/lazytools/sshchecker
Related to #81418
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 109852 run on x86_64-darwin 1

1 package built:

sshchecker

This is a semi-automatic executed nixpkgs-review which is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 109852 run on x86_64-darwin 1

1 package built:

sshchecker


Sidecar load WebVTT

It would be useful to be able to sidecar load webvtt by providing an alternative url from which to fetch them in addition to getting them from the .mpd. Thoughts?

Note: We require this behaviour to be able to late-bind Closed Captions to a stream. This is a side effect of existing content we already have ingested. We receive the MPD and WebVTT URLs separately, so we would like to be able to attach a WebVTT URL to the content.

It seems reasonable to have a separate subtitles URL. I think the cleanest implementation would be to add a subtitles URL to the StreamVideoSource constructor, which would of course require knowing the subtitles URL before playback.
@dougdoe: Although you are fetching the MPD and subtitles separately do you have both before playback? Or do you want to be able to fetch subtitles during playback?

We will have both the MPD and subtitle URL(s) prior to playback, so for our purposes your suggestion would be appropriate. In terms of priority, this is high priority for us – if needed, perhaps @sanbornhnewyyz can work with you on it.
Thanks!

I think we can do this for v1.5.0, which we have slotted for mid August.

Sounds good. We can give it a try when we see the commit go through. Thanks.

This works great but only for one track as far as I can tell. If I call it multiple times then only the last one will get used. I wouldn't be able to add say, a french and an english track. Is that correct? If so, would it be possible to make it so that we can call this multiple times or have it accept an array of vtt urls?

We have also determined that the WebVTT URLs that we are receiving are in the form of a playlist, which contains multiple .vtt files. Thus, we will have one playlist for each language (e.g. one playlist for English, one for Spanish, one for French).
Apologies for these lack of details. I can send you an example playlist via email, if it helps. Let me know.

With the current API you could parse the playlists in the application and call addExternalCaptions() with each subtitles URL. I'd like to avoid implementing anything too specific. Are the playlists in a standard file format?

Yes, I believe the playlists are standard format. An example is below. Each playlist contains many .vtt files for a single language.
 I had originally thought that each playlist pointed to a number of completely separate subtitles files. However, it looks like the playlist points to one subtitles track that is separated into multiple files. I believe the work required for supporting this scheme (correctly) overlaps with the work required to support TTML (#111); so it's a little beyond what we've scoped out for v1.5.0.
I will add an issue to track support for M3U based subtitles. In the meantime I think support for this scheme can be accomplished at the application level, e.g., by stitching each vtt file together as a string and passing a data URI to addExternalCaptions().

Thanks, this seems like a reasonable short-term strategy. I think the only outstanding question for this issue now is from @sanbornhnewyyz above. He indicates:

If I call addExternalCaptions() multiple times then only the last one will get used.
I wouldn't be able to add say, a french and an english track. Is that correct?"

Is this right, or is he not using the API correctly?

@sanbornhnewyyz was right. However, we've updated the API in 950b17354c11225768b6d8a714a295a2e843bcf3.


Revert "Roll Engine from 965fbbed1776 to 801559ac4ed3 (50 revisions)"

Reverts flutter/flutter#59384
This made the devicelab very red.
TBR @iskakaushik

/cc @gw280 who I think is sherrif right now.

Reference: https://flutter-dashboard.appspot.com/#/build
Not looking good. Investigating

Most likely culprit seems to be Dart roll, since it seems like all mac specific VM Service connection failures.

LGTM


Missing Header in the Umbrella Header for macOS

New Issue Checklist

[X] I have read and understood the CONTRIBUTING guide
[X] I have read the Documentation or CocoaDocs
[X] I have searched for a similar issue in the project and found none

Issue Info

Info
Value

Platform Name
macos

Platform Version
10.11

CocoaLumberjack Version
3.0

Integration Method
manually

Xcode Version
Xcode 8

Repro rate
all the time (100%)

Demo project link
NA

Issue Description and Steps
Please fill in the detailed description of the issue (full output of any stack trace, compiler error, ...) and the steps to reproduce the issue.
Getting this while on macOS. Seems like you forgot to add this header on macOS to the umbrella header.
:1:1: Umbrella header for module 'CocoaLumberjack' does not include header 'CLIColor.h'
When can this be fixed?

PR #781

Any ETA on when a fix for this will be released?

Fixed by #781. Will release 3.1.0 soon


extract amcl to separate repo

Motivation
The amcl package has many linting issues and doesn't belong in our repository.
Changes
Extract amcl to separate repository.

bors merge


Add Signup Form Frontend Feature.

Summary:

Adds the required form fields to SignUpPage.jsx, according to the User model.
Create a <FormInput /> component, that makes it easy to add more fields or customize the input's validation.
Add <PasswordValidation /> component. This is a reusable component, handy for password reset validation too.
Update the user authentication flow.

Create a "services" folder to organize modules making requests from the Frontend to the Backend, or from the Frontend to any API.
services > sendRequest.js: this module hosts the sendRequest function, designed to make HTTP requests using the Fetch API.
services > authService.js: this module is responsible for handling user authentication tasks such as signing up, logging in, logging out and managing user tokens.

User Authentication:

client/src/services/authService.js: Implemented new functions for user signup and created a skeleton for logIn, logOut, getUser, and getToken.
client/src/services/sendRequest.js: Created a sendRequest function to handle API requests with payload and token management.

Backend Updates:

server/controllers/api/userCtrl.js: Modified the create function to handle duplicate email errors and console.log when a user is succesfully created.
server/routes/api/userRouter.js: Added a new route for user signup.

Minor Changes:

client/src/App.jsx: Imported useState and passed setUser to SignupPage component. [1] [2]

Enhancements to Signup Page:

client/src/LandingPages/SignupPage/SignupPage.jsx: Added new form fields for first name, last name, username and confirm password, along with corresponding validation patterns and error messages. Updated the form submission to include these new fields. [1] [2]

New Components:
Form Input Field Component:

client/src/LandingPages/components/Forms/FormInput.jsx: Created a new FormInput component to handle individual form inputs, including focus handling, error message display, and icon integration.

Password Validation Component:

Added PasswordValidation component to provide real-time feedback on password strength and requirements. (client/src/LandingPages/components/Forms/PasswordValidation.jsx, client/src/LandingPages/components/Forms/PasswordValidation.jsxR1-R40)

Huly®: GITHB-146

Thanks for the feedback and all the detail @psantos2107 !
I've taken care of all those observations and made 5 new commits:

Add console.log in the catch block for create function
Delete duplicate paths for usersCtrl.create.
Clear password fields and display errorMsg on failed signup
User is redirected to /demo after succesful signup

Keep those comments coming.

Thanks JP! Looks great! It's all merged now. Now it's my turn to do some login code haha


Dumper silently fails for ambiguous kinds with multiple groups

Steps to Reproduce the Problem

If there are kinds available on the cluster with ambiguous names, then the dumper will just dump the objects for only one of them.
Example:
k get crd | grep data
databases.mssql.sql.crossplane.io                          2024-10-04T11:15:04Z
databases.mysql.sql.crossplane.io                          2024-10-04T11:15:04Z
databases.postgresql.sql.crossplane.io                     2024-10-04T11:15:04Z

k get crd | grep user
users.minio.crossplane.io                                  2024-10-04T11:15:00Z
users.mssql.sql.crossplane.io                              2024-10-04T11:15:04Z
users.mysql.sql.crossplane.io                              2024-10-04T11:15:04Z

All these CRDs have the database or user kind, but different groups.
k get databases.mssql.sql.crossplane.io
No resources found

k get databases.postgresql.sql.crossplane.io
NAME                          READY   SYNCED   AGE
beck-pg-tbc5b-test-database   True    True     25m

k get users.minio.crossplane.io
NAME                 READY   SYNCED   EXTERNAL NAME        POLICIES             AGE
appcat-sla-reports   True    True     appcat-sla-reports   appcat-sla-reports   46m
beck-pg-tbc5b        True    True     beck-pg-tbc5b        beck-pg-tbc5b        40m

k get users.mssql.sql.crossplane.io
No resources found

Actual Behavior
It dumps all the objects for of each kind only for the first group it finds. The ordering is alphabetical.
objects-database.json
{
    "apiVersion": "v1",
    "items": [],
    "kind": "List",
    "metadata": {
        "resourceVersion": ""
    }
}

objects-users.json
{
    "apiVersion": "v1",
    "items": [
        {
            "apiVersion": "minio.crossplane.io/v1", # only minio users
            "kind": "User",
            "metadata": {
                "annotations": {
*SNIP*

Expected Behavior
It should create lists for all the groups, not just one.

Created a PoC to replace this insane bash script. Also does away with known-to-fail because we have all information we need from the discovery API.
My vote would be to productise the PoC.