Storing section- and pages-data in XML-files

I believe this was already done by the Symphony 3 project, but I was wondering the other day that it would be better to store sections and pages in configuration-XML files, rather than directly in the database.
The reason for this is interchangability. When creating websites as a full-time job, it's rather a waste of time to have to recreate the same logic for news-sections, faq, images gallery, etc. over and over again. It would be great to:

store the structure of sessions in seperate XML-files (just like the scaffold-extension sort of does with JSON-files, but then only for importing and exporting)
store pages configuration in an XML file (type, datasources, events, etc.)

This change could provide for interchangable pieces of code. Think about a couple of files which you could throw in your Symphony root, and they would provide the sections, (demo?)pages, utililities, (demo?)css and javascript for fully functional widgets for your site. This could be a great addition next to extensions and XSLT-utilities.
Now, before we start with any further details on why this (or why not) this could work, please share your thoughts on something like this. I'm sure I'm not the only one creating news sections, faqs and image slideshows over-and-over again.

The answer I think is; Yes, it would be better.
Whether that is possible with the current layout of the core is another question. That is what the Symphony 3 project was all about. The changes require a vast amount of development work, and would be completely 'feature breaking' for all extensions.
The idea was dropped for development and release due to this breaking, as a large majority of the Working Group felt that more support for the current Symphony was a priority. Also there was a cost consideration, as to get the Symphony 3 project into a release state would take so much time that hiring a developer in to do it would be a requirement, and as Symphony is now fully open source, there is no financial aspect to do this with.
I would say that it is definitely on the cards for the future, but when is the question.
Do any devs out there think it's viable to try and resurrect the ideas of Sym 3?

It's where the 2.x branch is headed. The main goal I have to get our API's working so that to an extension developer it doesn't really matter where or how this information is stored.
This was one of the reasons behind the creation of a PageManager in 2.3, so that there is a defined way to access information about pages without extensions having to query the tables directly. It also a reason why I grimace when extensions alter system tables (tbl_sections, tbl_pages etc.) because in the future, those tables may not even exist at all ;)
So to answer your question. Yes it's definitely what we want to do and while it won't be in for 2.3, it may be in for 2.4 :)

That's good to hear, perhaps a good stepping stone for this feature to 2.4 would be to change the core functionality of storing sections out of  the database and into the XML (workspace/sections for example). It would mean altering classes like the SectionManager and such, so they would still provide the exact same functionality as they do now, except with the difference that instead of storing section information in the database they store it in XML.
Another step that could be taken is to make events and datasource id-independent. They are partly now, but there are still some hard references to ID's, like the getSource()-function and the $dsParamFILTERS-convention. I think if we take it by steps to get something like this working for 2.4 we could (in order):

Make datasource ID-independent
Make events ID-independent
Make sections database-independent (store in XML)
Make pages database-independent (store configuration in XML)

When these 4 steps are taken, I see great possibilities in how one could create interchangable packages (for example a news-item, containing the sections, datasources, utilities, (example-)pages, (example)CSS and Javascript, or even more complex features like blogs, forums or shopping carts). This could mean great potential for community-driven content, even for those not trained in stuff like PHP and programming.

That's good to hear, perhaps a good stepping stone for this feature to 2.4 would be to change the core functionality of storing sections out of the database and into the XML (workspace/sections for example). It would mean altering classes like the SectionManager and such, so they would still provide the exact same functionality as they do now, except with the difference that instead of storing section information in the database they store it in XML.

Yep, that's the idea. We don't have time to implement such a change for 2.3 however.

When these 4 steps are taken, I see great possibilities in how one could create interchangable packages (for example a news-item, containing the sections, datasources, utilities, (example-)pages, (example)CSS and Javascript, or even more complex features like blogs, forums or shopping carts). This could mean great potential for community-driven content, even for those not trained in stuff like PHP and programming.

Definitely, we are actually using a bastard build of Symphony 3 at work for this very reason.

I could see if I could fit in some time somewhere if I could look into the ID-independency. But I'm rather busy at work now so I can't promise anything...

Removing IDs from the generated files will be a massive step forward, from the event/DS getSource() methods and data source filters. This could potentially be a non-breaking change (for 2.3 or 2.4) since the code could detect whether the string is numeric (an ID) or a handle which needs resolving. This alone would make data sources and events shareable between builds so long as field/section handles remain consistent, regardless of whether sections/pages themselves are XML.
We have tools for migrating sections between builds (Scaffolds?) which make the storage of sections/pages in XML a lower priority I think.

2.4??
Wow. I thought it'd be at least 2.6 or thereabouts.
I will be working with Symphony again from a development point of view form March onwards, so I can, and would like, to help out where I can here.

As you said, there is probably a number of extensions that expect the function to return an ID as well and because of this the other consideration is updating getSource when a Section is renamed and then how extensions might also be notified of such a change.

I've put on my developers hat and I made a brance to make ID-independent datasources:
https://github.com/kanduvisla/symphony-2/tree/ID-Independent-Datasources
I've made two commits which store datasource ID-less:

getSource() returns ds:[handle] instead of the ID of the datasource.
$dsParamFILTERS uses field:[element_name] instead of the ID of the field for sorting.

Now, it works when creating and editing datasources, but I haven't done any logic yet on where to go from here. So how the correct data goes in the XML from here. I also might have missed a spot here and there.
Are there any participants who want to join me in the quest for ID-independent datasources? (and perhaps more in the future?)

That's a decent start, from there the datasource.section.php file (which will be obsolete by the time this is implemented) used the ID's to pull the field instances from the FieldManager. That should be a relatively simple change to FieldManager::fetchFieldIDFromElementName which will return the 'id' that can then be used with the FieldManager::fetch() function.
Keep in mind there'll be a little bit changing in terms of datasources and their setup (see #723 and the wiki)

If this is all being done with handles... Doesn't this throw up problems for multi-language handles? Aren't there possibly more than one handle with multi-language extensions??

I Don't think so, since we're talking about section names, datasource names, event names and field names, and not their values.

Fortunately I think @kanduvisla is right, multilingual doesn't apply to the internal structure of the site, it's content that gets translated.

Still it was rather quick to accomplish since most of the functionality relies on abstract functions like SectionManager::fetch()

That's the idea, once things are abstracted it's trivial to where the data is actually stored :)

Good. Just checking...
Looking forward to all this!

Me too! ;-)

Is this going to hold up 2.3?

I don't think so, at least... I consider it to be a playground to see how we could apply some 'loose entanglement' here and there... Just messing around with the code could provide some insight on how certain things could be looked at in other ways to provide an opener system for the future.

Hmm... just thinking, I'm playing with the code now, and although ID-less datasources and events seem possible, it still wouldn't mean the interchangeability of functionality where we all wish for. Not as long as sections and pages' configuration are stored in the database. And when we change those two (especially sections), the code I have been working on might have to be rewritten again...
I've got some ideas on how to store sections seperate from the database, but I might have to write these down more thouroughly and get some community response...

@kanduvisla great stuff — I am keen as mustard to see this feature brought back from Symphony 3 :)
I wonder, though — won't this land us right back where we were with the Symphony 3 developer preview? Extension incompatibilities were one of the reasons used by some of the working group members to scuttle development of Symphony 3.

That's a pretty good summary @kanduvisla. One thing I would pause on is the concept of modules as well as extensions (unless you were using these terms interchangeably). I think it would make more sense that an extension could provide Sections and Pages to extend on what they currently provide rather than inventing a new (additional) concept.
As you mentioned, the key is getting the system API correct so that extensions can do what they need without directly querying the database. That will help ease initial pains. Of course I don't mean this to be  seen as a 'we need 100% API coverage' thing. 80/20 will work, extensions that are doing something a little bit odd can always do a bit of a manual dance to get what they want :)
And just to be clear, this won't be a 2.3 feature. 2.4 is a nice goal (however ambitious it might be!)

@tonyarnold Although I understand your concern, I think extension compatibility could be maintained to a certain level. Nevertheless I think some adjustments need to be done by the developers, but this has been the case for each major symphony update (think about the changes like _Parent and the static Manager::fetch()-functions for example).
In my opinion you can't make groundshaking changes without some changes to the flow here and there.

@brendo I introduced modules as a new concept. I don't think the features I've described as 'modules' should be implemented as extensions, since when you installed a news module for example, you might want to make some adjustments to it to make it suit your needs. And they would be stored under he same path as the rest of your sections and datasources. I might explain this next week, but I'm currently typing this from my IPhone and autocorrect is killing me! :-P

I've had sleepless nights with the problem of loose coupling, and I came up with a possible solution. Could you guys please provide some feedback/insight on your thoughts on this?
I've edited my wiki page, but here's the main part that's added:
The problem of loose coupling
The main problem with loose coupling is that there are no ID's to make matches with. A practical example: Entries are stored in the database. These entries are stored in a section. Currently, this section is linked to the section by it's ID. But if you store sections in XML-files for interchangeability, ID's make no sense, since they could conflict with the ID's of other sections.
You could use the handle of a section to tie it all together, but this would raise problems as soon as the name of the section changes. The same problem rises for fields: they are also matched in the database by ID's.
A possible solution for loose coupling
A possible solution for this problem is to provide each section and field with a unique hash. This could for example be a hash of a timestamp + name when the section or field was created. This unique hash could be stored in the XML file.
In the database you would then create a lookup table on how the sections and fields are matched with ID's. In this case, the ID's are thus used *purely internally by Symphony for making the correct relations.
A lookup table could look like:
| ID_SECTION | HASH                             |
|------------+----------------------------------|
| 1          | 97dcae5e3bd3c9fb78fa302c0a083947 |
| 2          | bb2c28e57b6f2cd75723f67948e3c73c |

In this scenario, the hash is used to couple the section stored in the XML file with an ID used internally.

I think this is what Symphony 3 did, no? I recall seeing UUIDs (or similar) in the XML.

Symphony 3 stored hashes (I think obtained using md5_file) of the Section definition XML and then stored it as an attribute. This hash was then saved in a tbl_sections_sync table. If the two values differed, something had changed and the user was prompted to 'sync' their section.
The S3 code that went out never really got this sync logic working 100% from memory and for our internal S3 use it's always felt a little off.

I'd make two suggestions to the structural changes you've suggested:

Name the "cms" folder something more descriptive like "components", "modules", or keeping in line with their similarity to a small subset of an ensemble, maybe something symphony-related like "composition"
I don't think there's a need for namespaced packages within the "cms" folder — it's an unnecessary level of organisation. It's up the developer of the site to name the folders within uniquely — hopefully this could be designed that if the developer wanted to name a folder using a UUID, or using a namespace (ie. com.thecocoabots.my-great-component) they could do so. I'd imagine that we'd want to provide something similar to the extension.meta.xml file for each of the items within this directory, so the names can be unique and independent of what's displayed to the authors.

Have you had a good look through Symphony 3? Might be worth doing so — some of the questions you will need answered could possibly be answered by the S3 developer preview, even though it was never finished.
Otherwise, it's looking good — don't stop!

I don't quite see how there is similarity between this idea and relationships to be honest. What we're talking about here is pretty much simply removing persistence of Symphony's structural elements (pages, fields, sections) from MySQL and storing them as flat XML files instead (over-simplified, I know). Relationships need to be re-thought at a much higher level regardless of where they are stored/configured.
Looking forward to seeing this XML idea progressing. I know there are some extensions which modify the Pages and Sections tables in the database, so perhaps there should be getters/setters on these objects for adding arbitrary columns (or elements) to the table (or XML file).

What we're talking about here is pretty much simply removing persistence of Symphony's structural elements (pages, fields, sections) from MySQL and storing them as flat XML files instead

True, let's focus this discussion on doing just that and save relations for later. But the relations I'm concerned about at the moment are fields like Select Box Link for example which store a related_field_id in the database. But instead of related_field_id something else should be covered here since this would be data that would be stored in the configuration-node of a fields' (read: sections') XML. Perhaps this should become related_hash which refers to another field in the module?

I also think that a lot of this can be achieved by refactoring the manager classes without breaking too much elsewhere.

I think this too. If the manager classes are far enough abstracted, this could turn out to be fairly easy to implement. Let's hope so!

I described relations in the wiki page. This is what's added:
Relations in fields
Some fields store relations to other fields, like Select Box Link for example. When you look at tbl_fields_selectbox_link for example, you see a field called related_field_id. However, in this setup a field would be stored inside the <fields>-node of the section XML. Therefore an ID cannot be stored.
A logical solution for this would be to store the unique hash of the related field in the XML. In this concept, a Selectbox Link Field could look like this:
<field>
    <label element_name="category">Category</label>
    <unique_hash>b747b6e5ffa3db6bff396b6759fddcd7</unique_hash>
    <type>selectbox_link</type>
    <required>true</required>
    <sortorder>4</sortorder>
    <location>sidebar</location>
    <show_column>false</show_column>
    <configuration>
        <allow_multiple_selection>false</allow_multiple_selection>
        <show_association>true</show_association>
        <related_field_hash>61c00b1e696c954ff681558938a44d1d</related_field_hash>
        <limit>20</limit>
    </configuration>
</field>

That, or a field ID is an compound of the section handle and its own handle. In your example if the Category field was in a section Articles its internal ID (unique_hash) might be articles_category. And if the related_field was in a section My Stuff pointing to a field TItle then the value would be my-stuff_title.
Either way, what you're describing is a move from numeric IDs to unique strings (whether they be hashes, UUIDs/GUIDs, compound keys or other), not just for fields and their relationships, but also sections and pages, and anything else structural within Symphony that might be modularised?

Either way, what you're describing is a move from numeric IDs to unique strings

Yes, that's right. I want to separate the section/field/page logic from the database so they can be interchanged with other Symphony installations. I thought about using the handle (or a combination of handles) as unique string, but this would pose a problem when you would change the name of a section or a field. Now you could do a find-and-replace when changing sections and stuff in the CMS, but that would only provide for new possible problems in the system. Also, then it also wouldn't be possible to manually edit the XML-files, since their reference with the database (lookup table) would be lost. So that's why I thought this should be handled with unique strings (hashes in my example), which would only be created once, and never change afterwards. Also with the idea that modules could be exchanged on the community (think of forum modules, image galleries, blogs, news, etc...), unique strings/hashes are a must.
In my wiki pages, pages don't seem to have the unique_hash-node, but it seems that this might also be necessary there.

I don't see 'my' approach as a 'sync' of sections. The sections would be completely removed from the database and only stored in XML. Therefore I only see the reason to create a unique hash one time only, and that's when creating a section (and/or field). That's also the reason why I used md5(timestamp) as hash.

Yes on first glance, but it's not this simple. Consider if you had a news section defined in your XML and a couple of entries. The requirements change and you no longer need a teaser field, so you remove the XML from the section. When you view News entries in the backend, the definition for the teaser field wouldn't exist anymore, so that data would be dropped. Just like that.
The concept of sync was to protect for accidental data loss. You were free to change the file as you would like, but destructive operations would only run after you had 'Synced' the section.
The concept of GUID's is not new, S3 had @guid attributes for sections and fields. GUID's do not have to be created by the user, they will be generated by Symphony when the defintion is parsed.
Your concerns about relationships are valid, the Link field in S3 would has that bug as it only stores the reference to a field using section::field. So if the section or field were renamed, this link would become invalid.
Thanks for your continued efforts in this, I'm sure you can now appreciate the effort and thought that goes into this sort of change :D

@brendo I sure can, but where S3 sort of 'stranded' (sorry I have to say it), I hope this could get picked up. As soon as the concept is 99% well thought through, I'm going to start with the actual development of it.
As for your sync-option: I thought of this too, but a more user-friendly approach (in my opinion) would be a simple notifaction in the top bar, saying something like 'There is field data stored in the database which no longer refers to a field (hash: [...]) click here to delete'.
The way I see it, the following should have unique hashes: sections, fields, pages. Am I missing something here?

All file based definitions will need unique hashes and some record in Symphony (whether it's the database or config file) to keep track of what's up to date and what isn't.
The simple notification might work actually, would have to see how it scales if there a number of changes. There is also other edge cases we'll need to consider, such as changing a field type but maintaining the same name (you would need to confirm this change as well so the uninstall and install logic can run).

to keep track of what's up to date and what isn't

What's your definition of 'up to date' here? Did you mean that:
a. There would still be sections/fields/pages-tables in the database which merely are 'shadows' of the XML-files and need to be updated whenever the XML-files are?
b. Or are you merely talking about checking if a field or a section has already been initiated or not.
Because approach b is where I would be going. Just to make sure we're all on the same page here.

changing a field type

This might even be a nice feature! It wouldn't be the first time I have to manually change the database because I'm changing my input elements to multilingual input fields, or file uploads to unique file uploads...
But perhaps that's something to consider for the future. As for now, let's keep our main focus on the core idea, which is seperating sections/fields/pages-logic from the database and making them interchangeable.
I'll update the wiki page to describe what would happen when a field gets added or deleted.

wiki updated:
Adding and deleting fields
Data would still be stored in the database. Therefore, a field should still be initiated when it's added for the first time. This is easily done by checking if the hash already exists in the database. If not, then the initialization/installation-method of the specific field should be executed, followed by inserting its unique hash into the database.
Deleting fields could be done the same way. When a unique hash is detected in the database, and no corresponding field is found, it would mean that the field is removed from the XML-file. The data-table of the field could then be dropped. Now, to be sure that we're not accidentally dropping data from the database, an extra security measure should be made. This could be done in the form of a notification message on the top off the page, showing a message similar to: 'There is field data stored in the database which no longer refers to a field (hash: [...]) click here to delete'
Mutating field types
The XML-approach makes it theoretically possible to change the type of a field, whilst retaining its data. You could for example change a field from type upload to unique_upload. In this concept we don't deal with these situations, since Symphony has no solution for mutation of field types. This should be considered a whole new concept and should not be discussed in this concept.

Because approach b is where I would be going. Just to make sure we're all on the same page here.

A). You need to store the GUID, the MD5 of the XML, and the XML.
The GUID is consistent between the definition on disk and in the database. The MD5 of the XML is used to quickly compare whether the definition on disk (ie. /workspace/sections) is the same as the one in the database. The XML is stored so that if the two MD5's differ, you can tell the user why they are different.
Your idea of storing not only the GUID for sections, but also for fields could be a way around not storing XML in the database.

This might even be a nice feature! It wouldn't be the first time I have to manually change the database because I'm changing my input elements to multilingual input fields, or file uploads to unique file uploads...

Yes, in those situations it's a pain, but if you change a Text Input field to an Upload field then all hell will break loose as the schema for these fields is different. As you said, it is another kettle of fish of how to handle that elegantly.

@brendo: ok, so if I understand correctly, you do support the idea of no longer storing section and field structure in the database, but you still would store the entire XML and make comparisons, etc.
I'm interested in why you would approach it like this. Does it still have to 'sync' the changes before they are effective? The way I see it, the XML would not have to be stored in the database, since it's roughly speaking the replacement of how things are handled now. Syncing it would add an extra feature. It would be the same as if in the current Symphony installation (S2.2.5) you would add a 'feature' that would show you a diff report after you save a section, and then would require you to 'sync' it (actually clicking 'save' two times). I don't know if I'm making a lot of sense, but that the way I would see this approach.
The approach I see before me is that the SectionManager on initialization would roughly be (in order):

Build a list of sections according to the XML-files found.
Iterate through the fields of each section.
Determine for each field if it should be installed, uninstalled or do nothing.

Now, any changes in the XML-files would automaticly be parsed on each load, since the XML-files get re-evaluated on each page load. Now some of you might wonder if this doesn't affect performance, but I don't think so. The XML-files are read only once on initialization and the configuration could then be stored in PHP variables. I even think this could even be a lot faster than have to make database-queries all the time when some information is needed about a section or field, since they would be stored as variables in a PHP object.

I see is treating the XML file as the one true source as risky and may result in users losing their data if fields are automatically renamed, changed, added or deleted one every page load in the backend (consider copy + paste mistakes, corrupt FTP uploads etc)
I see the main reason behind XML definitions is that they can be kept in source control, not as a primary method for maintaining your site's structure. Maybe sync is the wrong word, but the idea is that yes, you can change your definition in your IDE, but Symphony won't do anything automatically, you have to tell it to 'install' this definition.
It's much like how you need to enable extensions to update them. By making the process controlled by Symphony, it then allows you to store history in the database so that you can rollback and forth to different versions of your site. Sure, this would be possible by checking out the file from a previous commit and then syncing the changes back, but the benefit is there is a UI.
Why theoretically you could implement your approach, I think don't think it'd be practical in use.
Does that make more sense?

@brendo: Al your arguments make perfect sense. I might have to think this one through. Ofcourse, storing a copy of the XML in the database (or at least a MD5 of the files' content) is more rock-solid, but it also forces the developer to do an extra action when changes are found. On second thought, I assume this would only occur when editing the XML-files, not when using the GUI/CMS to change sections, so it might be feasible to have this 'sync'-option...
@tonyarnold @nickdunn @eKoeS @designermonkey  I'm curious, what are the other Symphony developers thoughts on this subject?

On second thought, I assume this would only occur when editing the XML-files, not when using the GUI/CMS to change sections, so it might be feasible to have this 'sync'-option...

Correct. Using the UI to alter sections functions as it does now, it's only changes that happen outside of this that will require a manual 'sync' or 'migrate' option.

I'm curious, what are the other Symphony developers thoughts on this subject?

Yeah please! I'm mainly relaying the discoveries that myself, @rowan-lewis, @allen and @pointybeard discovered while building the S3 prototype. Fresh eyes may lead to a more elegant solution.

I've been discussing this with @brendo tonight, so similar to what he's already said. I don't think @brendo is proposing storing the XML directly in the database, just continuing to use similar structures to what we already store. Whenever the XML file is modified on disk, sync options are presented as you've already proposed. Whenever the developer changes the section via Symphony's section editor, those changes are persisted to the database, and also to the XML file in workspace/sections.
I think the main problem we both saw with always reading and writing against the XML file was performance — disk I/O is expensive and can't be scaled as easily as a database can. This would seem to be the best of both worlds to my thinking.

I've actually tried to keep out of this discussion so far, as it seems to be rehashing things that were already achieved for Symphony 3.
As I haven't actually looked at the codebase though, I can't really say what should/shouldn't be done. I will ask this though:
Why can't we just take the code that does this in Symphony 3, fix it's issues and make it 'Symphony 2' compatible, and add it to the codebase we have? It is probably a very naive question, so forgive me.

I have no further opinion I'm afraid.

I've actually tried to keep out of this discussion so far, as it seems to be rehashing things that were already achieved for Symphony 3.

That's true, but a lot of this discovery wasn't shared with the wider community, so rehashing it now is a bit of a crash course for everyone.

Why can't we just take the code that does this in Symphony 3, fix it's issues and make it 'Symphony 2' compatible, and add it to the codebase we have? It is probably a very naive question, so forgive me.

The codebases differ wildly in a couple of respects. S3 is written with no regards for backwards compatibility or PHP 5.2 support. The concepts of managers is also dead in S3, with objects being responsible for themselves. The underlying concepts can be shared, but a copy + paste is probably a little way off ;)

I guess my one opinion would be... could this discussion please be deferred until 2.3 is released :-P

could this discussion please be deferred until 2.3 is released

Wholeheartedly agree with this.

Was anyone here aside from @brendo actually going to be working on Symphony 2.3 tonight?</snap>
I say carry on — it's not doing any harm at the moment.

Why can't we just take the code that does this in Symphony 3, fix it's issues and make it 'Symphony 2' compatible

I also don't think it's this simple. At the time when the S3 prototype was released, Symphony itself was still at something like 2.0.6 or something. A lot has changed since then, and I believe now the time is right to implement these features as experimented with in S3 in the current codebase (perhaps something for S2.4? Or... perhaps S2.4 could be the new S3? Just a radical thought!)

could this discussion please be deferred until 2.3 is released

I don't know, although the release of S2.3 is also high on my wish list I think it's very good to keep on brainstorming about ideas on how to improve Symphony. Deferring a topic that is currently getting a lot of attention could result into 'never discussed about again'. I don't want to be pessimistic, but we know there are also other ideas that showed great potential, got great interest from the community, and subsequently died a silent death :-( (I'm looking at you, relations-concept!)

Using the UI to alter sections functions as it does now, it's only changes that happen outside of this that will require a manual 'sync' or 'migrate' option

I think this might be the best approach to go for. I've also discussed it with a colleague of mine (@Gaya), and it looks from here that also most developers consider this to be the best practice. Also since @rowan-lewis @allen and @pointybeard practiced this method in their S3 prototype.
I'm going to update the wiki page to support this new insight.

wiki updated:
A possible solution for loose coupling
A possible solution for this problem is to provide each section and field with a unique hash. This could for example be a hash of a timestamp + name when the section or field was created. This unique hash could be stored in the XML file.
In the database you would then create a lookup table on how the sections and fields are matched with ID's. In this case, the ID's are thus used purely internally by Symphony for making the correct relations.
There also should be a hash of the corresponding file stored in the database. This is to determine if the content of the XML-file has been changed outside of the UI. This could otherwise cause issues (think about typo's, duplicate unique hashes, or accidentally the removal of sections/fields).
When the XML-files and the hash in the database don't match, the developer should get a notification about this, and click on a 'sync'-button to synchronize the sections/fields. The XML content should also be stored in the database, for when a mismatch is found, the stored XML could be used instead of the external file.
The stored XML data could optionally also be used to show a diff between the external XML file and the stored XML data.
A lookup table could look like:
| ID_SECTION | HASH                             | FILE_HASH                        | XML_DATA        |
|------------+----------------------------------+----------------------------------+-----------------|
| 1          | 97dcae5e3bd3c9fb78fa302c0a083947 | b747b6e5ffa3db6bff396b6759fddcd7 | (some xml ...)  |
| 2          | bb2c28e57b6f2cd75723f67948e3c73c | 61c00b1e696c954ff681558938a44d1d | (some xml ...)  |

In this scenario, the hash is used to couple the section stored in the XML file with an ID used internally.

[kanduvisla] As for your sync-option: I thought of this too, but a more user-friendly approach (in my opinion) would be a simple notifaction in the top bar, saying something like 'There is field data stored in the database which no longer refers to a field (hash: [...]) click here to delete'.

If you revert your section definition to a previous version with, say, three less fields and one whose handle has changed, you would end up with around four notify messages at the top of the backend. I know it's just UI and it's premature to talk about it, but I think @brendo is right about scalability and a "sync" concept deserves more than just a bunch of notifications.
We can still use them, but for a broader purpose: "One or more sections require syncing. [...]"

We can still use them, but for a broader purpose: "One or more sections require syncing. [...]"

True, perhaps syncing should have it's own diff-page, instead of being a single button on the section editor screen.

developers should be free to "solve" a subset of them if they want

In the current setup this is possible. Each field will have it's unique hash, but all configuration of the field is stored in the XML, nowhere else. So if I would have a field like:
<field>
    <label element_name="name">Name</label>
    <unique_hash>97dcae5e3bd3c9fb78fa302c0a083947</unique_hash>
    <type>input</type>
    <required>true</required>
    <sortorder>1</sortorder>
    <location>main</location>
    <show_column>true</show_column>
    <configuration>
        <validator />
    </configuration>
</field>

I could change everything (except for the hash ofcourse) and it would be taken into account. So I could set required to false, change it's name, or even it's internal configuration by setting a validator or something.
Any change would cause the MD5 of the file to change, and the diff-flag would rise; again resulting in a (single ;-)) notification refering to a diff-page. Does this answer your concern or did I misunderstood your question?

Any change would cause the MD5 of the file to change, and the diff-flag would rise; again resulting in a (single ;-)) notification refering to a diff-page.

Let's suppose you are inside the diff-page. You are shown a list of differences between the database version of a section and the relative XML files. Say there are 4 changes in total. I'd like to apply only three of them, thus asking for a "partial sync" of my two versions of the same section.
As you said I think it's possible with the current setup... I was just tossing in some ideas. :)

I was just tossing in some ideas. :)

And you're doing a great job! That's what we're here for! ;-)

thus asking for a "partial sync" of my two versions of the same section.

Hmm... this could be functionality of the diff page, but should it be it too?  @rowan-lewis, @allen and @pointybeard: How did S3 handle these conflicts?

As another practical example, I can imagine downloading a News module which has more fields that I want. From the diff-page (which acts as an install page too) I can choose which fields to install.

@kanduvisla, you know @brendo built a lot of the S3 dev preview, right? You've been directing questions at the other guys (of which only Allen is still active in the community right now), but @brendo is here answering already.

S3 didn't offer a partial sync. If there were changes a notification allowed you to 'Sync' or 'View Details'. Clicking View Details pulled down a table which showed all the fields in the section and their status across three columns, new, updated, removed. This was clunky and definitely needed a bit of refinement though, so it's probably a good idea for us to think fresh on it.
The benefit of it was that a user didn't have to go to separate page or click a special tab. They were notified through a familiar mechanism in Symphony (notifications). Where the UX fell down was the sync table, it wasn't the most readable.
I'll screenshot this on Monday

I think that a partial sync is needless complexity. You're either using someone else's section definition (which you can modify), or you're creating your own. In either case, you have permissions to change the section before or after it's synced with the persisted database version.

You're either using someone else's section definition (which you can modify), or you're creating your own.

How? By manually editing the XML file? I think we can do better than this.

you have permissions to change the section before or after it's synced with the persisted database version.

That's true, but that would be more steps than is necessary.
Even if we don't implement partial sync, from a programming point of view I think we would eventually need to sync a section by iterating through its changes, syncing one field at a time. From that point of view, having an interface for partial sync wouldn't introduce any more complexity as the logic is already there.
To me a partial sync positively affects the workflow and as such should be a must.

How? By manually editing the XML file? I think we can do better than this.

Not sure I follow here? Editing a sections' definition XML manually (ie. Textmate) is the only way a user will be prompted to sync. If they edit through the Symphony UI, it's the same as what happens now, whatever you add/remove/change is saved.

To me a partial sync positively affects the workflow and as such should be a must.

Because of what I said above, I think it'd be an edge case and would be used rarely. Lets get a sync working before we deliberate over details like this because as you say, it should be trivial to implement if it's required.

Not sure I follow here? Editing a sections' definition XML manually (ie. Textmate) is the only way a user will be prompted to sync. If they edit through the Symphony UI, it's the same as what happens now, whatever you add/remove/change is saved.

I'll try to explain it better: if you downloaded a module which was already used somewhere else and you want to import it in your current setup, then a partial sync would avoid you manually editing it as long as you need a subset of the fields stored in the file.
Sure you can open the file and manually remove what you don't need, but wouldn't it be better to have a UI that takes care of that? I'm not sure it's that rare as you say, I don't think I'll always need all the fields and all the sections of a module (especially when developing complex sites).
The benefit of a partial sync can be explained as follows:

If your section has a field that is not present anymore in the XML defintion, a total sync deletes that field even if you want to retain it. On the other hand, a partial sync would allow your to "cherry-pick" the changes you want to apply so to preserve that field.

If your XML defintion has a field that is not present in your database, a total sync adds that field even if you don't need it. On the other hand, a partial sync would allow your to "cherry-pick" the changes you want to apply so to avoid that field.

If your section has a field whose metadata are changed in the XML defintion, a total sync updates that field even if you don't want to change the field. On the other hand, a partial sync would allow your to "cherry-pick" the changes you want to apply so to avoid that update.

Which are the cases where you get a XML file that doesn't match your database and, more important, your vision?

Modules shared by other members, which can be updated as time goes by.
Modules shared across setups by the same developer, which could require slight changes on a per-setup basis.

I hope it's clearer now.

We need to think about whether we want to encourage such inconsistency though.
"Oh I'm using Bob's News module, but only fields x, y and z".
What if the News module has datasources or events that require particular fields to exist?
We are making this too complex.

What if the News module has datasources or events that require particular fields to exist?

Then such datasources/events wouldn't be imported (easy way). However, I see your point (and @tonyarnold's) and the drawbacks of the partial sync approach. As I said I was just throwing in ideas. :)

The concept of partial sync deviates from what the section syncing's original purpose: to maintain parity between multiple builds. Allowing multiple versions of the same build introduces a whole new set of challenges that the original concept tries to solve.
I also think that adding UI control for partial sync is not as simple and straightforward as one might think. New edge cases begets more edge cases.

Soooo... anyone? If not, I'm going to finish the concept, summarize it and start making a simple diagram on how I think the code should flow.

Summarise and continue on, any further questions (and they will exist), we can talk about as they arise.

To be more concrete: This means that in my opinion the current concept should go with the full sync-way of working (just like S3 did). When this is working we have achieved the main goal what was uttered in this concept. A partial sync could be something that could be discussed later on from there (perhaps something for S2.4.x?).

I think that's a good plan, keep up with the good work! :)

I summarized it and proposed some of the code flow: Modules
I might alter the content here and there, but I think the bigger picture is clear now. Also, I've written it in 'the future', as in how Symphony 'should' work. Like some sort of future documentation for future developers who want to understand what's going on under the hood. ;-) A colleague of mine once mentioned that it's very good practice to document something thoroughly before even writing a single line of code. Especially when it comes to open source!

That's really good, and the flowchart will be helpful when testing and implementing!
The implementation should first work on the saving of the files into /workspace/sections and workspaces/pages.
One thing I want to stress is that modules are not required for this flat file saving. It's a concept of how definitions might be shared between installs. Modules need to be discussed, later, when we have definitions saving as flat files.

So, what you are proposing is not to store the information like:
workspace
    modules
        [modulename]
            sections
            datasources
            events

but rather like:
workspace
    sections
    datasource
    events

Is this correct?

The implementation should first work on the saving of the files into /workspace/sections and workspaces/pages.

I'm inclined to agree. I think the concept of "modules" needs a little more thought to see how it ties in with ensembles (which it could potentially replace) and extensions (which might provide modules). It's another term to add to Symphony knowledge, whose meaning isn't immediately obvious (like an ensemble) so it needs to be very clear to users and developers.
Can't wait to see the page/section XML working into workspace. Storing these in source control is going to be a big win!

Is this correct?

Correct.
@nickdunn summarizes the concerns quite well. We'll have three similar (but slightly different) ways to share 'parts' of Symphony. Lets not get stuck on it here, it's a discussion to be had another day as it will potentially have far reaching effects.

Allright, then I'm going to drop the workspace/modules-concept in favor of workspace/sections. However, for now-sake, I'm still going to refer to a collection of datasources, events and sections as a 'module'. If that's okay with you guys ;-)
Or is there perhaps another term which summarizes it better than 'modules'?


implement e2e cleaner

implement e2e cleaner for MSFT subscription

subscription keeps filling up, I think this should help us - wdyt?

@julienstroheker - thanks - all good, I've done the pipeline config too.


Update to v0.12.dev0

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.39%. Comparing base (c4e9913) to head (37c0021).
Report is 15 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #390   +/-   ##
=======================================
  Coverage   98.39%   98.39%
=======================================
  Files          37       37
  Lines        4293     4293
=======================================
  Hits         4224     4224
  Misses         69       69

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


addImports ignores side effect-only imports

Hey, thanks for the very quick response for the other issue and for the project in general, it's incredibly helpful!
That being said, I stumbled across another issue when importing side effect-only modules.
The following code produces an empty string:
const root = j("");
addImports(root, [ statement`import "my-module"`]);
console.log(root.toSource());

while the expected behaviour should be that it adds the import statement:
import "my-module"

Okay, no worries, this is another valid issue 👍  I'll figure it out

Okay, no worries, this is another valid issue 👍  I'll figure it out

:tada: This issue has been resolved in version 1.0.10 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 1.0.10 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


discord-gamesdk: init at 3.2.1

Description of changes
A library that allows programs to interface with Discord.
Note: Some games or applications might have this library vendored; I'm not sure how to discover those, just something to be aware of.
(I'm not aware of a better homepage URL; if there's a better one, please link me to it!)
Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux: Not applicable
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Whoops, I made a really big screwup. Sorry everyone, removing the review requests.


Code splitting: Output list of imports for each entry point

When entry points contain import statements for shared/split chunks, it'd be useful to output details of this so that user can preload all chunks upfront to avoid multiple server round trips.
e.g. with this output:
// entry1.js
import fn from './chunk.XXXXXXXX.js';
fn(1);

// entry2.js
import fn from './chunk.XXXXXXXX.js';
fn(2);

// chunk.XXXXXXXX.js
export default function(n) { console.log(n); }

Livepack should provide info to the caller that entry1 and entry2 both import chunk.XXXXXXXX.js.
How to provide this info
serialize() / serializeEntries() could include this info as a property of the file objects returned:
[
  { filename: 'entry1.js', content: '...', chunks: [ 'chunk.XXXXXXXX.js' ] },
  { filename: 'entry2.js', content: '...', chunks: [ 'chunk.XXXXXXXX.js' ] },
  { filename: 'chunk.XXXXXXXX.js', content: '...', chunks: [] }
]

CLI could have an option --stats to output a stats file which includes this information.
Related to #144.

Stats file output implemented in v0.3.2. It does not have chunk names included yet.


OpenJS Foundation Security Collab Space Meeting 2024-08-19

Time
US PDT Mon 19-August-2024 08:30 AM
Or in your local time:

https://www.timeanddate.com/worldclock/fixedtime.html?msg=Node.js+Foundation+Security+Collab+Space+Meeting&iso=20240819T0830&p1=224
or https://www.wolframalpha.com/input?i=8:30am+PDT+August+19+2024+in+local+time

Links

Minutes Google Doc: https://docs.google.com/document/d/15Am1QD6LQpaajwHZcUpsJ8u1uXAfjR9KZ1sPfWI1kXk/edit

Agenda

https://github.com/openjs-foundation/security-collab-space/issues/177

Invited

OpenJS Foundation Security Collab Space Members
OpenJS Foundation Cross Project Council
OpenJS Foundation Project Maintainers
OpenJS Foundation Board of Directors
All are welcome

Observers/Guests
Notes
Joining the meeting
Meeting link for participants:  https://zoom-lfx.platform.linuxfoundation.org/meeting/98301969246?password=889b7578-29b6-4be9-96c1-f74cbce812c4

For those who just want to watch: https://livestream.openjsf.org

Invitees
Please use the following emoji reactions in this post to indicate your
availability.

:+1: - Attending
:-1: - Not attending
:confused: - Not sure yet

#177 has no content in it, and i won’t be able to make the meeting tomorrow. Is there anything i can review async?


Adds --with-vagrant option

Motivation:
I am often building packages for an environment (debian8) that does not match my local box (a mac).  Consequently, I generally add a virtual machine provisioner (vagrant) to my package.

Coverage decreased (-1.2%) to 94.536% when pulling 4270bcdfef9c2eae5e2d633d9526840f9bb6f951 on brianbruggeman:master into 615198a2927cb5a913da8fd3196fa28d2d267abb on blue-yonder:master.

Coverage decreased (-0.4%) to 95.364% when pulling 806d614d50562b5668ca0b30cacd23660528d4b8 on brianbruggeman:master into 615198a2927cb5a913da8fd3196fa28d2d267abb on blue-yonder:master.

I do like that this dispatches to vagrant init and does not provide a full template that we have to maintain in pyscaffold. On the other hand, executing vagrant init manually after putup is not difficult at all.  I.e. I do not see a huge benefit of using putup --with-vagrant myproject over putup myproject && cd myproject && vagrant init.
To summarize: I think vagrantfile generation should be decoupled from pyscaffold.

You may have missed my template (see: https://github.com/blue-yonder/pyscaffold/pull/93/files#diff-715cc25ad498112a1b7a826167892f0a), which is more than the standard vagrant init.
At least in my experience over the past 5 years using Vagrant, rarely do my team members know Ruby and Vagrant.


Error: Expression 'undefined' used with directive 'mwlCalendar' is non-assignable!

Hi, after trying various combinations I can't make it work. I'm obtaining the error Expression 'undefined' used with directive 'mwlCalendar' is non-assignable!.
Here is my html:
<mwl-calendar
   calendar-events="events"
   calendar-view="calendarView"
   calendar-current-day="calendarDay"
   calendar-on-event-click="eventClicked($calendarEvent)"
   calendar-edit-event-html="''"
   calendar-delete-event-html="''"
   calendar-on-edit-event-click="eventEdited($calendarEvent)"
   calendar-on-delete-event-click="eventDeleted($calendarEvent)"
   calendar-auto-open="true">
</mwl-calendar>

And here is my controller:
.controller('CalendarCtrl', function ($scope, $modal, moment) {

  var currentYear = moment().year();
  var currentMonth = moment().month();

  //These variables MUST be set as a minimum for the calendar to work
  $scope.calendarView = 'month';
  $scope.calendarDay = new Date();
  $scope.events = [
    {
      title: 'Event 1',
      type: 'warning',
      startsAt: new Date(currentYear,currentMonth,25,8,30),
      endsAt: new Date(currentYear,currentMonth,25,9,30)
    },
    {
      title: 'Event 2',
      type: 'info',
      startsAt: new Date(currentYear,currentMonth,19,7,30),
      endsAt: new Date(currentYear,currentMonth,25,9,30)
    },
    {
      title: 'This is a really long event title',
      type: 'important',
      startsAt: new Date(currentYear,currentMonth,25,6,30),
      endsAt: new Date(currentYear,currentMonth,25,6,60)
    }
  ];

  $scope.eventClicked = function(event) {
    console.log("wtf");
  };

  $scope.eventEdited = function(event) {
    console.log("hi");
  };

  $scope.eventDeleted = function(event) {
    console.log("lol");
  };

  $scope.setCalendarToToday = function() {
    $scope.calendarDay = new Date();
  };

  $scope.toggle = function($event, field, event) {
    $event.preventDefault();
    $event.stopPropagation();

    event[field] = !event[field];
  };
});

I don't know what I am missing. Maybe it is worth to mention that I'm using:
.config(function($interpolateProvider) {
  $interpolateProvider.startSymbol('{$');
  $interpolateProvider.endSymbol('$}');
})

because I'm working with Django. It may be related to the error because once rendered, I obtain the following:

but not sure... any guides?
Thanks! =).

For now you will need to override the templates and replace all interpolated braces with your start and end symbols. I can't think of a nice solution that will also let you override the templates.

Hi, thanks for your help. I'm now able to render the calendar correctly with the assigned events but it still displays the same error and events are not triggered (clicked, edited, etc.). I have this now:
        <mwl-calendar
            calendar-events="events"
            calendar-view="calendarView"
            calendar-current-day="calendarDay"
            calendar-event-click="eventClicked"
            calendar-edit-event-click="eventEdited"
            calendar-delete-event-click="eventDeleted"
            calendar-edit-event-html="''"
            calendar-delete-event-html="''">
        </mwl-calendar>

Also, events are not triggered. One thing I've seen in the example, in the html you have calendar-event-click="eventClick" but in the demo.js the event is called $scope.eventClicked and it still calls it. What am I missing?
Thanks

The click events are angular expressions not variables. As per the example you should do calendar-event-click="eventClicked($event)" where eventClicked is a function on the scope not a variable. It works exactly the same as you would use the ng-click directive.

Damn, I had it that way but modified it in order to try to fix the error because the angular page says the following:

Well, everything seems to be working now (although the error is still there).
Thanks for your help! =).

NP :)


Discussion: Can pages be related to models?

REPOST form Reddit, maybe it is good to have this conversation here as a unique source of knowledge.

Hey Didier, I want to check if the gem can be adjusted to our needs:
We have a platform where users can create their own e-commerce site under a subdomain of our app.
Example: https://carpatitashomeschool.salduu.com/
There, users have an store front with their products and currently each product has its own page
Example: https://carpatitashomeschool.salduu.com/p/descubrimiento-de-america-para-ninos
Those pages are composed by a fixed erb view that renders a action_text field called "description"
I thought I could enhance the basic "description" and offer the ability to create a full featured landing page for the product using maglev.
Like in the product edit form offer them
[on/off] User the new page builder
That would launch the maglev editor for that specific product
So in the model it could be something like
Product has_one :maglev_page, optional: true
and in the view it could be
<% if product.maglev_page.present? %>
  <%= render full_featured_maglev_page %>
<% else %>
  <%= render partial: old_fixed_page_with_action_text_description %>
<% end %>

I also don't want anyone to create their page and override any of my pages / or other user pages, so "pages" don't need to have a path (*can't have), because the path is bound to the product_path
Do you think this is possible? I think it is a very common use case.

👋 @silva96. Of course I was aware of that use case but since I haven't faced it yet myself in my applications, I avoided to implement it.
My concern was about how you deal with the "unicity" of the product page / template. Sorry, it's quite confusing.
Let me explain.
So let's say you've got a Product model and you already installed Maglev to handle the other pages of your e-commerce site. And you've got 2 products: Product A & Product B.
How different should be the 2 product pages?
Should they have the same sections, like a banner maybe, a product detail section which should be required and maybe dynamic sections that might appear on the Product A page but not on the B one.
For instance, should the banner section share the same content between the 2 products?
Or do you have to treat the 2 products like 2 separate pages?
Do we want the content editors to require the "product details" section at a specific position of the page.
I think you got the idea, there are quite a few scenarios which depends on lot of the context.
This why I need feedbacks from users like you before tackling the right implementation :-)

Actually our use case is much more simple. Each product is a unique page that can be built from zero and can have dozens of sections and blocks.
They want to have something more like a product landing page instead of a regular ecommerce product page.
Our users sell infoproducts and they are using costly themes built for wordpress and redirect traffic to our payment gateway.
I want them to have the page editor inside the platform instead.
For instance, this is built outside our platform but users want something like this (wordpress or similar).
https://soyfunnelbox.com/oferta
Currently our users can only have pages like this (our platform)
https://carpatitashomeschool.salduu.com/p/descubrimiento-de-america-para-ninos

I see. This is a special case in my point of view. A product -> a landing page. An user can have one or many products. In that right?
First thing, Maglev (MIT version) is only mono-site. So the open source won't work for your use case. The Maglev PRO which is multi-sites (so multi-landing pages in your case) will.
In other words, your Maglev site (and not page, this is the difference) will belong to a product.
I guess you could trick the system and try to use the MIT version of Maglev with the idea of that a page belongs to a product. First you'll fight against the Maglev framework and then, you'll get some unwanted side-effects like your users will see other asset's users because they share the same site. You see what I mean?

Yes I see it, I actually made a concept proof on that path and kind of worked but felt very hacky, I will probably build a minimalist version of a page builder inside the platform near in the future. Thanks for the clarifications!


remove unused packages

I did some cleanup of the package dependencies. It copied basically all of my changes over.
It turns out, you have to build json-c with autotools instead of cmake to make it recognised by  sway.

:+1:
Will take a look.


Add comments to the framework & included modules

Resolves #15

the pr size bot is drunk

Wait, no it's not.


Invert scale produces unexpected results

You simply need to invert the green and red values in this case.

I added a few validations in the option panel that helps the configuration.


Issues when logging/authorizing-chrome-extension on Tempo plugin from Jira

Right now I'm not sure what domain is causing problem but, so far I've been struggling with this since yesterday and that was solved today by removing temporarily the hosts file (ad + malware), and then logging in. At the moment I'm still using the hosts file and I haven't had any other issues (although the console is full of errors).
However, I'll be checking this issue again during this week but, I just wanted to open the ticket if anyone else is having the same issues.
Btw, thanks for this project!

You should open the dev-tool in your browser and look under the network tab and see which domains that ain't loading (404)
But it seems like you have something like bitbucket or atlassian.com in your hosts file.
Can you verify any of my suspected domains?


Suggestion - A way to notify users about this functionality

Hello,
when the users go in the page with the list of items (only the first time?),
would be nice to have a way to show a short and strategic self-moving of the cells,
to inform the users that is possible to slide the cells
Thanks for your work!

@matteopiccioni yes, you can achieve it very easily
just call
await contextMenu.ScrollToAsync(40, 0, true);
contextMenu.ForceClose(true);

Hi Andrei Misiukevich,
I'm relative new to Xamarin and have a question.
How can I access a single element of ContextMenu when used in a ListView to trigger that ScrollToAsync on a single View only?

@CodeOfDarkness i think you should do it through code

I don't know how to access it by code

@CodeOfDarkness if you use ListView, you may create custom ViewCell e.g.

@AndreiMisiukevich Ok thanks for your quick help 👍


Define remaining work to remove Font Awesome

User story
As a front-end developer, I want to know the remaining uses of Font Awesome, so that I can replace them and remove the font.
Resources

bug 1451261 - Replace FontAwesome with SVG icons

Acceptance criteria

Remaining uses of Font Awesome are identified
Remaining tasks are estimated

@schalkneethling I'll put my notes on bug 1451261, unless you already have a document. I'm not sure if there is enough remaining work for an epic + issues, or if an issue would cover the remaining work.

Thanks @jwhitlock I reckon an issue will do the trick.

@jwhitlock Do you want me to close this one and then open separate issue(s) for the actual work? I want to see if I can break this into either two 2 pointers or maybe even four 1 pointers. Let me know.

Let's leave the investigation in Sprint 2 and open new implementation issue(s) for sprint 3.


Issue on 2.30 TLD

The warmth bunus does not stack and if is stacking turned off, bedroll have +0°C warmth bonus.
TY for updating <3

This mod is not currently maintained.


[MPG][.Net] Unexpected diff for CognitiveSearch test project

There is no InputPrimitiveType to represent char, it maps to string instead
Corresponding declaration in swagger:
https://github.com/Azure/azure-rest-api-specs/blob/e606243e5297312781dd7dbfd7ab76d2329cc088/specification/search/data-plane/Azure.Search/preview/2019-05-06-preview/searchservice.json#L3138-L3143

this might depend on the generic representation of decorator in codemodel.json work

Regardless typeSpec will support char as first-class type or not, I think we should add InputPrimitiveType.Char to represent CSharpType char in the .NET generator.


Proof of Possession Sample

Hi,
As  identityserver3 supports it is there a sample how to use pop tokens on the client side (is it possible at all)?
bets regards Murat

This is the wrong repo for that. Our PoP work is here:
https://github.com/IdentityModel/IdentityModel.Owin.PopAuthentication
and
https://github.com/IdentityModel/IdentityModel.HttpSigning


Add actual newlines not literal \n's to the strings

I thought I could get away with just the quote change in https://github.com/buildkite/agent/pull/1494 but missed the newlines were literals rather than interpreted. Maybe I should have just merged #1206 😅

buildkite-agent-windows-amd64.exe.zip

buildkite-agent-windows-amd64.exe.zip

It works :tada: I found another missing character, the trailing quote of the beforeEnvFile interpolation.


Add minimal support for missing instructions

I made a short list of unsopported instructions using the serial testcases, with examples:

bt eax, edx
movsx eax, al
cmpxchg dword ptr [edi], ecx

These instructions (probably) should be supported to obtain correct resuts in our testcases.

An informal survey on missing instructions (made using a large number of x86 binaries from Debian) is available here:
fstp   :3268
fld    :2729
movsx  :1276
fmul   :1009
fild   : 860
lock   : 677
fxch   : 579
fldcw  : 486
faddp  : 451
fadd   : 426
fst    : 398
fnstsw : 365
sahf   : 365
fistp  : 248
fldz   : 246
fdiv   : 190
fnstcw : 188
fsub   : 174
fucom  : 166
fmulp  : 143
fld1   : 108
fucompp: 102
fucomp :  97
idiv   :  97
fdivp  :  83
fsubr  :  69
fchs   :  66
fsubp  :  65
fdivrp :  46
fsubrp :  46
cwde   :  39
setle  :  29
fdivr  :  25
fsqrt  :  23
jnp    :  20
setge  :  15
setl   :  12
bswap  :   9
fabs   :   8
frndint:   7
cdq    :   6
fist   :   4
setnp  :   4
bsr    :   3
cli    :   1
das    :   1
in     :   1
insb   :   1
int    :   1

I'm surprised to see movsx on this list

Hi! That list should be updated. movsx is currently supported along with setcc and jnp. Regarding FPU instructions, there's no plan to support them in the near future.

idiv seems to be a very useful instruction to support, since it is used to check some overflow constrains that we want to verify..

The most frequently used instructions has been implemented (FPU instructions are not included, there is no plan to implement them in the near future), so I'm closing this issue.


[Event Code Change Request] Table 39 - "Purchase Line" - OnBeforeJobTaskIsSet

Hello AL-Team,
the current implementation of the event publisher "OnBeforeJobTaskIsSet" allows only to force the return value to TRUE, but not to force it to FALSE.
I suggest the following change:
[External] JobTaskIsSet() : Boolean
// >> Change
// JobTaskSet := FALSE;
// OnBeforeJobTaskIsSet(Rec,JobTaskSet);

// EXIT(
//   (("Job No." <> '') AND ("Job Task No." <> '') AND (Type IN [Type::"G/L Account",Type::Item])) OR
//   JobTaskSet);

JobTaskSet := ("Job No." <> '') AND ("Job Task No." <> '') AND (Type IN [Type::"G/L Account",Type::Item]);
OnBeforeJobTaskIsSet(Rec,JobTaskSet);

EXIT(JobTaskSet);
// << Change

This would allow the event subscribers to change the value of JobTaskSet both ways.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Deprecating weak cryptographic standards (TLS 1.0 and 1.1) in
#Azure DevOps Services - Azure DevOps Blog

https://devblogs.microsoft.com/devops/deprecating-weak-cryptographic-standards-tls-1-0-and-1-1-in-azure-devops-services/

@tmamb - What Docs article is your issue related to? Please add a link to it in this issue.

@tmamb We respond to Power Apps documentation questions and issues here. We do not handle Azure DevOps questions or issues unless it is related to Power Apps.


--extensions-dir '[Your Path]' have a bug.

Issue Type: Bug
I used this bat code to launch the Visual Studio Code and finded a bug.
@echo off
"%cd%\VSCode\Code - Insiders.exe" --extensions-dir ‘.\VSCode\plugins’
Such as you move the VSCode to a root of disk, and run this bat. A directory will be created and it named '. (It's a single mark and a dot).  I can't to delete and open the directory.
Using the DiskGenius, I can see this:
I:\'.\VSCode\plugin\'
The last directory named ' (a single mark) and "I" is my USBdisk.
VS Code version: Code - Insiders 1.51.0-insider (7a3bdf4ee9588755d447aa1c3b5db4a123fc11a9, 2020-10-23T07:36:26.010Z)
OS version: Windows_NT ia32 10.0.19041

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz (6 x 2904)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: disabled_offopengl: enabled_onprotected_video_decode: enabledrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
7.84GB (1.24GB free)

Process Argv
--extensions-dir .\VSCode\plugins --crash-reporter-id d256dd06-8d84-4fb7-bd44-0237bc558342

Screen Reader
no

VM
0%

Extensions (18)

Extension
Author (truncated)
Version

code-gnu-global
aus
0.2.2

doxdocgen
csc
1.1.0

c-cpp-compile-run
dan
1.0.12

vscode-pull-request-github
Git
0.20.1

vscode-gnu-global
jay
0.3.3

better-cpp-syntax
jef
1.15.0

vscode-language-pack-zh-hans
MS-
1.51.0

remote-containers
ms-
0.146.0

remote-ssh
ms-
0.55.0

remote-ssh-edit
ms-
0.55.0

remote-wsl
ms-
0.51.0

cmake-tools
ms-
1.4.2

cpptools
ms-
1.1.0-insiders3

cpptools-extension-pack
ms-
1.0.0

vsliveshare
ms-
1.0.3046

vsliveshare-audio
ms-
0.1.91

vsonline
ms-
1.0.3073

cmake
twx
0.0.17

(1 theme extensions excluded)

        string launchCommand = Command.Replace(" ", "").Length > 0 ? Command : Config[0];
        if (launchCommand.Length > 0) launchCommand += " ";
        if (!Command.Contains("--extensions-dir")) launchCommand += "--extensions-dir \'" + Config[2] + "\'";
        Process = System.Diagnostics.Process.Start(Config[1], launchCommand);

In this code, if I used " it will missing the first double mark. result is VSCode crashed, so I changed to using single mark.
I'm sorry to I have a bad English and I'm a 1 Year  student of University in China.

@echo off
"%cd%\VSCode\Code - Insiders.exe" --extensions-dir ‘.\VSCode\plugins’

The problem is you used the ‘ character in ‘.\VSCode\plugins’ instead of the correct ' as in .\VSCode\plugins.
This is an issue with Command Prompt, not VS Code.


Gene tree shows correct locus tags, but organism name is not correct

I noticed that the MSA and the gene tree line up correctly when the locus tags are displayed, but not when the organism names are displayed.
In the screenshot, the shortened gene is actually Brucella sp. F5/99, but B. suis 1330

When looking at the locus tags, the shortened one is correctly lined up with the Brucella sp F5/99 gene.

I was aligning members of PLF_234_00000374

@ARWattam After careful analysis I don't believe this is a bug. A sequence can be shorter but have gaps in the back OR the front of the alignment. I checked the sequences in question and they match to the AA record from the feature table. If you recreate the alignment check the front AND back of Brucella sp. F5/99 and B. suis 1330


GHC 8.10 support

Once staging is merged, will Plutarch continue to support GHC 8.10?
I think it should, because not all smart contracts will be moving over to GHC 9.2.

I think yes, and AFAICT the core library should still compile fine, even if the tests don't.

So, when using the flake.nix from this PR, and changing ghcVersion to ghc8107 - I seem to be getting compile errors for fourmolu during nix develop-
src/Ormolu/Utils/Extensions.hs:24:1: error:
    Could not find module ‘Distribution.Utils.Path’
    Perhaps you meant
      Distribution.Utils.IOData (from Cabal-<IP_ADDRESS>)
      Distribution.Utils.MD5 (from Cabal-<IP_ADDRESS>)
      Distribution.Utils.Base62

is it expecting a more up to date cabal version?

FWIW I did get that flake and everything to work with 8.10.7 by removing fourmolu.

@TotallyNotChase Why not modify Plutarch's flake.nix? Did you notice it requiring more than compiler version change?

¯_(ツ)_/¯ I'm just not very good with nix. It works flawlessly on my testbed with ghc8107 after removing fourmolu. I'm sure you could make it work with fourmolu as well by modifying the nix a bit.

For the record, it only requires changing compiler version, removing fourmolu/ format checks and adding back HLS.
Diff: https://github.com/Plutonomicon/plutarch/compare/staging...srid/ghc8107
Everything but the examples build:
https://hercules-ci.com/github/Plutonomicon/plutarch/jobs/353
I wonder if all of this can be parameterized so we can do nix develop using either of the compilers.

I think we should at the very least in CI make sure compilation with GHC 8.10.7 works.

I changed the name of the issue.

#188 adds CI for the library, but we are not running tests.

The tests can't be run. They depend on GHC 9.2.1.

Only a couple of test suites depend on GHC 9, Fields and API I think. We could guard them in examples/Main.hs with #if MIN_VERSION_GLASGOW_HASKELL(9,1).

@blamario Do you know of a way to check for GHC version in plutarch.cabal?

https://cabal.readthedocs.io/en/3.4/cabal-package.html#conditions
if impl(ghc >= 9.0) should do it.

I didn't know you could do that!

8.10 generates a slightly different error for pattern match failures, and that broke one golden test:

I'm starting to think most compilation outputs are rather flaky

At least it is flakiness across space (ghc versions) rather than flakiness across time. The latter being non-deterministic is the annoying one.

This also happens in 8.10 (but not 9.2):


Green Horizontal Lines in Diff Tab

Version 1.1.3
Language: Javascript
Zoom level: doesn't matter.
Reproducible only with this theme.
Screenshot:

Great theme! Thank you!

@adioanca-rpma I'm having the same problem. Did you find a way around it?

@adioanca-rpma I'm having the same problem. Did you find a way around it?

Haven't searched. I switched to other themes.

Guys, do you know how disabled this lines?


Update install-or-uninstall-language-packs-0.md

Fixed Windows Locale Codes for German and Greek as per issue 501
https://github.com/MicrosoftDocs/OfficeDocs-SharePoint/issues/501

Thanks for fixing, @dmpe!

@dmpe thanks,  The original error is still there (wrong codes for German and Greek). I will try to fix without breaking the tables if I can. My apologies

@MikePlumleyMSFT  I see that the right codes are in the german version (https://docs.microsoft.com/de-de/sharepoint/install/install-or-uninstall-language-packs-0)  of the document (with one formatting error). In the English version the wrong codes are shown for German and Greek. As I don't quite understand how the different versions of the document are generated, perhaps @dmpe  could assist to correct as follows:
|| German |de-de | 1031
|| Greek | el-el | 1032

@nenonix, I've updated with the codes above. (Translation is a separate process, so we can update the English version and it goes into a queue to get translated to other languages.)


Fix markdown linter github action

Description
This PR fixes markdown GH action linter and switched to action instead of image.
Context: Previously we introduced an issue https://github.com/Dynatrace/dynatrace-operator/commit/ce23e48d37fc00046fe829d1b8c15506eb9555fe
How can this be tested?
see GH comments
Checklist

[x] Unit tests have been updated/added
[x] PR is labeled accordingly with a single label
[x] I have read and understood the contribution guidelines

Codecov Report

Merging #2176 (d781f2b) into main (02d76c7) will not change coverage.
Report is 1 commits behind head on main.
The diff coverage is n/a.

:exclamation: Current head d781f2b differs from pull request most recent head 97bdb3d. Consider uploading reports for the commit 97bdb3d to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@           Coverage Diff           @@
##             main    #2176   +/-   ##
=======================================
  Coverage   64.30%   64.30%
=======================================
  Files         242      242
  Lines       15286    15286
=======================================
  Hits         9830     9830
  Misses       4732     4732
  Partials      724      724

@chrismuellner fixed.


API overview for using typings programmatically

Is there an API overview for using typings programmatically?

Not yet. The main require will give you access to all the currently used commands, but there's nothing officially documented - probably not until 0.7 anyway, when I'll be splitting the CLI and core apart.

I switched from tsd to typings in my yeoman generator, which installs typings among other stuff.
I had to do some reverse engineering to be able to install packages from my generator. I hope that it will help you @philkunz (but keep in mind that any upgrade of typings may break that, as @blakeembrey said it's not documented yet).
// Typings
class TypingDependency {
  constructor(name, ambient, source) {
    this.name = name;
    this.ambient = !!ambient;
    this.source = source ? source : 'dt';
  }
}

var typingsToInstall = [
  new TypingDependency('jquery', true),
  new TypingDependency('angular', true),
  new TypingDependency('angular-mocks', true),
  new TypingDependency('angular-route', true),
  new TypingDependency('jasmine')
];

var typings = require('typings');
var cwd = this.destinationPath(); // or __dirname + '/' not in yeoman
// Init the "typings.json" file
typings.init({cwd: cwd}).then(() => {
  var typingCwd = cwd + 'typings/';
  var registry = require('typings/dist/lib/registry');
  typingsToInstall.forEach((typing) => {
    var version = registry.parseRegistryPath(typing.name).version;
    // Get information about version etc
    return registry.getVersions(typing.source, typing.name, version).then((project) => {
      var versions = project.versions;
      version = versions[0];
      // Finally can install the dependency
      return typings.installDependency(version.location, {
        name: typing.name,
        cwd: typingCwd,
        saveDev: true,
        source: typing.source,
        ambient: typing.ambient
      }).then(() => {
        // ... some stuff when a dependency is installed, like loging
      });
    });
  });
});

The public API of typings-core will not break on a minor release going forward. Does using the completion provided by TypeScript help you? I wanted to use typedoc, but it can't compile TS 1.8 projects yet.

Also, I'd love feedback on the programmatic API - what should be easier?

I didn't use Typescript in my case, can't tell you for completion. But the function name etc talk by themselves, so it was not that difficult.
My main wish for the programmatic API is provide an easy way to install one or more typings. It should search for the version if the user don't provide them etc. I would like to use something like that:
var typings = require('typings');
typings.install({name: 'jquery', version: 'xx', saveDev: false, save: true, source: 'dt'}).then(() => {
  return typings.install([{name: 'angular', version: 'xx', saveDev: true}, 'angular-mocks', 'jasmine');
});

Last point: the ambient part is not clear to me at all, so to finish my task I tried to install dependencies with our without the flag to see if it works.

Seems it mostly does this already. There could be a function exposed that allows you to build the registry paths with versions as you've requested. If you want to install multiple typings, I recommend you use Promise.all for now. I'd check the install script in this repo for how it's done, and you could follow the same process: https://github.com/typings/typings/blob/master/src/bin-install.ts#L76-L78.
On the versions and such, we can expose the function I mentioned above. That would change the code to:
typings.installDependencyRaw(typings.buildRegistryPath({ name: 'jquery', source: 'dt', version: 'xx' }), { save: true })

The difference between ambient or not is portability. Basically, using ambient typings is a good fallback for now but those type definitions can not be re-used in a dependency tree situation (E.g. as a module author).

That would be nice to have it exposed yes, way easier than my hack ^^
Thanks for the explaination, I think I will have to read a little bit more about this precise subject.

Stumbled over this issue since I used typings.install programmatically in version 0.6.x and it doesn't work anymore ;-)
With v2.0.0, this works for me:
var typings = require("typings-core");
typings.install({
    cwd: "directory with typings.json in"
}).then(function () {
    console.log("yoooo")
});

Other commands work as well, see https://github.com/typings/core/tree/master/src


[AppService] Fix #17850 - prevent duplicate rules for service endpoints

Description
Small change to ensure duplicate rules are not created for service endpoints (subnet/vnet) and make remove command case-insensitive.
Testing Guide
az webapp config access-restriction add -g  -n  -p 2 --subnet  --vnet-name
az webapp config access-restriction add -g  -n  -p 2 --subnet  --vnet-name
run twice should throw an exception - even if casing is different
az webapp config access-restriction remove -g  -n  --subnet  --vnet-name
rule should be removed - even if casing is different.
History Notes

[App Service] az webapp/functionapp config access-restriction add: Prevent duplicate rules using service endpoints.
[App Service] az webapp/functionapp config access-restriction remove: Remove service endpoints are case-insensitive
This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR title and description has followed the guideline in Submitting Pull Requests.

[x] I adhere to the Command Guidelines.

[x] I adhere to the Error Handling Guidelines.

@Juliehzl is this waiting for you?

@Juliehzl would you mind approving this - it has been hanging for a while.
Thanks :-)


Multi-repositories drop down not working

Issues:
The main repository is replaced by a blank link, linking back to the home page
The button spacing is off

See https://rcos.io/projects/smaltby/canary/profile


Can format dot (.) to (,) for decimal places

As for Brazil and other countries we use formats like:
Brazil = 1000,00
US     = 1000.00

And also format thousands separators like:
Brazil = 1.000,00 / 1.000.000,00
US     = 1,000.00 / 1,000,000.00

@wggley For the feature there is the need for another input type (Currency based on region?).

I'm closing the issue for now.


invalid option nameoppz6aro/terraform_docs.sh: line 2: set: pipefail

Hi,
I've just setup my repo to use pre-commit. Here is my .pre-commit-config.yaml:
repos:
  - repo: git://github.com/antonbabenko/pre-commit-terraform
    rev: v1.50.0
    hooks:
      - id: terraform_docs

When trying to pre-commit I've the following error:
$ pre-commit try-repo -a git://github.com/antonbabenko/pre-commit-terraform terraform_docs
===============================================================================
Using config:
===============================================================================
repos:
-   repo: git://github.com/antonbabenko/pre-commit-terraform
    rev: c7d6d002ed510c1a3e63af6d0e132e8abf2448d4
    hooks:
    -   id: terraform_docs
===============================================================================
[INFO] Initializing environment for git://github.com/antonbabenko/pre-commit-terraform.
Terraform docs...........................................................Failed
- hook id: terraform_docs
- exit code: 2

: invalid option nameoppz6aro/terraform_docs.sh: line 2: set: pipefail

Anyone known what's wrong ?
Thanks

What shell are you using?

I'm using zsh + oh-my-zsh

It looks like the script is running under something other than bash.
Can you run this and show the output:
$(command -v bash) --version

Here is the output:

Are you on windows, by any chance?
https://askubuntu.com/questions/1044280/set-eo-pipefail-not-working-in-windows-subsystem-for-linux-ubuntu-16-04

Yes I'm using WSL2 with a ubuntu distrib
Le lun. 14 juin 2021 à 15:27, Robin Bowes @.***> a
écrit :

Are you on windows, by any chance?
https://askubuntu.com/questions/1044280/set-eo-pipefail-not-working-in-windows-subsystem-for-linux-ubuntu-16-04
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/antonbabenko/pre-commit-terraform/issues/204#issuecomment-860684052,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABJOGZXTRWYMEQD7NAN3PTTSX7T3ANCNFSM46VCGDLQ
.

--
Jean-Jacques Martrès

I've change my git setup to git config --global core.autocrlf false and now I have the following error:
$ pre-commit try-repo -a git://github.com/antonbabenko/pre-commit-terraform terraform_docs
===============================================================================
Using config:
===============================================================================
repos:
-   repo: git://github.com/antonbabenko/pre-commit-terraform
    rev: c7d6d002ed510c1a3e63af6d0e132e8abf2448d4
    hooks:
    -   id: terraform_docs
===============================================================================
[INFO] Initializing environment for git://github.com/antonbabenko/pre-commit-terraform.
Terraform docs...........................................................Failed
- hook id: terraform_docs
- exit code: 1

Error: Invalid escape sequence: The symbol "/" is not a valid escape sequence selector.

Any idea ?

You may need to re-download/install the pre-commit hooks for that to take effect.

Hi @jjmartres
I think you just did something wrong. It's not a bug for sure.
Feel free to reopen the issue with additional details about your system and steps to reproduce it if not yet fixed.
22:06 test-204 git:(main +)
➜ pre-commit try-repo  -a /mnt/c/Users/vm/code/open-source/pre-commit-terraform terraform_docs
===============================================================================
Using config:
===============================================================================
repos:
-   repo: ../../mnt/c/Users/vm/code/open-source/pre-commit-terraform
    rev: c013378066cd2cebe8404a3acbfc7859e960be02
    hooks:
    -   id: terraform_docs
===============================================================================
[INFO] Initializing environment for ../../mnt/c/Users/vm/code/open-source/pre-commit-terraform.
Terraform docs...........................................................Passed

22:06 test-204 git:(main +)
➜ uname -a
Linux DESKTOP-C7315EF 5.4.72-microsoft-standard-WSL2 #1 SMP Wed Oct 28 23:40:43 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

22:06 test-204 git:(main +)
➜ pre-commit --version
pre-commit 2.14.0

22:06 test-204 git:(main +)
➜ terraform-docs --version
terraform-docs version v0.15.0 f3b6eab linux/amd64

22:06 test-204 git:(main +)
➜

I had the same issue on MacOS Sonoma 14.6.1: invalid option nameache/pre-commit/repocc26z53r/hooks/terraform_fmt.sh: line 2: set: pipefail
It was caused by having git config --global core.autocrlf true. Changing this alone did not fix things because the git repo was cached in ~/.cache/pre-commit. Deleting ~/.cache/pre-commit fixed it.


DropwizardAppExtension does not cleanup database resources

When a suite of tests is torn down, the open database connections to PostgreSQL remain. Thus, when multiple suites are run from the same regression artifact, eventually the tests exceed the maximum connections configured for the PostgreSQL database.
I will attempt to look into why this is happening, but if someone has an answer/workaround already, I would appreciate it.
Related Version Information:

Dropwizard: 2.0.23
JDBI: 3.20.1
PostgreSQL JDBC Driver: 42.2.22
Liquibase: 4.4.0

EDIT: The idle connections not being cleaned all last executed /* Health Check */ SELECT 1.

The issue is not in fact DropWizard but is Liquibase; forcing Liquibase to 4.3.5 fixes the issue.

The issue is not in fact DropWizard but is Liquibase; forcing Liquibase to 4.3.5 fixes the issue.

Thanks for the update!
Did you open a bug report with Liquibase which we could follow?


Updating for Socket.IO 1.0

Socket.IO 1.0 has released - http://socket.io/blog/introducing-socket-io-1-0/. Has any plan for update the library for working with Socket.IO 1.0.?

Closing as Duplicate of #75


Test time display format

Hello,
Test times below 1 second should rather be displayed in ms instead. As this is much easier to read and interpret.
Thanks,

Just a quick question, there's a "Total test time:" label at the top. Should this also be formatted? or just the test cases time?


EmrEtlRunner: bump AMI version in example config to 5.9.0

Since 5.8.0 Amazon at last started to bundle EMR AMI with AWS SDK 1.11.*. Older versions introduced lots of problems with API incompatibility, shading etc. Also it now provides Application History, which should make debugging a lot easier.
Though, there's more notable changes, like Spark 2.2 and it would likely take more effort than just bumping AMI, but still would be great to move forward and not let Snowplow ecosystem stale.

Can we fit this into R95 @BenFradet?

we could the changes should be lightweight


Assert log.dirs Property not Misconfigured

Describe the issue
Something wrong with the task Assert log.dirs Property not Misconfigured. The error shows 'dict object' has no attribute 'name'. It seems that error come from kafka_broker_properties. In my configuration, I use kafka_broker_custom_properties. It's normal when debug printing but failed with kafka_broker_properties (default, I dont set this one in my config)
To Reproduce
Steps to reproduce the behaviour:
Expected behaviour
A clear and concise description of what you expected to happen.
Inventory File
inventory.txt
Logs
failure.txt
Environment (please complete the following information):

OS: Ubuntu20.04
CP-Ansible Branch: 7.3.x
Ansible Version: ansible 2.13.2

Additional context
Add any other context about the problem here.

The above config works well with CP-Ansible: 7.0.x. It seems similiar to ticket 1183. Somehow have conflict between group level and host level setting, I have to remove kafka_broker_custom_listeners in the setting of each host. It works now. So, the question is where I can set kafka_broker_custom_listeners for each host in case a host have multiple ip address and I want to set  listener on a specific ip?


How can I specify version of python (3 or 2) for build of boost-python (how can I pass arguments to bootstrap?)

I've read Brief overview section and do understand basic concepts. [Yes]
I've read F.A.Q. section and haven't found an answer to my question. [Yes]
I've read Code of Conduct, I promise to be polite and will do my best at being constructive. [Yes]

I've checked that all the hunter_add_package/find_package API used by me in example is the same as in documentation. [Yes]
I'm using latest Hunter URL/SHA1. [Yes]

Directly related: #514

I'm building on [Linux Ubuntu].
[I'm using system CMake]
I'm not using toolchain
Specific problem:

Locally, my Hunter-based boost builds detect Python 3 by default.
On Travis-CI server, the same builds detect Python 2 by default instead. I'm not certain why, since python3-dev is installed.
I need Python 3 to build the python3 component, specifically, so my Travis-CI build fails.

I need to be able to specify to boost the python version.
According to various sources, this can be done by passing the information to bootstrap.sh, not to b2:
./bootstrap.sh --with-python=/usr/bin/python3 --with-python-version=3.5 --with-python-root=/usr/lib/python3.5
Alternatively, I can modify user-config.jam file.
Inspecting the Hunter boost package documentation, I didn't find any way to do either one. Please, advise.
P.S. Adding PYTHON_buildid=3 (which produces b2 -s buildid=3) alone doesn't seem to do anything (buildid is somehow related).

I guess we need new CMake option to control the behavior, similar to:

https://docs.hunter.sh/en/latest/packages/pkg/Boost.html#cmake-options

Alternatively, I can modify user-config.jam file.

Sounds better than adding arguments to ./bootstrap.sh, in my opinion.

@ruslo thank you, I'm looking into what would be needed for the .jam files

@ruslo
So, what exactly do we want, a low-level way to customize user-config.jam, such as this?
hunter_config(
    Boost
    VERSION ${HUNTER_Boost_VERSION}
    CMAKE_ARGS
JAM_USER_CONFIG=using python : 3.6 : /usr/bin/python3 : /usr/include/python3.6m : /usr/lib ; #<potentially other lines for user_config.jam here separated by semicolons>
)

Or do we want something really high-level, as shown below?
hunter_config(
    Boost
    VERSION ${HUNTER_Boost_VERSION}
    CMAKE_ARGS
 REQUESTED_PYTHON_VERSION=3.6 #attempts to find python 3.6 on the system, if it is found, configures user_config.jam with the appropriate arguments, otherwise doesn't touch user_config.jam at all
)

P.S. apparently, user-config.jam should be copied to / created in the user's home directory during the Boost build. (Verified in the 1.68 boost instructions.
Do we still want to make one, or would that be sort of a sandbox violation? Perhaps passing arguments to ./bootstrap.sh would be a better way, considering this?

Or do we want something really high-level, as shown below?

High-level. Variables with full paths should not be part of CMAKE_ARGS since they will be saved in cache, and cache should be relocatable.

P.S. apparently, user-config.jam should be copied to / created in the user's home directory during the Boost build. (Verified in the 1.68 boost instructions.)

I think all work with jam file is already handled correctly, you just need to inject your part there.
This line did the trick I guess:

https://github.com/ruslo/hunter/blob/master/cmake/projects/Boost/schemes/url_sha1_boost_library.cmake.in#L212

Work-around: by default, the Boost build engine will use whichever python version it finds linked to "python" in the system path. Hence, if you ensure that "python" in the path links to the desired version of python, it will build the Boost Python library for the correct version.
NB: On Ubuntu, by default, python 2 and python 3 are both installed, but python is by default linked to python2.7. update_alternatives can be used to change this choice if one has sudo access. I'm not sure simply using "alias python=python3" in the shell would work, since it only changes the alias on the current shell level.

@ruslo , ага, спасибо, I can see where the file is being written to and appended with various things.
I guess we'll want to somehow uncover if the requested python version is available.
I can see there is a very basic find_python.cmake script here:
https://github.com/ruslo/hunter/blob/master/scripts/find_python.cmake
It won't be sufficient. I already have some code I can adapt for this, perhaps tomorrow, originally borrowed from the OpenCV project.
Shall I make a separate cmake file, e.g. Boost/schemes/find_requested_python.cmake?

I can see there is a very basic find_python.cmake script here

This script is a workaround for some issues when Hunter used for cross-compiling. Standard root paths removed from search in this case (e.g. we don't need system macOS libraries when we are building iOS), and you can run isolated script with find_package(PythonInterp) call to find tools in host environment. Hence in Boost case probably find_package(PythonInterp) will be what we actually need.
Also in case if https://github.com/ruslo/hunter/issues/388 will be solved, then you can just do hunter_add_package(python) + find_package(python CONFIG REQUIRED).

I think it could make sense to pass, as components, "python2" or "python3"

Components for hunter_add_package taken from Boost --with-<component> option. List of components from documentation:

https://docs.hunter.sh/en/latest/packages/pkg/Boost.html#boost

Example from build logs (this part of log produced by Boost itself, not by Hunter):

https://travis-ci.org/ingenue/hunter/jobs/451966777#L1393-L1424

There are no python2 or python3 component there so we should not use this approach.
I know that in this case COMPONENTS from hunter_add_package doesn't match find_package. Such peculiarities should be mentioned in documentation, e.g. see boost.math:

https://docs.hunter.sh/en/latest/packages/pkg/Boost.html#math

Or we could go with the original CMAKE_ARGS REQUESTED_PYTHON_VERSION=3.6 option in hunter_config.

Sounds good for me :+1:

@ruslo , please let me know if there is anything you'd like me to change in or add to the PR.


[release-1.17] Use helm repo for integration tests instead of tarballs

This is an automated cherry-pick of #42864

/retest

/retest

@ericvn @johnma14 planning to get this in for 1.17? This would be needed if we would like to have Helm Promotion to Beta in 1.17

Changes are mostly to the integration test framework. LGTM for 1.17

Thanks @johnma14 ! @pmerrison might have to approve as well.