Upgrade of Insights 2022.3 does not work

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request

Module Version

4.0.2

Affected Resource(s)

Insights for ArcGIS

Configuration Files

{
    "AllNodes": [
        {
            "NodeName": "ServerHost",
            "Role": [
                "Server"
            ]
        },
        {
            "NodeName": "PortalHost",
            "Role": [
                "Portal"
            ]
        }
    ],
    "ConfigData": {
        "Version": "10.9.1",
        "OldVersion": "10.9.1",
        "ServerContext": "arcgis",
        "PortalContext": "portal4arcgis",
        "ServerRole": "GeneralPurposeServer",
        "InsightsVersion": "2022.3",
        "OldInsightsVersion": "2022.1.1",
        "DownloadSetups": false,
        "DownloadPatches":false,
        "Credentials": {
            "ServiceAccount": {
                "UserName": "user",
                "Password": "password",
                "IsDomainAccount": true,
                "IsMSAAccount": false
            }
        },
        "Server": {
            "LicenseFilePath": "C:\\data\\installation\\ArcGISGISServerStandard_ArcGISServer_1144804.prvc",
            "Installer": {
                "Path": "C:\\data\\installation\\ArcGIS_Server_Windows_1091_180041.exe",
                "InstallDir": "C:\\bin\\ArcGIS\\Server",
                "InstallDirPython": "C:\\bin\\Python27",
                "EnableArcMapRuntime": true,
                "EnableDotnetSupport": true,
                "PatchesDir": "C:\\data\\installation\\arcgisPatches",
                "PatchInstallOrder": [
                    "ArcGIS-1091-S-SP-Patch.msp",
                    "ArcGIS-1091-S-MR-PatchB.msp",
                    "ArcGIS-1091-S-Log4j-PatchB.msp",
                    "ArcGIS-1091-S-SQ-Patch.msp",
                    "ArcGIS-1091-S-UNDM1-Patch.msp",
                    "ArcGIS-1091-S-VRPG-Patch.msp",
                    "MappingandChartingServer1091Update1.msp",
                    "ArcGIS-1091-S-MPS-Patch.msp",
                    "ArcGIS-1091-S-SEC2022U1-PatchB.msp",
                    "MappingandChartingServer1091Update2.msp",
                    "ArcGIS-1091-S-UNDM2-Patch.msp",
                    "ArcGIS-1091-S-SEC2022U2-Patch.msp",
                    "MappingandChartingServer1091Update3.msp",
                    "ArcGIS-1091-S-UNDM3-Patch.msp",
                    "ArcGIS-1091-S-GSQ-Patch.msp"
                ]
            },
            "ServerDirectoriesRootLocation": "C:\\arcgisserver\\directories",
            "ConfigStoreLocation": "C:\\arcgisserver\\config-store",
            "LocalRepositoryPath": "C:\\arcgisserver\\local",
            "ServerLogsLocation": "C:\\arcgisserver\\logs",
            "ServerDirectories": [
                {
                    "name": "arcgiscache",
                    "physicalPath": "C:\\arcgisserver\\directories\\arcgiscache",
                    "directoryType": "CACHE",
                    "cleanupMode": "NONE",
                    "maxFileAge": 0
                },
                {
                    "name": "arcgisjobs",
                    "physicalPath": "C:\\arcgisserver\\directories\\arcgisjobs",
                    "directoryType": "JOBS",
                    "cleanupMode": "TIME_ELAPSED_SINCE_LAST_MODIFIED",
                    "maxFileAge": 360
                },
                {
                    "name": "arcgisoutput",
                    "physicalPath": "C:\\arcgisserver\\directories\\arcgisoutput",
                    "directoryType": "OUTPUT",
                    "cleanupMode": "TIME_ELAPSED_SINCE_LAST_MODIFIED",
                    "maxFileAge": 10
                },
                {
                    "name": "arcgissystem",
                    "physicalPath": "C:\\arcgisserver\\directories\\arcgissystem",
                    "directoryType": "SYSTEM",
                    "cleanupMode": "NONE",
                    "maxFileAge": 0
                }
            ],
            "PrimarySiteAdmin": {
                "UserName": "siteadmin",
                "Password": "siteadmin"
            },
            "Extensions": {
                "MappingChartingSolution": {
                    "Installer": {
                        "Path": "C:\\data\\installation\\ArcGIS_Mapping_and_Charting_Solutions_Server_1091_180544.exe",
                        "InstallDir": "C:\\bin\\ArcGIS\\MappingChartingSolution"
                    },
                    "Features": [
                        "Maritime"
                    ]
                }
            }
        },
        "Portal": {
            "LicenseFilePath": "C:\\data\\installation\\ArcGIS_Enterprise_Portal_1091_367221_20211130.json",
            "PortalLicenseUserTypeId": "creatorUT",
            "Installer": {
                "Path": "C:\\data\\installation\\Portal_for_ArcGIS_Windows_1091_180052.exe",
                "WebStylesPath": "C:\\data\\installation\\Portal_for_ArcGIS_Web_Styles_Windows_1091_180053.exe",
                "InstallDir": "C:\\bin\\ArcGIS\\Portal",
                "ContentDir": "C:\\arcgisportal",
                "PatchesDir": "C:\\data\\installation\\arcgisPatches",
                "PatchInstallOrder": [
                    "ArcGIS-1091-PFA-Log4j-PatchB.msp",
                    "ArcGIS-1091-PFA-QCS-Patch.msp",
                    "ArcGIS-1091-PFA-SEC2022U1-Patch.msp",
                    "ArcGIS-1091-PFA-SEC2022U2-PatchB.msp",
                    "ArcGIS-1091-PFA-ESFD-Patch.msp"
                ]
            },
            "ContentDirectoryLocation": "C:\\arcgisportal\\content",
            "EnableAutomaticAccountCreation": false,
            "DefaultRoleForUser": "viewer",
            "DefaultUserLicenseTypeIdForUser": "viewerUT",
            "PortalAdministrator": {
                "UserName": "user",
                "Email": "mail",
                "Password": "password",
                "SecurityQuestionIndex": 1,
                "SecurityAnswer": "vanilla"
            },
            "EmailSettings": {
                "SMTPServerAddress": "host",
                "From": "mail",
                "Label": "Portal Admin",
                "AuthenticationRequired": true,
                "SMTPPort": "port",
                "UserName": "user",
                "Password": "password",
                "EncryptionMethod": "TLS"
            }
        },
        "Insights": {
            "Installer": {
                "Path": "C:\\data\\installation\\ArcGIS_Insights_Windows_2022_3_184093.exe"
            }
        }
    }
}

Expected Behavior

Insights for ArcGIS should be updated
my expectation would be, that I just need to specify the oldInsightsVersion, InsightsVersion and the Insights Block in the ConfigData section, since I just want to update Insights and do not need any changes on server and portal itself.

Actual Behavior

no installation task of insights is logged
version.txt in the installation folder of insights on the portal and server hosts, show the old version

Steps to Reproduce

execute the following command referencing the above mentioned configuration file.
Invoke-ArcGISConfiguration -ConfigurationParametersFile ./Deployment-upgrageInsights.json -Mode Upgrade -EnableMSILogging -DebugSwitch

Important Factoids

References

I tried the installation (Install) mode instead of upgrade, with the following configuration file:
{
    "AllNodes": [
        {
            "NodeName": "ServerHost",
            "Role": [
                "Server"
            ]
        },
        {
            "NodeName": "PortalHost",
            "Role": [
                "Portal"
            ]
        }
    ],
    "ConfigData": {
        "Version": "10.9.1",
        "ServerContext": "arcgis",
        "PortalContext": "portal4arcgis",
        "ServerRole": "GeneralPurposeServer",
        "InsightsVersion": "2022.3",
        "DownloadSetups": false,
        "DownloadPatches":false,
        "Credentials": {
            "ServiceAccount": {
                "UserName": "user",
                "Password": "password",
                "IsDomainAccount": true,
                "IsMSAAccount": false
            }
        },
        "Insights": {
            "Installer": {
                "Path": "C:\\data\\installation\\ArcGIS_Insights_Windows_2022_3_184093.exe"
            }
        }
    }
}

But I get the following errors:
Dot Sourcing the Configuration:- ArcGISInstall
PSDesiredStateConfiguration\Node : Es ist nicht möglich, eine Methode für einen Ausdruck aufzurufen, der den NULL hat.
--> TRANSLATION: it is not possible, to invoke a method on a null value expression.
In C:\Program Files\WindowsPowerShell\Modules\ArcGIS\4.0.2\Configurations-OnPrem\ArcGISInstall.ps1:27 Zeichen:5
+     Node $AllNodes.NodeName {
+     ~~~~
    + CategoryInfo          : InvalidOperation: (:) [PSDesiredStateConfiguration\node], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : InvokeMethodOnNull,PSDesiredStateConfiguration\node
WARNUNG: Es wird davon abgeraten, Domänenanmeldeinformationen für den Knoten 'portalHost' zu verwenden.
Kompilierungsfehler bei der Verarbeitung der Konfiguration "ArcGISInstall".
In C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\PSDesiredStateConfiguration\PSDesiredStateConfiguration.psm1:3917 Zeichen:5
+     throw $ErrorRecord
+     ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (ArcGISInstall:String) [], InvalidOperationException
    + FullyQualifiedErrorId : FailToProcessConfiguration

Hi @alexanderwalkowski,
When using -Mode Upgrade the json configuration file will need to include both ConfigData.OldVersion, and ConfigData.OldInsightsVersion attributes.
And when using -Mode Install the json configuration file will need to include ConfigData.Portal block and ConfigData.Server block.
For examle:
{
    "AllNodes": [
        {
            "NodeName": "host1",
            "Role": [
                "Server"
            ]
        },
        {
            "NodeName": "host2",
            "Role": [
                "Portal"
            ]
        }
    ],
    "ConfigData": {
        "Version": "10.9.1",
        "OldVersion": "10.9.1",
        "ServerContext": "arcgis",
        "PortalContext": "portal4arcgis",
        "ServerRole": "GeneralPurposeServer",
        "InsightsVersion": "2022.3",
        "OldInsightsVersion": "2022.1.1",
        "DownloadSetups": false,
        "DownloadPatches":false,
        "Credentials": {
            "ServiceAccount": {
                "UserName": "username",
                "Password": "password",
                "IsDomainAccount": true,
                "IsMSAAccount": false
            }
        },

        "Server": {
            "LicenseFilePath": "C:\\data\\installation\\ArcGISGISServerStandard_ArcGISServer_1111807.prvc",
            "Installer": {
                "Path": "C:\\data\\installation\\ArcGIS_Server_Windows_1091_180041.exe",
                "InstallDir": "C:\\bin\\ArcGIS\\Server",
                "InstallDirPython": "C:\\bin\\Python27",
                "EnableArcMapRuntime": true,
                "EnableDotnetSupport": true,
                "PatchesDir": "C:\\data\\installation\\arcgisPatches",
                "PatchInstallOrder": [
                    "ArcGIS-1091-S-SP-Patch.msp",
                    "ArcGIS-1091-S-MR-PatchB.msp",
                    "ArcGIS-1091-S-Log4j-PatchB.msp",
                    "ArcGIS-1091-S-SQ-Patch.msp",
                    "ArcGIS-1091-S-UNDM1-Patch.msp",
                    "ArcGIS-1091-S-VRPG-Patch.msp",
                    "MappingandChartingServer1091Update1.msp",
                    "ArcGIS-1091-S-MPS-Patch.msp",
                    "ArcGIS-1091-S-SEC2022U1-PatchB.msp",
                    "MappingandChartingServer1091Update2.msp",
                    "ArcGIS-1091-S-UNDM2-Patch.msp",
                    "ArcGIS-1091-S-SEC2022U2-Patch.msp",
                    "MappingandChartingServer1091Update3.msp",
                    "ArcGIS-1091-S-UNDM3-Patch.msp",
                    "ArcGIS-1091-S-GSQ-Patch.msp"
                ]
            },
            "ServerDirectoriesRootLocation": "C:\\arcgisserver\\directories",
            "ConfigStoreLocation": "C:\\arcgisserver\\config-store",
            "LocalRepositoryPath": "C:\\arcgisserver\\local",
            "ServerLogsLocation": "C:\\arcgisserver\\logs",
            "ServerDirectories": [
                {
                    "name": "arcgiscache",
                    "physicalPath": "C:\\arcgisserver\\directories\\arcgiscache",
                    "directoryType": "CACHE",
                    "cleanupMode": "NONE",
                    "maxFileAge": 0
                },
                {
                    "name": "arcgisjobs",
                    "physicalPath": "C:\\arcgisserver\\directories\\arcgisjobs",
                    "directoryType": "JOBS",
                    "cleanupMode": "TIME_ELAPSED_SINCE_LAST_MODIFIED",
                    "maxFileAge": 360
                },
                {
                    "name": "arcgisoutput",
                    "physicalPath": "C:\\arcgisserver\\directories\\arcgisoutput",
                    "directoryType": "OUTPUT",
                    "cleanupMode": "TIME_ELAPSED_SINCE_LAST_MODIFIED",
                    "maxFileAge": 10
                },
                {
                    "name": "arcgissystem",
                    "physicalPath": "C:\\arcgisserver\\directories\\arcgissystem",
                    "directoryType": "SYSTEM",
                    "cleanupMode": "NONE",
                    "maxFileAge": 0
                }
            ],
            "PrimarySiteAdmin": {
                "UserName": "siteadmin",
                "Password": "siteadmin"
            },
            "Extensions": {
                "MappingChartingSolution": {
                    "Installer": {
                        "Path": "C:\\data\\installation\\ArcGIS_Mapping_and_Charting_Solutions_Server_1091_180544.exe",
                        "InstallDir": "C:\\bin\\ArcGIS\\MappingChartingSolution"
                    },
                    "Features": [
                        "Maritime"
                    ]
                }
            }
        },
        "Portal": {
            "LicenseFilePath": "C:\\data\\installation\\ArcGIS_Enterprise_Portal_1091_367221_20211130.json",
            "PortalLicenseUserTypeId": "creatorUT",
            "Installer": {
                "Path": "C:\\data\\installation\\Portal_for_ArcGIS_Windows_1091_180052.exe",
                "WebStylesPath": "C:\\data\\installation\\Portal_for_ArcGIS_Web_Styles_Windows_1091_180053.exe",
                "InstallDir": "C:\\bin\\ArcGIS\\Portal",
                "ContentDir": "C:\\arcgisportal",
                "PatchesDir": "C:\\data\\nstallation\\arcgisPatches",
                "PatchInstallOrder": [
                    "ArcGIS-1091-PFA-Log4j-PatchB.msp",
                    "ArcGIS-1091-PFA-QCS-Patch.msp",
                    "ArcGIS-1091-PFA-SEC2022U1-Patch.msp",
                    "ArcGIS-1091-PFA-SEC2022U2-PatchB.msp",
                    "ArcGIS-1091-PFA-ESFD-Patch.msp"
                ]
            },
            "ContentDirectoryLocation": "C:\\arcgisportal\\content",
            "EnableAutomaticAccountCreation": false,
            "DefaultRoleForUser": "viewer",
            "DefaultUserLicenseTypeIdForUser": "viewerUT",
            "PortalAdministrator": {
                "UserName": "portaladmin",
                "Email": "mail.de",
                "Password": "password",
                "SecurityQuestionIndex": 1,
                "SecurityAnswer": "answer"
            },
            "EmailSettings": {
                "SMTPServerAddress": "mailserver",
                "From"<EMAIL_ADDRESS>                "Label": "Portal Admin",
                "AuthenticationRequired": true,
                "SMTPPort": "587",
                "UserName": "username",
                "Password": "password",
                "EncryptionMethod": "TLS"
            }
        },
        "DataStore": {
            "ContentDirectoryLocation": "C:\\arcgisdatastore",
            "EnableFailoverOnPrimaryStop": false,
            "EnablePointInTimeRecovery": false,
            "Installer": {
                "Path": "C:\\data\\installation\\ArcGIS_DataStore_Windows_1091_180054.exe",
                "InstallDir": "C:\\bin\\ArcGIS\\DataStore",
                "PatchesDir": "C:\\data\\installation\\arcgisPatches",
                "PatchInstallOrder": [
                    "ArcGIS-1091-DS-Log4j-Patch.msp",
                    "ArcGIS-1091-DS-DE-Patch.msp"
                ]
            }
        },
        "Insights": {
            "Installer": {
                "Path": "C:\\data\\installation\\ArcGIS_Insights_Windows_2022_3_184093.exe"
            }
        }
    }
}

Thanks,
Cameron K.

Hi @cameronkroeker,
thanks for clarifying the dependencies between the -Mode parameter and the required blocks in the configuration date. It might be helpfull to include this in the wiki.
With your clarification in mind, I double checked my configuration file, and the initial issue still appears. Just to make sure I understood you right, let me sumarize my setup:

Test 1
** mode upgrade
** configuration includes oldVersion=10.9.1 version=10.9.1, insightsVersion and oldInsightsVersion
** configuration includes server and portal block
** result
*** dsc script exits without failure, but no installation steps concerning insights are logged and the version.txt in the installation folder is not updated
Test 2
** mode upgrade
** configuration includes oldVersion=10.9.1 version=10.9.1, insightsVersion and oldInsightsVersion
** configuration does not include server and portal block
** result
*** tbd
Test 3
** mode install
** configuration includes only insightsVersion=2022.3
** configuration includes configData.server and configData.portal block
** result
*** tbd

Hi @alexanderwalkowski,
The -Mode Upgrade will first uninstall the previous version of Insights (ConfigData.OldInsightsVersion) and install the new version (ConfigData.InsightsVersion). This ideally should be used when upgrading both ArcGIS Enterprise and ArcGIS Insights. Though it should technically work when only upgrading ArcGIS Insights because the module is idempotent and will skip upgrading ArcGIS Enterprise.
If only upgrading ArcGIS Insights, then perhaps it makes more sense to use -Mode Install (as you did in Test 3). This will do an "in-place" upgrade of Insights. This is fine for newer versions, however I believe 10.7 and prior an "in-place" upgrade was not always supported.
https://doc.arcgis.com/en/insights/2022.1/get-started/install-insights.htm#ESRI_SECTION1_E87D9F78C1BD469DA9F75AF040F44B50
I suspect that Test 1 did not fully work because ConfigData.Version and ConfigData.OldVersion matched. I would be interested in seeing this test ran again but with ConfigData.OldVersion  set to something prior to 10.9.1, for example 10.8.1.
Your assessment of Test 2 is accurate. My apologies, I meant the ConfigData.Portal and ConfigData.Server block should contain the original attributes and values (usernames/passwords/install dir etc).
Thanks,
Cameron K.

Hi @cameronkroeker,
Sorry, but I am not able to rerun test 1, since the installation was successful in test 3.
In my opinion it would be helpful, to enhance the documentation

concerning the dependencies between the configData blocks and the mode
by an example how to update Insights without updating the enterprise version

kind regards,
Alexander

Hi @alexanderwalkowski,
No worries, I was able to verify Test 1. Test 1 didn't work because of 2 reasons, first the ConfigData.OldVersion attribute value needs to be lower than ConfigData.Version, and second the full Enterprise stack (Server\Portal\DataStore\WebAdaptor) needs to be specified in the Role sections for it to invoke the Insights upgrade.
So what we have learned from this is, when upgrading both ArcGIS Enterprise and Insights then it's best to use -Mode Upgrade, and when only upgrading ArcGIS Insights then its best to use -Mode Install. I also do agree with you that there should be documentation that provides these details.
Thanks!

A new resource has recently been published that goes into detail about how to use the ArcGIS module to upgrade ArcGIS Insights.
https://github.com/Esri/arcgis-powershell-dsc/wiki/Upgrade-ArcGIS-Insights


Updated to be able to run things locally in docker

Updated code base in #6 to use python watchdog for local running.  Updated the bash script to just run this Local_Dockerfile.
This assumes a local directory structure of /tmp/filing_bucket/upload to already exist.  I updated the docker compose filing env file to work with that structure and created a mount to it instead of the /usr/app directory it was under.
There's still the need to store validation results in the database, which expects the filing postgres database.  So this isn't a complete isolated from filing solution (@hkeeler and I talked briefly about that last week, and how eventually we'll want to have a validation solution that other people can pull in and just run, so eventually we'll want some configurable way to avoid having to store in the filing db, probably an env var?)
@lchen-2101 I have not modified your existing lambda code as part of this branch, I didn't want to muck with it until we settled on dir structure, etc.  But if you want I can.
So this is a continuation of #6.  Might want to just put that all in here, but it's a lot of changes.  If it's just easier to work through the code together please reach out.  I already demonstrated the code out in the dev cluster and somewhat of the structure with @hkeeler last week.  So @lchen-2101 just let me know if you want to set aside some time to go over it all.

got the gist from a quick glance, just starting to the chew on the other PR now :joy:, will reach out next week to see if my understanding is any different.

got the gist from a quick glance, just starting to the chew on the other PR now 😂, will reach out next week to see if my understanding is any different.
there was the one thing remained that I didn't get to which was to have the upload folder just be the user uploaded csv, all other files would be in a separate "folder", keeping that folder to be semantically correct of just being "upload"

Not sure I'm getting what you mean here.  The parquets are under separate folders (#_pqs and #_res).  The report.csv gets put at the same level as the initial csv, and the .done files I'm creating to trigger new sqs messages (though I'm looking into using eventbridge instead for that) are at the initial csv level.  But the parquets are separated.  Is that what you're referring to?

got the gist from a quick glance, just starting to the chew on the other PR now 😂, will reach out next week to see if my understanding is any different.
there was the one thing remained that I didn't get to which was to have the upload folder just be the user uploaded csv, all other files would be in a separate "folder", keeping that folder to be semantically correct of just being "upload"

Not sure I'm getting what you mean here. The parquets are under separate folders (#_pqs and #_res). The report.csv gets put at the same level as the initial csv, and the .done files I'm creating to trigger new sqs messages (though I'm looking into using eventbridge instead for that) are at the initial csv level. But the parquets are separated. Is that what you're referring to?

As in upload/year/lei/... for the original csv, (parquet|processing)/year/lei/... or something to that effect for the parquets, report/year/lei/... for the report.csv, so upload/year/.... folder only contains the uploaded csv file, keeping the root folder names semantically mean what they actually contain. This is just a general thing, not specific to what you are doing.


See another faces of cube

Hi, I am trying to use Swiper with "Cube" effect. My project only have two faces and when I go to the first slide, I can see the other face but I don't want see it.
How I can fix my problem?

I already know the problem, the other face is visible because backface-visibility does not work in IE9.


docs: cleanup contribution guide

Cleanup the contribution guide -- grammar, spelling, clarity, structure, etc.

Thanks that was really need it :)

I've also realized there aren't any instructions for setting up a development environment!
Running npm install and npm run dev:start results in many errors. I will tackle that in another MR.

Yes there is in the wiki section

We put it here because is easier to update

That makes perfect sense. I've added some references to help others find the wiki from the contribution doc.

I meant, I like much more in your PR than is is now in master :-)


Crash opening a valid epub

Hi,
I get a crash when opening a valid epub with
ERROR: Could not find table of contents resource. The book don't have a TOC resource."
But I have validated the file successfully here: http://validator.idpf.org
I can also see the toc file.

Please find attached a screenshot with the error and the toc file.
Thanks a lot

I'm in the middle of that. I'll send a pull request I guess when I'm done.
It seems to work.
I'll send you the file to your email

Disclaimer: I am closing GH Issues on this project, I don't work full time on it and I haven't been able to update that recently if you would like to help to maintain that and to push the development of this project forward, just reach me out or open a PR and we can discuss it there.


update Activiti/Activiti to master

UpdateBot pushed version changes from the source code in repository: Activiti/Activiti ref: master

UpdateBot commands:
updatebot push --ref master https://github.com/Activiti/Activiti.git


ISSUE#1.2.1- SANTIAGO - README DOCUMENTATION

Weekly Build Documentation Template - Issues [Insert Issue Numbers]
Repository: [Insert Repository Name with Link]

Week [Insert Week Number] - Documentation for Issues [Insert Issue Numbers]
Objective:
Document the progress for the following issues:

Issue #X.X: [Insert Issue Title (e.g., Feature Development)]
Issue #X.X: [Insert Issue Title (e.g., Testing and Validation)]
Issue #X.X: [Insert Issue Title (e.g., Pull Request Preparation)]

Issue #X.X: [Insert Issue Title (e.g., Feature Development)]
Week [Insert Week Number] Tasks:

[ ] Define the core components of the feature or functionality.
[ ] Implement the feature in the relevant language or framework.
[ ] Ensure that input/output specifications are correctly handled.

Progress Summary:

Feature Development:

The initial design is complete, and the basic implementation has been written.
Code is modularized to enhance readability and maintainability.

Challenges Encountered:

Handling edge cases required additional consideration, such as invalid input or specific data structure constraints.

Key Commit:

feat: implemented core functionality for [Insert Feature Name]

Next Steps:

Focus on refining edge cases and improving error handling or optimization.

Issue #X.X: [Insert Issue Title (e.g., Testing and Validation)]
Week [Insert Week Number] Tasks:

[ ] Develop comprehensive test cases (typical, boundary, and edge cases).
[ ] Use appropriate testing tools (e.g., JUnit, PyTest, etc.).
[ ] Debug the implementation using test-driven development.

Progress Summary:

Test Case Development:

Basic tests for typical cases have been written.
Boundary cases (e.g., small input, large input, etc.) are also covered.

Challenges Encountered:

Handling edge cases that cause the system or algorithm to fail unexpectedly, such as invalid inputs.

Key Commit:

test: added basic and edge case tests for [Insert Feature/Functionality Name]

Next Steps:

Conduct performance and stress testing to ensure the feature handles large datasets or complex input efficiently.

Issue #X.X: [Insert Issue Title (e.g., Pull Request Preparation)]
Week [Insert Week Number] Tasks:

[ ] Prepare the code for submission in a pull request.
[ ] Ensure the repository follows coding guidelines and best practices.
[ ] Create documentation for reviewers and link related issues.

Progress Summary:

Pull Request:

The pull request draft is ready, with all changes documented and code commented for clarity.
Linked all relevant issues for proper tracking.

Challenges Encountered:

Ensuring all tests pass before submission and integrating feedback from code reviews.

Key Commit:

chore: prepared pull request and linked relevant issues

Next Steps:

Finalize the pull request submission and ensure that the feature passes all required tests.

Commit Guidelines for This Week:
Each commit will adhere to the following format for consistency:

Commit Message Format:

feat: [description of feature]
fix: [description of bug fix]
test: [description of test]
refactor: [description of refactoring task]

Example Commit Messages for this Week:

feat: implemented core functionality for [Insert Feature]
fix: handled edge cases for [Insert Feature]
test: added test cases for [Insert Feature]

Next Steps and Upcoming Tasks:

Continue refining the feature based on peer feedback.
Finalize testing and documentation for the final pull request.
Conduct performance testing and review for optimizations.

Version Control Summary:
This section summarizes all commits made during the week to ensure proper tracking of changes.

Date
Changes Made
Commit ID

[Insert Date]
Implemented basic feature functionality
[Commit ID]

[Insert Date]
Added test cases for typical and edge cases
[Commit ID]

[Insert Date]
Refactored code and prepared pull request
[Commit ID]

Documentation Guidelines and References:
For more information on how to structure the repository, make effective commit messages, and ensure security, refer to:

How to Write READMEs for Repositories
How to Write Effective Commit Messages
How to Secure Repositories

Weekly Build Documentation Template - Issues #5564
Repository: binary-tree-dfs-bfs
Week [Insert Week Number] - Documentation for Issues #5564
Objective:
Document the progress for the following issue:
Issue #5564: Adding the Travelling Salesman Problem (BFS Algorithm in Java)
Week  Tasks:
Describe the essential elements of the capability or feature. #30 Could you put Traveling's BFS algorithm into practice?
.Salesman Problem in Java
Ensure that input/output specifications are correctly handled.
Progress Summary:
Feature Development:
The initial design for the Travelling Salesman Problem (TSP) using Java's Best-First Search (BFS) algorithm is complete.
The basic implementation is functional with a modular approach for readability and maintainability.
The distance matrix input handling for multiple cities is implemented, ensuring scalability.
Challenges Encountered:
We are handling cities with a large number of permutations that require optimization for performance.
Edge cases like circular paths and cities with zero distances added complexity in ensuring the correctness
Issue #5564: Testing and Validation
Week [Insert Week Number] Tasks:
Develop comprehensive test cases (typical, boundary, and edge cases) for the BFS algorithm.
Use JUnit to validate the Java implementation.
Debug the BFS implementation using test-driven development.
Progress Summary:
Test Case Development:
Basic tests for typical cases are written.
Boundary cases for small and large city inputs are covered.
Tests for circular paths and cities with equal distances are designed.
Challenges Encountered:
Ensuring correct BFS behavior when multiple cities have similar paths or costs.
Handling large city inputs where performance becomes critical.
Key Commit:
test: added basic and edge case tests for Travelling Salesman Problem using BFS
Next Steps:
Conduct performance testing to ensure the BFS algorithm handles large datasets efficiently.
Integrate additional test cases to validate edge scenarios (e.g., cities with zero distances).
Issue #5564: Pull Request Preparation
Week [Insert Week Number] Tasks:
Prepare the BFS implementation for submission in a pull request.
Ensure the repository follows Java coding guidelines and best practices.
Create documentation for reviewers and link related issues.
Progress Summary:
Pull Request:
The pull request draft is ready, with comments and clear documentation on the BFS approach for the Travelling Salesman Problem.
All relevant issues, including performance and testing, are linked for proper tracking.
Challenges Encountered:
Ensuring all JUnit tests pass before submission.
Handling feedback from code reviews, especially regarding optimization.
Key Commit:
chore: prepared to pull requests and linked relevant issues for Travelling Salesman Problem
Next Steps:
Finalize pull request submission after performance and testing are confirmed.
Ensure the feature passes all required tests before merging.
Memory updated
Here’s how you can fill in the Weekly Build Documentation Template for issue Adding the Travelling Salesman Problem #5564 using the BFS algorithm in Java:
Weekly Build Documentation Template - Issues #5564
Repository: [Insert Repository Name with Link]
Week [Insert Week Number] - Documentation for Issues #5564
Objective:
Document the progress for the following issue:
Issue #5564: Adding the Travelling Salesman Problem (BFS Algorithm in Java)
Week [Insert Week Number] Tasks:
Define the core components of the feature or functionality. #30
Implement the BFS algorithm for the Travelling Salesman Problem in Java.
Ensure that input/output specifications are correctly handled.
Progress Summary:
Feature Development:
The initial design for the Travelling Salesman Problem (TSP) using Java's Best-First Search (BFS) algorithm is complete.
The basic implementation is functional with a modular approach for readability and maintainability.
The distance matrix input handling for multiple cities is implemented, ensuring scalability.
Challenges Encountered:
We are handling cities with a large number of permutations that require optimization for performance.
Edge cases like circular paths and cities with zero distances added complexity in ensuring correctness.
Key Commit:
feat: implemented core functionality for Travelling Salesman Problem using BFS algorithm
Next Steps:
Focus on refining edge cases, particularly cities with multiple paths of equal distance.
Optimize performance for larger inputs and ensure scalability of the BFS approach.
Issue #5564: Testing and Validation
Week [Insert Week Number] Tasks:
Develop comprehensive test cases (typical, boundary, and edge cases) for the BFS algorithm.
Use JUnit to validate the Java implementation.
Debug the BFS implementation using test-driven development.
Progress Summary:
Test Case Development:
Basic tests for typical cases are written.
Boundary cases for small and large city inputs are covered.
Tests for circular paths and cities with equal distances are designed.
Challenges Encountered:
Ensuring correct BFS behavior when multiple cities have similar paths or costs.
Handling large city inputs where performance becomes critical.
Key Commit:
test: added basic and edge case tests for Travelling Salesman Problem using BFS
Next Steps:
Conduct performance testing to ensure the BFS algorithm handles large datasets efficiently.
Integrate additional test cases to validate edge scenarios (e.g., cities with zero distances).
Issue #5564: Pull Request Preparation
Week [Insert Week Number] Tasks:
Prepare the BFS implementation for submission in a pull request.
Ensure the repository follows Java coding guidelines and best practices.
Create documentation for reviewers and link related issues.
Progress Summary:
Pull Request:
The pull request draft is ready, with comments and clear documentation on the BFS approach for the Travelling Salesman Problem.
All relevant issues, including performance and testing, are linked for proper tracking.
Challenges Encountered:
Ensuring all JUnit tests pass before submission.
Handling feedback from code reviews, especially regarding optimization.
Key Commit:
chore: prepared to pull requests and linked relevant issues for Travelling Salesman Problem
Next Steps:
Finalize pull request submission after performance and testing are confirmed.
Ensure the feature passes all required tests before merging.
Commit Guidelines for This Week:
Each commit will adhere to the following format for consistency:
Commit Message Format:
feat: [description of feature]
fix: [description of bug fix]
test: [description of test]
refactor: [description of refactoring task]
Example Commit Messages for this Week:
feat: implemented core functionality for Travelling Salesman Problem using BFS
fix: handled edge cases for BFS implementation
test: added test cases for the Travelling Salesman Problem in the BFS algorithm
Next Steps and Upcoming Tasks:
Continue refining the BFS feature based on peer feedback.
Finalize performance testing and documentation.
Conduct stress testing for large datasets to ensure scalability.
.


Start a change log

Need to start a proper change log.
Doesn't need to be detailed. Just logs major changes between versions.
Easy to append a one-liner every time we change something structural.
Every API change that isn't a demo or library module should be logged in change log.

Closed by 7a19bcca644181023d621e6e1deb5997b5a70679


Installation failed due to timeout. Possibly due to unreachable api-server?

helm version: version.BuildInfo{Version:"v3.2.4", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"clean", GoVersion:"go1.13.12"}
kubectl version: Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:52:00Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:43:34Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Cloud Provider/Platform (AKS, GKE, Minikube etc.): Baremetal using Kubeadm
Steps taken:
`kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml -n monitoring
kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml -n monitoring
kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml -n monitoring
kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_prometheusrules.yaml -n monitoring
kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml -n monitoring
kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_thanosrulers.yaml -n monitoring`
Installation command
helm install prometheus-operator stable/prometheus-operator -n monitoring --set prometheusOperator.createCustomResource=false --atomic --wait --debug
The output of kubectl logs prometheus-operator-admission-create-fmqdj -n monitoring is:
{"err":"Get https://<IP_ADDRESS>:443/api/v1/namespaces/monitoring/secrets/prometheus-operator-admission: dial tcp <IP_ADDRESS>:443: i/o timeout","level":"fatal","msg":"error getting secret","source":"k8s/k8s.go:104","time":"2020-07-15T16:22:02Z"}
Installation logs from using --debug flag:
install.go:159: [debug] Original chart version: ""
install.go:176: [debug] CHART PATH: /home/worker17/.cache/helm/repository/prometheus-operator-8.15.14.tgz
client.go:108: [debug] creating 1 resource(s)
install.go:140: [debug] CRD alertmanagers.monitoring.coreos.com is already present. Skipping.
client.go:108: [debug] creating 1 resource(s)
install.go:140: [debug] CRD podmonitors.monitoring.coreos.com is already present. Skipping.
client.go:108: [debug] creating 1 resource(s)
install.go:140: [debug] CRD prometheuses.monitoring.coreos.com is already present. Skipping.
client.go:108: [debug] creating 1 resource(s)
install.go:140: [debug] CRD prometheusrules.monitoring.coreos.com is already present. Skipping.
client.go:108: [debug] creating 1 resource(s)
install.go:140: [debug] CRD servicemonitors.monitoring.coreos.com is already present. Skipping.
client.go:108: [debug] creating 1 resource(s)
install.go:140: [debug] CRD thanosrulers.monitoring.coreos.com is already present. Skipping.
client.go:258: [debug] Starting delete for "prometheus-operator-admission" PodSecurityPolicy
client.go:108: [debug] creating 1 resource(s)
client.go:258: [debug] Starting delete for "prometheus-operator-admission" ServiceAccount
client.go:287: [debug] serviceaccounts "prometheus-operator-admission" not found
client.go:108: [debug] creating 1 resource(s)
client.go:258: [debug] Starting delete for "prometheus-operator-admission" ClusterRole
client.go:108: [debug] creating 1 resource(s)
client.go:258: [debug] Starting delete for "prometheus-operator-admission" ClusterRoleBinding
client.go:108: [debug] creating 1 resource(s)
client.go:258: [debug] Starting delete for "prometheus-operator-admission" Role
client.go:287: [debug] roles.rbac.authorization.k8s.io "prometheus-operator-admission" not found
client.go:108: [debug] creating 1 resource(s)
client.go:258: [debug] Starting delete for "prometheus-operator-admission" RoleBinding
client.go:287: [debug] rolebindings.rbac.authorization.k8s.io "prometheus-operator-admission" not found
client.go:108: [debug] creating 1 resource(s)
client.go:258: [debug] Starting delete for "prometheus-operator-admission-create" Job
client.go:287: [debug] jobs.batch "prometheus-operator-admission-create" not found
client.go:108: [debug] creating 1 resource(s)
client.go:467: [debug] Watching for changes to Job prometheus-operator-admission-create with timeout of 5m0s
client.go:495: [debug] Add/Modify event for prometheus-operator-admission-create: ADDED
client.go:534: [debug] prometheus-operator-admission-create: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:495: [debug] Add/Modify event for prometheus-operator-admission-create: MODIFIED
client.go:534: [debug] prometheus-operator-admission-create: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
install.go:382: [debug] Install failed and atomic is set, uninstalling release
uninstall.go:92: [debug] uninstall: Deleting prometheus-operator

Hi @bhandari8sujit. As this issue relates to a very specific chart (prometheus-operator), please open a ticket against the prometheus-operator chart. This does not appear to be an issue on helm's side. Thank you.


Fix label for "Test Target" search paths in docs

I'm pretty sure this one should be "Test Targets" :)

Thanks!


Not working

Thanks for developing this!
I followed the steps and got "None" to show up in the compatability list. However, the game (CrossCode in this case) won't start. Steam tries and just spins forever, and has to be restarted to close the spinner. Steam Linux Runtime doesn't work either (unless started outside Steam, from the desktop executable).
When I switch back to Proton, the game runs as it did before (not smoothly). Anything I can try to get it going?

Huh interesting. If you switch between Proton and None, it should re-download the game. If it doesn't do that you could try manually uninstalling it

It does re-download the game automatically, but Steam cannot open it. It just shows the Steam loading spinner until I force close it.

I gave it another look and figured it out.

Set game compatibility mode to Linux Runtime.
In desktop mode, make sure the downloaded folder sits in /home/deck/.local/share/Steam/compatibilitytools.d/
If the Steam-Play-None folder doesn't exist inside of there (and instead just its contents), you did it wrong. The path goes /home/deck/.local/share/Steam/compatibilitytools.d/Steam-Play-None and then the folder contents. "None" will show up in the compatibility list if you did it wrong, which was my issue.
Go into Game mode. Switch compatibility mode to "None".
Play the game.


LaTeX

LaTeX could be implemented with a number of libraries. I'm partial to KaTeX, and it's what the original demo page uses, but there are certainly others.
Not sure how sophisticated peoples' LaTeX would get.
Leaving out the integration for now. If anything it'll be implemented as a separate file, like:
import { Latex, latex } from "mafs/latex"

<Latex expression={latex`\frac{1}{2}`} />

(the latex template tag would be a wrapper around String.raw to avoid JavaScript escaping backslashes)

I think that KaTeX is the best choice due to its high performance and compatibility with most LaTeX expressions.
const latex = (strings: TemplateStringsArray, ...values: any[]) => String.raw({ raw: strings }, ...values);

const a = 1;
const b = 2;

const expression = latex`\frac{${a}}{${b}}`;
// expression will be "\frac{1}{2}"

import { Latex } from "mafs/latex";

<Latex expression={latex`\frac{1}{2}`} />

I believe this approach is both simple and effective. I'm willing to try and implement the solution

I love Katex! Feel free to take it on. It will need a <foreignObject> I think.

I think it would also be nice if we could use the <Latex ... /> tag as a child of e.g. a <FuncionGraph ... /> and render it as a description or legend.
I would also be down to implement and help out!

@PhilippRitzberger what would this look like visually? Like the LaTeX would be attached to the graph in some way?


[6.4.x] BPMSPL-252: [GSS-RFE] Ability to check duplicate of ReleaseId (GroupId, ArtifactId and Version) with existing projects. Updates following QE feedback.

See https://issues.jboss.org/browse/BPMSPL-252.
(cherry picked from commit f48bd4965e709825ba727a8dce85238342a58931)
Corollary of https://github.com/droolsjbpm/guvnor/pull/252

I included https://github.com/droolsjbpm/guvnor/pull/257 in this PR as a separate commit.

+1
Just a note. I think we need to rename the BuildMessages to something else since we use them for everything, not just build status :)

Merged into 6.4.x


[Split saml.ts #1] Move getAdditionalParams out of saml.ts

Split of this PR https://github.com/node-saml/node-saml/pull/19/ - first commit

It's a bit late here, but I know what to do on this one, see https://github.com/node-saml/node-saml/pull/19#discussion_r759776914
I'll do that tomorrow.


Freecam doesn’t reset location on new hole

When using the freecam feature on a hole and then progressing to a new hole, the freecam takes the player back to their last location on the previous hole. This can be inconvenient if the player has progressed several holes and has to move the camera all the way back to the current hole.
Video of bug: https://github.com/sboxgame/minigolf/assets/15969024/fd0e4d04-d5f9-43e1-872f-70103a90688b

This is actually fixed (0ca43b09d416c3c853570df61291dc7467fd5556) in the master branch, but I do not believe the game has been uploaded to asset.party since.


franchise-client SyntaxError: missing ) after argument list

OSX
node v6.11.3
I'm having issues getting the franchise client bridge to work locally.
I've tried the following:
Benjamin@MacBook-Pro:~/projects/franchise$ npx franchise-client
missing ) after argument list

Benjamin@MacBook-Pro:~/projects/franchise/node_modules/franchise-client$ ./server.js
/Users/Benjamin/projects/franchise/node_modules/franchise-client/server.js:16
	ws.on('message', async message => {
	                 ^^^^^

SyntaxError: missing ) after argument list
    at createScript (vm.js:56:10)
    at Object.runInThisContext (vm.js:97:10)
    at Module._compile (module.js:542:28)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:389:7)
    at startup (bootstrap_node.js:149:9)

I've added all env variables required by credentials.js  to my env.
Any suggestions?
Thanks

+1

This is throwing an error because the version of node you're running doesn't support async functions. For now, you should install a version that does, eg node 8.
We're looking into relaxing this requirement.

The node version requirement should be slightly relaxed. Does npx franchise-client still throw that error?

Closing this for inactivity, but feel free to reply if it's still not working for you.


fix: Sender Icon

Description
The sender wallet website has been updated with a new icon. Accordingly, the wallet-selector should be updated at the same time.

Preview image:


Invariant Violation: addComponentAsRefTo

I try to use FixedDataTable from simple example at http://facebook.github.io/fixed-data-table/. For my component I get the following error:
Uncaught Error: Invariant Violation: addComponentAsRefTo(...): Only a ReactOwner can have refs. This usually means that you're trying to add a ref to a component that doesn't have an owner (that is, was not created inside of another component's render method). Try rendering this component inside of a new top-level component which will hold the ref.

Hey @hipertracker i assume you got this going, the repo seems to be working fine for me. Pretty sweet demo as well!


Build native sql for a dialect without to string

There has been a problem that knex hasn't been able to be used as a just query builder, except with SQL where all values are interpolated to one string, which has been passed as a string to driver.
This PR adds a way to get toSQL() result also in native SQL dialects format, where bindings are given separately and SQL string with dialects replacements are provided separately.
For example with postgresql dialalect query would be like:
{
  sql: `select * from "table" where id = $1`,
  bindings: [1]
}

Feature is implemented to the result of toSQL()as a getter function to prevent memory overhead of storing SQL strings and bindings always in two different format.
Here is also one related issue, where this was first requested https://github.com/tgriesser/knex/issues/1286

@wubzz any comments? To me its a bit hacky to add anonymous function to every toSQL() result, but it felt a better option than changing return type of toSQL() to some new class instance.

@elhigu I agree adding it a new class is overkill. A new function is sufficient enough. However for reading purposes I'd probably suggest something along the lines of:
  toSQL(method, tz) {
    this._undefinedInWhereClause = false;

    method = method || this.method
    let val = this[method]()

    const query = {
      method,
      options: reduce(this.options, assign, {}),
      timeout: this.timeout,
      cancelOnTimeout: this.cancelOnTimeout,
      sql: this[method](),
      bindings: this.formatter.bindings || [],
      __knexQueryUid: uuid.v4(),
      toNative: () => ({
       sql: this.client.positionBindings(query.sql),
       bindings: this.client.prepBindings(query.bindings),
     }),
    };

   if ((method === 'select' || method === 'first') && this.single.as) {
       query.as = this.single.as;
    }

    if(this._undefinedInWhereClause) {
      debugBindings(defaults.bindings)
      throw new Error(
        `Undefined binding(s) detected when compiling ` +
        `${method.toUpperCase()} query: ${query.sql}`
      );
    }

    return query;
}

But this suggestion isn't really because of your change, the function was already kind of strange to interpret, especially the part where it forces sql to become an object only to call assign for evidentally no reason at all?
Pardon the strange indentation..

@wubzz Thanks for comments. I agree that original way how that method was written is a bit odd. I'll try to do something to it before merging.

Looks like actually that assign value to query is necessary, since there are more attributes than just sql inside the value. I cleaned it up a bit anyways though.


added contributing guide

[x] Closes #640
[x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This is something we've talked about for a while and I'm capitalizing on a moment of inspiration.   Full disclosure, I've taken most of this from Pandas and edited it just where it makes sense for Xarray.
If others would like specific changes to this,  please comment only on doc/contributing.rst, ~CONTRIBUTING.md is auto generated with pandoc~. @pydata/xarray, feel free to push directly to this branch if there are larger edits you'd like to add.

Anymore comments here, if not, I'll merge at COB on Monday.

Thanks @shoyer for the review. I've addressed each of your comments in ffb44dd.

💯  @jhamman


Does migrate.sh need to include a call to psql?

Thanks so much for creating this example.  There is very little out there about Postgres security for Django
A question:
I can see that psql is called after manage.py migrate in migrate.sh.

Why can't we include the GRANT calls for djangouser in roles.sql?

Surely a call to psql is going to require a password file somewhere?

Ah.. So I understand that we cannot issue GRANT statements to tables if they don't exist yet, so the calls have to go after the migrate.py call.

I found you don't need to have calls in migrate.sh if you include the following
    ALTER DEFAULT PRIVILEGES IN SCHEMA <your_schema> GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO <your_user>;
    ALTER DEFAULT PRIVILEGES IN SCHEMA <your_schema> GRANT USAGE ON SEQUENCES TO <your_user>;

ALTER DEFAULT PRIVILEGES allows you to set the privileges that will be applied to objects created in the future
postgres docs

Here is my full init script in case it is of use:
#!/bin/bash
# initialises the dev db only
# https://github.com/sellonen/django-security-tips
# https://aws.amazon.com/blogs/database/managing-postgresql-users-and-roles/

set -e
SCHEMA=myschema
RW_ROLE=readwrite

echo "creating database"
createdb $DB_NAME

echo "setting default roles"
psql -v ON_ERROR_STOP=1 --dbname "$DB_NAME" <<-EOSQL
    -- lockdown permissions on public schema
    REVOKE CREATE ON SCHEMA public FROM PUBLIC;
    REVOKE ALL ON DATABASE $DB_NAME FROM PUBLIC;

    -- Migrator role
    CREATE USER $MIGRATOR_DB_USER WITH PASSWORD '$MIGRATOR_DB_PASS';
    CREATE SCHEMA $SCHEMA AUTHORIZATION $MIGRATOR_DB_USER;

    -- The user is allowed to create databases for the purposes of running the Django test suite.
    -- The user must have access to the 'public' schema because that is where new db tables for
    -- the test db will be created
    ALTER ROLE $MIGRATOR_DB_USER CREATEDB;
    ALTER ROLE $MIGRATOR_DB_USER SET SEARCH_PATH TO $SCHEMA, public;

    -- App User role
    CREATE USER $DB_USER WITH PASSWORD '$DB_PASS';
    ALTER ROLE $DB_USER SET SEARCH_PATH TO $SCHEMA;

    -- Read/write role
    CREATE ROLE $RW_ROLE;
    GRANT CONNECT ON DATABASE $DB_NAME TO $RW_ROLE;
    GRANT USAGE, CREATE ON SCHEMA $SCHEMA TO $RW_ROLE;
    GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA $SCHEMA TO $RW_ROLE;
    GRANT USAGE, SELECT, UPDATE ON ALL SEQUENCES IN SCHEMA $SCHEMA TO $RW_ROLE;

    -- Grant privileges to users
    GRANT $RW_ROLE TO $MIGRATOR_DB_USER;
    GRANT $RW_ROLE TO $DB_USER;

    -- Alter privileges so that the RW Role will be able to access tables created in future
    -- by the Migrator User
    ALTER DEFAULT PRIVILEGES FOR USER $MIGRATOR_DB_USER GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO $RW_ROLE;
    ALTER DEFAULT PRIVILEGES FOR USER $MIGRATOR_DB_USER GRANT USAGE, SELECT, UPDATE ON SEQUENCES TO $RW_ROLE;
EOSQL


Add 4.0.0-rc.1 release candidate 1 to be installable via nodenv

Pretty sure I copied the right format from the other install files, would be nice to be able to get this and the rest of the 4.0.0 pipeline into nodenv so our company can test it out. :+1:

closed by #63


⚠ dev-deces.matchid.io search API has degraded performance

In a286774, dev-deces.matchid.io search API (https://dev-deces.matchid.io/deces/api/v1/search?q=pompidou) experienced degraded performance:

HTTP code: 200
Response time: 4691 ms

Resolved: dev-deces.matchid.io search API performance has improved in c25d00c after 6 minutes.


Parsing generic Array's element doesn't work

The bug appears because of the mistake in the parseArrayElementType method in Composer.
It says: if name.hasSuffix("Array<") but it should be if name.hasPefix("Array<").

👍 good catch, mind adding that to the pr ?

@krzysztofzablocki I will open another one because I want to add a failing tests as well 😄


Teach client connection about "Sec-WebSocket-Protocol" header.

To simplify testing of ui.subprotocol, using just the uWebSockets library, teach the uWS::WebSocket<uWS::CLIENT> how to set an optional Sec-WebSocket-Protocol header.
To attempt to minimise API change this is done by adding an extra optional parameter to the end of the Hub::connect() function.

There are two problems: you shouldn't change anything in the Socket.h / Node.h files, and you have lots of styling issues like mixed spaces and tabs and too many spaces and such. Otherwise it looks fine

Tried to fix this up but the branching went wrong and force pushing made a mess.
Am redoing it in a new pr.


Implement the 6D Analytical Gradient

As of now, we make use of the expression of the analytical gradient in the context of a 4D problem (i.e., 3D for the location of the centroid, 1D for the rotation around the z-axis).
It might be beneficial to move toward a 6D problem formulation where we're required to express the complete analytical gradient by resorting e.g. to Vaskevicius2019^1.
Keeping the 4D and 6D analytical gradients separated and both available could be also advantageous.
year = {2016},
title = {{Revisiting Superquadric Fitting: A Numerically Stable Formulation}},
author = {Vaskevicius, Narunas and Birk, Andreas},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
issn = {0162-8828},
doi = {10.1109/tpami.2017.2779493},
pmid = {29990010},
pages = {220--233},
number = {1},
volume = {41},
keywords = {}
}

It would be interesting to test recent software for automatic differentiation, like CppAD that is also linked in the Ipopt website, to evaluate the gradient.


Handling ECONNRESET issue and resend the past notification

Hi,
In a day around 15-20 times, I get { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } error, due to which some of the notifications get missed. I want to retry those notification. I have seen a lot related issue earlier but none seems to give proper answer for retrying failed messages. I have the errorCallback in place, but it seems apn doesn't send the notification and device infor in errorCallBack, how can I retry it in case this happens ?
var apnsConnection = new apns.Connection(apnConfig);
apnConfig.errorCallback = function (err, msg) {
console.log("APN ERROR : ", err, " s: ", msg);
}
apnConfig have all certificate and key and production true variables. So its production code running fine, but in betweeen I get ECONNRESET errors, how to solve this issue ? I haven't touched retryLimit which is -1 by default and which means apn will try infinitely. So basically I want to track such errors and resend the notifications for which this error encountered first.

Have you configured a connectionTimeout value?

Its the default one which I didn't touch. Its connectTimeout: 10000, by default

When the connection timeout occurs do you see it connecting and disconnecting?

For the next message it reconnects, and message gets delivered.

As I said, occurrences is like avg 30per day in a week. But its directlt impacting production.

My bad about connecting, disconnecting question, here are the frequency of error occurrence, I see that errors are happening in batches and at a particular time, below are the time and logs:
= 28 Nov 2015 10:29:53.522 2015-11-28T04:59:46.396Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 10:29:53.616 2015-11-28T04:59:46.397Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 10:29:54.486 2015-11-28T04:59:47.351Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 10:29:54.608 2015-11-28T04:59:47.351Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 10:29:55.503 2015-11-28T04:59:48.341Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 10:29:55.562 2015-11-28T04:59:48.340Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 10:29:56.314 2015-11-28T04:59:49.274Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 10:29:56.351 2015-11-28T04:59:49.274Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 11:13:22.793 2015-11-28T05:43:15.625Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 11:13:22.829 2015-11-28T05:43:15.625Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 11:28:12.628 2015-11-28T05:58:05.327Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 11:28:12.682 2015-11-28T05:58:05.327Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 11:30:21.370 2015-11-28T06:00:14.066Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 11:30:21.462 2015-11-28T06:00:14.066Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 12:42:42.363 2015-11-28T07:12:35.075Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 12:42:42.556 2015-11-28T07:12:35.075Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 12:53:26.997 2015-11-28T07:23:19.514Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 12:53:27.024 2015-11-28T07:23:19.514Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 14:07:08.656 2015-11-28T08:37:01.147Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 14:07:08.681 2015-11-28T08:37:01.148Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 14:07:09.117 2015-11-28T08:37:01.673Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 14:07:09.119 2015-11-28T08:37:01.590Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 14:07:10.028 2015-11-28T08:37:02.472Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 14:07:10.291 2015-11-28T08:37:02.642Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 14:07:11.121 2015-11-28T08:37:03.589Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 14:07:11.179 2015-11-28T08:37:03.589Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 15:16:42.980 2015-11-28T09:46:35.310Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 15:16:42.984 2015-11-28T09:46:35.311Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 15:36:23.702 2015-11-28T10:06:16.090Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 15:36:23.704 2015-11-28T10:06:16.090Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 17:52:12.335 2015-11-28T12:22:04.262Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 17:52:12.371 2015-11-28T12:22:04.262Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 17:52:46.767 2015-11-28T12:22:38.653Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 17:52:46.791 2015-11-28T12:22:38.653Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 17:52:50.638 2015-11-28T12:22:42.662Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null
» 28 Nov 2015 17:52:50.817 2015-11-28T12:22:42.662Z info APN ERROR : { [Error: read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' } s: null

Just one question, is it possible that device which is inactive at APPLE server, and we try to send a notification to it, APPLE will send APN error 8 and after that breaks the connection ?

@mohit3081989 yes,  because APNS think this device is illegal, and APNS will break down this connect

@mohit3081989 that is exactly what will happen. You need to ensure that you are using the data from the feedback service to prune old tokens from your database.

@argon but it might happen that device for which notification failed is out of network, in that case I can't consider it for pruning out from database, don't you think so ?

Please read this: https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Appendixes/BinaryProviderAPI.html#//apple_ref/doc/uid/TP40008194-CH106-SW4

@argon I have already seen that, this document says that if the notification is not delivered it means that destination App doesn't exist or the token has expired. I have implemented this service and seen that notification for which it was failing has again been active after a day or 2. How would Apple eventually get to know that App doesn't exist, I understand it can confirm about expired token

If you have ensured that the token is being refreshed in the app after every launch and your server is processing feedback correctly, then you are doing all you can. Apple will take care of the rest.
—A

On 29 Jan 2016, at 05:34, Mohit Agrawal<EMAIL_ADDRESS>wrote:
@argon I have already seen that, this document says that if the notification is not delivered it means that destination App doesn't exist or the token has expired. I have implemented this service and seen that notification for which it was failing has again been active after a day or 2. How would Apple eventually get to know that App doesn't exist, I understand it can confirm about expired token
—
Reply to this email directly or view it on GitHub.


[SMALLFIX] Fix multipleFlush test

@calvinjia  PTAL? I've updated the test.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/14627/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/14631/
Test PASSed.


[QST] Decoding item-id in NextItemPredictionTask

❓ Questions & Help
02-end-to-end-session-based-with-Yoochoose-PyT describes how to produce recommendations for items.
At the bottom, the encoded item id's are given as predictions:
- Top-5 predictions for session `11394056`: 6510 || 7055 || 4136 || 4204 || 4541

As you noticed, we first got prediction results (logits) from the trained model head, and then by using a handy util function visualize_response we extracted top-k encoded item-ids from logits.

If I understand correctly, these item-ids are encoded. How do I get the original item-id back?
The encoding is done with nvtabular Categorify, similar to here: https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/end-to-end-session-based/01-ETL-with-NVTabular.ipynb
In my case the original item-id is a string.
I would like tritonserver to return the decoded item-id's, since I will make the request from a server which I hope will not need to know anything about the internals of the model.
Thank you!

@samhedin please refer to this ticket where I explain what triton returns, incase you have set start_index=1 in the Categorify op, and in case you dont.
currently we dont have an automatic way to return to decoded item ids from Triton.  We are working on a wrapper and a utility function for that. but you can convert encoded triton ids to original ids following up the discussion in the ticket. what you need is the unique_item_id.parquet file in the categories folder.  you can create a simple mapping utility function to do post-processing decoding.

Thanks for the quick response, using unique_item_id.parquet is easy enough!
One last question, in the linked ticket you say
if you used start_index =1 in the Categorify() op then your item_id encoding starts from 2 not 1. Because we reserve 0 for padding, 1 for OOV +nulls, and then the other item categories starts from 2. so in that case, you will get 2 + n_unique_items from your prediction results.

I used Categorify(start_index=1), meaning that I should get n_unique_items + 2.
My unique_item_id.parquet looks like this, it seems to lack 0 for padding.
0                                               <NA>         0
1                                            <item1>     4500
2                                             <item2>       1800
3                                             <item3>      1200
4                                             <item4>     1200
5                                              <item5>       700
6                                             <item6>       600

Which to me looks like n_unique_items + 1. If I produce some random recommendations on this tiny sample dataset, I get
5
1
7
2
3
5
1

Which again seems like n_unique_items + 1 (6 unique items, one OOV/null item).
Is the padding token not included?
And to be clear, to decode these, I would do something like (notice the -1 in df.iloc)
    df = pd.read_parquet("unique.item_id.parquet")
    for user, next_items in zip(users, top_preds):
        decoded = [df.iloc[e - 1] for e in next_items]
        print(f"User {user} recommendations: {decoded}")

Because the output from TIS contained a 7 index which is out of bounds.

I used Categorify(start_index=1), meaning that I should get n_unique_items + 2.
yes but not in the unique_item_id.parquet file..  you get n_unique_items +1 in the unique parquet file.  you will get +2 in your results coming from Triton server response.

so you only need to shift the index of the parquet file +1 . if you used start_index=1 , just  add +1 to the index column.
                                              item_id    item_id_size

1                                                        0
2                                                4500
3                                                   1800
4                                                  1200
5                                                  1200
6                                                     700
7                                                    600
So  you can do like that, get the scores from Triton. Note that first score belong to padding, second belong to OOV, third belong to encoded item id 2, fourth score belong to encoded item id 3, so on so forth....
so you can first sort the scores by preserving/storing their indices as well,  and then do the reverse mapping using the  logic above that I mentioned.

Thank you so much!

Hi @samhedin
I've written extra step in t4r ensemble which returns both items and scores:
item_model.py file for Triton. This file is copied by next function.
import logging
import pathlib

import cudf
import triton_python_backend_utils as pb_utils

LOG = logging.getLogger("nvtabular")

class TritonPythonModel:
    """
    Triton model used for running NVT Workflows
    """

    def initialize(self, args):
        # Arg parsing
        repository_path = pathlib.Path(args["model_repository"])

        # Handle bug in Tritonserver 22.06
        # model_repository argument became path to model.py
        if str(repository_path).endswith(".py"):
            repository_path = repository_path.parent.parent

        items_path = (
            repository_path
            / str(args["model_version"])
            / "workflow/categories/unique.item.parquet"
        )
        model_device = args["model_instance_kind"]

        # Workflow instantiation
        self.items = cudf.read_parquet(items_path).item.to_numpy().reshape(1, -1)
        self.items = pb_utils.Tensor("items", self.items)

    def execute(self, requests):
        """Transforms the input batches by running through a NVTabular workflow.transform
        function.
        """
        responses = []
        for request in requests:
            probas = pb_utils.get_input_tensor_by_name(request, "proba")

            result = [probas, self.items]

            responses.append(pb_utils.InferenceResponse(result))

        return responses

It ads new model t4r_item and changes generated ensemble model. Maybe you need to fix some paths.
import os
import pathlib
import shutil

import tritonclient.grpc
import tritonclient.grpc.model_config_pb2 as model_config
from google.protobuf import text_format

def prepare_item_model(model_path):
    workdir = os.environ.get("WORKDIR", "/opt/recs")

    item_model_path = f"{workdir}/train/models/t4r/src/item_model.py"

    t4r_path = f"{model_path}/t4r"
    t4r_nvt_path = f"{t4r_path}_nvt"
    t4r_item_path = f"{t4r_path}_item"
    categories_path = "1/workflow/categories/"
    t4r_categories_path = f"{t4r_item_path}/{categories_path}"
    item_parquet_path = f"{categories_path}/unique.item.parquet"
    ensemble_schema_path = f"{t4r_path}/config.pbtxt"
    ensemble_schema_path_bak = f"{ensemble_schema_path}.bak"

    schema = model_config.ModelConfig()

    save_ensemble_backup = True

    if os.path.exists(ensemble_schema_path_bak):
        ensemble_schema_read_path = ensemble_schema_path_bak
        save_ensemble_backup = False
    else:
        ensemble_schema_read_path = ensemble_schema_path

    with open(ensemble_schema_read_path, "r") as f:
        graph_str = f.read()
        ensemble_schema = text_format.Parse(graph_str, schema)

    output = ensemble_schema.output[0]

    # construct t4r_item config
    t4r_item = model_config.ModelConfig(name="t4r_item", backend="python")
    t4r_item.input.append(
        model_config.ModelInput(
            name="proba", data_type=output.data_type, dims=output.dims
        )
    )
    t4r_item.output.append(
        model_config.ModelOutput(
            name="items", data_type=model_config.TYPE_STRING, dims=output.dims
        )
    )
    t4r_item.output.append(
        model_config.ModelOutput(
            name="proba", data_type=output.data_type, dims=output.dims
        )
    )

    pathlib.Path(t4r_categories_path).mkdir(parents=True, exist_ok=True)
    shutil.copy2(f"{t4r_nvt_path}/{item_parquet_path}", t4r_categories_path)
    shutil.copy2(item_model_path, f"{t4r_item_path}/1/model.py")

    with open(f"{t4r_item_path}/config.pbtxt", "w", encoding="utf8") as o:
        text_format.PrintMessage(t4r_item, o)

    ensemble_schema.output.clear()
    ensemble_schema.output.MergeFrom(t4r_item.output)
    item_step = model_config.ModelEnsembling.Step(
        model_name="t4r_item", model_version=-1
    )
    item_step.input_map["proba"] = "output"
    item_step.output_map["proba"] = "proba"
    item_step.output_map["items"] = "items"

    ensemble_schema.ensemble_scheduling.step.append(item_step)

    if save_ensemble_backup:
        shutil.move(ensemble_schema_path, ensemble_schema_path_bak)

    with open(ensemble_schema_path, "w", encoding="utf8") as o:
        text_format.PrintMessage(ensemble_schema, o)


Move front/body/back editors into components.

WIP - don't merge!

@oliver---- I created components for Article, Front, Body, Back and instantiate them using the registry. I would approach the author by overriding certain components. E.g. Front would do some cherry picking instead of rendering the whole front container as done in publisher mode. ok with that approach?
We enter some competing scenarios with our configurator api's. I'd like to start addressing this as well, so we can have author+reader+publisher packaged in one app. See https://github.com/substance/substance/issues/734

Fine with me. I can't say yet which is the ideal way. So let's go ahead and see where we get.


New version: OpenTofu.Tofu.Alpha version 1.6.0-alpha4

Pull request has been created with Komac v1.11.0 :rocket:
Microsoft Reviewers: Open in CodeFlow

@wingetbot waivers Add Validation-Executable-Error


Modules: Remove 'default_modules' folder logic

Brief description
This removes the old logic used to load modules from folder default_modules. With the recent move of all modules that logic has been replaced and the default folder does not exist thus this didn't do anything by default.
It also avoids the warning on startup:
*** WRN: >>> { ModulesLoader }: [  Could not find path when loading OpenPype modules "\openpype\modules\default_modules"  ]

Testing notes:

Test whether everything for modules still works.

I would not remove this logic as few features in development are still using the ability of that folder. If the warning is only reason of this PR then there are better solutions for that.

Thanks. Not sure what modules in development you are talking about. An open PR? Shotgun/Kitsu modules?
Anyway, I see now I never ended up posting my edited comment with the warning that if any studio relied on putting things into the default modules folder they'd also be affected.
How about instead of removing the logic to log a deprecation warning if any modules are found in that folder?

Deprecation warning for default_modules is good. This means to do the existing path check inside get_default_modules_dir and do not return invalid path.
I was also thinking about adding 1 other default path ~/modules/addons to have ability develop addons faster. As starting point it is good to have some kind of folder inside openpype that is loaded without changing settings even if is not existing in git.

Closing this in favor of recent changes on how modules are loaded since this code is outdated.


#2: update npm metadata to point to this repo

I've changed all links from kirontoo/insomnia-plugin-embark-theme to embark-theme/insomnia.
I've published the package and you can see the changes reflected at: https://www.npmjs.com/package/insomnia-plugin-embark-theme
reference issue: #2

Awesome thanks for taking care of that.


Closing a window does not remove it from the scratchpad

See title. In fact, I don't think it is ever possible to remove it again afterwards.
I also noticed that windows on the scratchpad still accept input, is that intended?

I also noticed that windows on the scratchpad still accept input, is that intended?

This is intended. The other issue has been fixed in the rewrite.


What's the best way to unsubscribe from topics?

In Kaffe, I've implemented brod_group_member.
I now want to be able to unsubscribe from topics dynamically (while apps are running).
I notice that brod_group_subscriber (here) does brod:unsubscribe/2.
Is that what I should be doing?
Thanks!

Hi @objectuser
There is no need to explicitly call the unsubscribe API if the member is to be terminated.
Otherwise yes, call brod:unsbcribe/2

Thank you!


New Twitter (X) not working

Since some weeks SW-DLT isn't working with the new Twitter (X).
I tried to reset the libraries and depencies, but has no effect.
Here a sample url: https://x.com/mccrushi/status/1696876940550971448?s=46&t=ivbqimqKkpGyL345eQ2_QA

yt-dlp has been updated and can now download videos from new Twitter (X).

look you're right. I tested it and now it works. So i close this ticket and hope it will continue to work ;)


Add CJS file to /fn entry

Add CJS file to /fn entry, which is especially useful in universal webpack apps, make it consistent with the main entry
Change files pattern in package.json, some dist files are missing (min/global), see<EMAIL_ADDRESS>

@ambar Thanks! I merged this in with some related fixes in #349.


CIP-0017 | Adjust preamble and structure w.r.t CIP-0001

Fixes #661.
(edited proposal)

@KtorZ when we were originally reviewing this it seemed like this was on the threshold of implementation, especially since Adafolio was already half way there, and maybe that's why it was merged as Active without the evidence that we would demand today.
Based on our current standards I've demoted it to Proposed but if can think of anything that qualifies it as Active then let's have another look.  I was also liberal about writing your Path to Active based on what we might expect along those lines & would be open to your suggestions or corrections.

Still @KtorZ I think it'll be helpful as a point of reference since the idea of the JSON portfolio has come up in other arrangements (e.g  #628) & will keep coming up.  I've changed the status... does anything else need to be reconsidered?


Delete unnecessary spaces

When copying from the web page, there is an unnecessary space after the code.

@changeworld : Thanks for your contribution! The author, @gregvanl, has been notified to review your proposed change.

@changeworld Thank you for this cleanup work! I'm going to merge this now.


Pubsub ordering keys subscriber

Add support for ordered delivery in subscribers (on ordering-keys branch)

LGTM - will approve after builds pass.

KMS build failure can be ignored; @kamalaboulhosn - we added a new IT presubmit check for KMS, next time you'll need to update the ordering keys branch to pick up the new kokoro config. Thanks!


Prerequisites for Teams cloud meeting recording

For the prerequisites part, please check the "!important" part.

It is saying that:
"Users won't need a Microsoft Stream license assigned if you want users to only record and download the recordings. This will mean that the recordings aren't stored in Microsoft Stream but are instead stored in Azure Media Services (AMS) with a 21-day limit before it's deleted."
We were doing recording successfully without enabling Stream license in the past. However, currently we cannot record it. We opened a case and the tech support said it is a must to enable Stream license for the Teams meeting recording.
If this is true, would you please modify the document to avoid customer's misunderstanding?
In addition, there's another sentence in the same article, which says:

“Currently, the meeting recording feature is turned off for customers whose Teams data is stored in-country if Microsoft Stream isn't available in the in-country data residency region where the data is stored.”
The support engineer said that it means we have to enable Stream license so that we can record Teams meeting. But I really don't think it is saying the same thing. This statement is confusing, would you please check it and make it more clear?
Thank you so much.

@Gary620  Thank you for submitting feedback and contributing to the docs. We are currently investigating this.

@scanum  Thank you Manuela, may I know the current status now?

@scanum Good day. Would you please let me know if there's any update? it's been more than a week. Thank you

Is there anybody who can help on this? It's been 2 weeks.
Thank you.

Hi @Gary620, thank you for your feedback and help us to improve docs.microsoft.com.
@tonysmit @garycentric could you kindly help to confirm this information? If you don't have a Stream license, where are the recordings stored?
Besides, about the second paragraph, I understand that recording is turned off if the Stream data is not available in the same region where the Teams data is stored:
https://docs.microsoft.com/stream/faq#which-regions-does-microsoft-stream-host-my-data-in
What do you think?
Thank you!

@dariomws Thank you Dario.
I agreed about the explanation about the second part.
For the first part, would you please let me know when it will be modified?
Thank you

@tonysmit @garycentric could you kindly help to confirm this information? If you don't have a Stream license, where are the recordings stored?

The recording are stored in AMS like it says. Adding @SamChung-MS to clarify.

I believe I responded in a different thread. Stream is not required. If recording isn't working for you, then it's most likely disabled by your admin.

@Gary620  Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from  to before v4.19.214",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=7623b1035ca2d17bde0f6a086ad6844a34648df1"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "7623b1035ca2d17bde0f6a086ad6844a34648df1",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "ocfs2: mount fails with buffer overflow in strlen\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v4.19.214 by commit 7623b1035ca2d17bde0f6a086ad6844a34648df1. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1001981


Secure Iframe Updates

Add allow-same-origin and allow-forms to sandbox scope
Support onLoad() prop
Support iframe resizer onResized() and onScroll() callbacks

From the research I've done, allow-same-origin and allow-forms should be safe: https://web.dev/sandboxed-iframes/
These two scopes are pretty critical for most iframes to function. This allows the iframe to access cookies on its own origin and submit forms.
For example, any website using Google Analytics will fail to render inside <iframe> due to the Google SDK choking on not being able to read the origin's own cookies. It seems reasonable to allow origins to read their own cookies and submit forms.
Let me know if you have any concerns. I might be overlooking something critical. Without these changes, most use cases for iframes won't work in the VM.

The main reason why it's restricted is it was a bonus to not have google analytics with tracking enabled. So not having cookies was a benefit

I'm going to close this since we can implement our requirements outside of BOS.


ARROW-6458: [Java] Remove value boxing/unboxing for ApproxEqualsVisitor

As discussed in https://github.com/apache/arrow/pull/5195#issuecomment-526157961, there are some problems with the current ways of comparing floating point vectors, we solve them in this PR:

there are if statements/duplicated members in ApproxEqualsVisitor, making the code redundant and less clear.
the comparion of float4 and float8 are based on wrapped objects Float and Double, which may have performance penalty.

@liyafan82 I think this change should probably be broken into two different changes. One that avoid boxing/unboxing and a second one that consolidates code. The code consolidation has the potential to actually hurt performance. As part of the boxing/unboxing we should also introduce a performance test to verify it actually improves performance (which can then be used to measure the impact of the code consolidation behind the interface).

It is a good suggestion to split into two issues for the two problems. I will revise the code and provide the performance benchmark accordingly.

@emkornfield I have split the issue into two, according to your suggestion.
This is the first part, to remove boxing/unboxing of floating point values.
Performance benchmark show that there is a performance improvement of over 10%:
Before:
Float8Benchmarks.approxEqualsBenchmark  avgt    5  7.161 ± 0.026  us/op
After
Float8Benchmarks.approxEqualsBenchmark  avgt    5  6.355 ± 0.004  us/op

Codecov Report

Merging #5304 into master will increase coverage by 0.98%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5304      +/-   ##
==========================================
+ Coverage   88.59%   89.57%   +0.98%
==========================================
  Files         950      702     -248
  Lines      126185   107156   -19029
  Branches     1495        0    -1495
==========================================
- Hits       111791    95990   -15801
+ Misses      14029    11166    -2863
+ Partials      365        0     -365

Impacted Files
Coverage Δ

cpp/src/arrow/filesystem/s3_internal.h
90.74% <0%> (-3.71%)
:arrow_down:

cpp/src/arrow/json/converter.cc
90.05% <0%> (-1.76%)
:arrow_down:

cpp/src/arrow/json/chunked_builder.cc
79.91% <0%> (-1.68%)
:arrow_down:

cpp/src/plasma/thirdparty/ae/ae.c
70.75% <0%> (-0.95%)
:arrow_down:

python/pyarrow/tests/test_parquet.py
96.09% <0%> (-0.12%)
:arrow_down:

r/src/recordbatch.cpp

go/arrow/math/uint64_amd64.go

r/R/list.R

r/src/symbols.cpp

r/src/array_to_vector.cpp

... and 244 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update abc7860...907c17d. Read the comment docs.

+1, thank you.


Add a videos section to the list

Also added my user group on there since we have recorded sessions.

Thank you!


Stutter while scrolling on Firefox Browser

Description
While using jellyseerr on the web with Firefox, scrolling and moving between UI elements is very slow as if its "too heavy"
Using Webkit based browsers (ie: Chromium / Safari) is working well.
Version
1.1.1
Steps to Reproduce

Open Firefox
Go to jellyseerr web UI (I'm using docker to run the app)
You should see the problem right away - scroll down in the main page

Screenshots
No response
Logs
No response
Platform
desktop
Device
Macbook pro
Operating System
MacOS 13
Browser
Firefox
Additional Context
No response
Code of Conduct

[X] I agree to follow Overseerr's Code of Conduct

i never even noticed it until now, but in my testing, scrolling does seem noticeably less snappy in firefox than chrome.

Ah. So same issue as overseerr too. I just noticed (lmfao I just thought it was bc my server was potato).
I recently read two comments on overseerr discord as well regarding similar issue

Does anyone use Firefox with overseerr? It's really slow for me
using a chrome based browser on the same machine for the same site and the slowness goes away. It's mostly with scrolling

Another feedback was:

if some requests a lot of requests it generally slows down

Worth looking into. Do you have a lot of requests as well?

Ah. So same issue as overseerr too. I just noticed (lmfao I just thought it was bc my server was potato).
I recently read two comments on overseerr discord as well regarding similar issue

Does anyone use Firefox with overseerr? It's really slow for me
using a chrome based browser on the same machine for the same site and the slowness goes away. It's mostly with scrolling

Another feedback was:

if some requests a lot of requests it generally slows down

Worth looking into. Do you have a lot of requests as well?

Interesting, not much ATM. Only 4 and they are already "approved".
Quite sure it's because of all the other pictures as well. (not exclusive to the request section.)
A screenshot is attached below.

@realitymolder Another thing from Overseerr discord by an overseerr dev. Turns out its firefox v103 issue as v102 does not have that stutter

Try with v102
https://ftp.mozilla.org/pub/firefox/releases/102.0/

I j

@realitymolder Another thing from Overseerr discord by an overseerr dev. Turns out its firefox v103 issue as v102.1.0esr does not have that stutter

Try with v102
https://ftp.mozilla.org/pub/firefox/releases/102.0/

and it was replied with

using the 102.1.0esr version fixes the scrolling issue

so maybe try on https://ftp.mozilla.org/pub/firefox/releases/102.1.0esr/

I just tested on v102.1.0esr and it indeed does not stutter. (I am on that now)

using the 102.1.0esr version fixes the scrolling issue

so maybe try on https://ftp.mozilla.org/pub/firefox/releases/102.1.0esr/

I just tested on v102.1.0esr and it indeed does not stutter. (I am on that now)

How can we let Firefox know about this bug? Is that in a specific package that we can put a finger on ?

How can we let Firefox know about this bug? Is that in a specific package that we can put a finger on ?

I am not sure. There should be a way to report bugs, maybe check their website?
As for the specific package, all I know (and can confirm) is that that v102.1.0esr has no issue however the latest one as in the 103 does have the issue present.

I Opened an issue in Bugzilla, lets see what happens now haha

I've opened an issue in Bugzilla, let's see what happens now haha

I heard that the issue has been fixed in Firefox nightlies so hopefully very soon it will be fixed for stable 🥳

I just tested this with the latest firefox 104.0.2 and it seems to have been fixed c:

I just tested this with the latest firefox 104.0.2 and it seems to have been fixed c:

that was a very fast fix tbh


Improve JSX parsing

I am using WP-CLI 2.2.0 and just ran wp i18n make-pot against https://github.com/google/site-kit-wp with the debug flag on.
For many of the JS files there were Unterminated string errors. To me that sounds like the JSX parsing in Peast is not bulletproof just yet.
I'll try to see if I can come up with some reproducible test cases.

@swissspidy - curious why closed

I've never managed to follow-up with some reproducible test cases showing where the parsing didn't work, so this ticket was never really actionable. Plus, there's no point in having a ticket here for things that needs to be fixed upstream in the Peast library.


Use context manager for reading Excel file

Please confirm that this PR has done the following:

[ ] Tests Added
[ ] Documentation Added
[ ] Name of contributors Added to AUTHORS.rst
[x] Description in RELEASE_NOTES.md Added

Description of PR
Closes #817.
Use a context manager to ensure that an Excel file is closed after reading.

Pinned ixmp4==0.6.0 after a test broke when using ixmp4 version 0.7.0.


sparklyr gives different results from Hive on simple query

I have a very simple query to grab two fields from a data 1) date 2) ID
q <- 'select startdate, calls from call_db where BizSeg = \"SM\" and startdate = \"2018-10-14"\ limit 100'
res1 <- sdf_sql(sc, q) %>% collect()

head(res1)

# A tibble: 6 x 2
  startdate  calls
  <date>     <chr>
1 2018-10-13 A8565000-1135-B055-6532-8C9EB0191CDE
2 2018-10-13 B5565000-CB1C-FF6F-0C19-9A5CE4C97243
3 2018-10-13 A8565000-1135-631E-6D33-E8D36B82F9EE
4 2018-10-13 B5565000-CB1C-7886-A760-C7BB17F22CFC
5 2018-10-13 8F565000-853E-9970-E08F-626D16EFADC1
6 2018-10-13 A8565000-1135-DEE6-3ADA-60381F85CE8F

res2 <- dbGetQuery(sc, q)
head(res2)

   startdate                                calls
1 2018-10-13 A8565000-1135-B055-6532-8C9EB0191CDE
2 2018-10-13 B5565000-CB1C-FF6F-0C19-9A5CE4C97243
3 2018-10-13 A8565000-1135-631E-6D33-E8D36B82F9EE
4 2018-10-13 B5565000-CB1C-7886-A760-C7BB17F22CFC
5 2018-10-13 8F565000-853E-9970-E08F-626D16EFADC1
6 2018-10-13 A8565000-1135-DEE6-3ADA-60381F85CE8F

And here is the result from Hive CLI

I have no idea how date got shifted by one day.
When I ran the exact same query with PySpark I got the correct results that match the Hive CLI.
Appreciate any help to decipher this.

Could you please provide a reprex?

I just created a Hive table with the following info and redo this query. SparkR also gives the correct answer but I do not understand why sparklyr and dbGetQuery fails to do so

It's probably the case that HIVE and sparkR collect the data field as character not date, you can match by casting the date column as character with cast(startdate as string), you can also set the folowing option to force sparklyr to collect as character:
options(sparklyr.collect.datechars = TRUE)

Usually this problem triggers when the timezone of the server is different that the timezone of R. So it might actually be correct to retrieve a day before if you are in different timezones.

Thanks @javierluraschi
I have tried both methods you suggested, but the problem still exists.
Here are my attempts to match TZ :
.rs.restartR()

Sys.timezone()
[1] "America/Chicago"

Sys.setenv(TZ = 'America/Chicago')
Sys.getenv('TZ')
[1] "America/Chicago"
system('echo $TZ')
[1] "America/Chicago"

And retired with this
options(sparklyr.collect.datechars = TRUE)

q <- 'select startdate, calls from call_db where BizSeg = "SM" and cast(startdate as string) = "2018-10-14" limit 10'

sdf_sql(sc,qq) %>% collect()

# A tibble: 10 x 2
   startdate  calls
   <date>     <chr>
 1 2018-10-13 A8565000-1135-B055-6532-8C9EB0191CDE
 2 2018-10-13 B5565000-CB1C-FF6F-0C19-9A5CE4C97243
 3 2018-10-13 A8565000-1135-631E-6D33-E8D36B82F9EE
 4 2018-10-13 B5565000-CB1C-7886-A760-C7BB17F22CFC
 5 2018-10-13 8F565000-853E-9970-E08F-626D16EFADC1
 6 2018-10-13 A8565000-1135-DEE6-3ADA-60381F85CE8F
 7 2018-10-13 A8565000-1135-5146-7F95-6E033B74DF61
 8 2018-10-13 A9565000-F677-DD52-6EDB-51DD5BAA5A27
 9 2018-10-13 B5565000-CB1C-5F1C-B280-9C4ABAA6A913
10 2018-10-13 A9565000-F677-FBC5-B6ED-91B4628C4961


Update minishift install instructions with template callouts

Provide a better reference to template instead of just saying here.
Consider making a better call out to clearing key cloak fields.

Fixed by cbf553e7db57d580048969478475d38084c547c5 and merged