field_of_movement algorithm is broken

Describe the bug
In this particular video below, you'll see me moving between two white tiles that have the exact same cost to reach another white tile further below. However, one of the white tiles I hover is showing that it's within my field of movement, while the other one does not. The budget in this case was 5.
To Reproduce
https://github.com/ManevilleF/hexx/blob/main/examples/field_of_movement.rs
Use the above example, put budget to a lower amount and look around your map to find that things don't add up
Expected behavior
To be able see a field of movement that corresponds to the set budget
Screenshots Or Code snippet
https://github.com/ManevilleF/hexx/assets/17499594/e5326e70-47b2-462b-b65e-b93c11e1d0ab

Not sure what's wrong with it or what algorithm it is based on. I'll try to come up with a fix but I don't understand how it works yet.

@orph3usLyre this might interest you

I came up with a working solution using the Dijkstra algorithm. I can try to implement it in a similar coding style as the current state of field_of_movement, however, it may take a little while as we're heading into Christmas festivities.
Merry Christmas!

I think this might be related to #128

Looking into this, and yeah the algorithm is definitely broken, an actual pathfinding algorithm should be used instead

Hi again,
I lack the time & probably the skill currently to convert my solution to something that looks more Rust idiomatic. But I'll post my solution below and hopefully that can help some;
fn field_of_movement(
    start: Hex,
    budget: u32,
    cost: impl Fn(Hex) -> Option<u32>,
) -> HashMap<Hex, (Hex, u32)> {
    let mut priority_queue = BinaryHeap::new();
    let mut cumulative_costs = HashMap::new();

    priority_queue.push(((start.x, start.y), 0));

    while let Some((current_hex, current_cost)) = priority_queue.pop() {
        let hex = Hex {
            x: current_hex.0,
            y: current_hex.1,
        };

        let neighbors = hex.all_neighbors();

        for neighbor in neighbors {
            if let Some(edge_cost) = cost(neighbor) {
                let total_cost = current_cost + edge_cost + 1;

                let current_cumulative_cost = *cumulative_costs
                    .get(&neighbor)
                    .map(|n: &(Hex, u32)| &n.1)
                    .unwrap_or(&std::u32::MAX);

                if total_cost < current_cumulative_cost && total_cost <= budget {
                    cumulative_costs.insert(neighbor, (hex, total_cost));
                    priority_queue.push(((neighbor.x, neighbor.y), total_cost));
                }
            }
        }
    }

    cumulative_costs
}

The reason why I'm returning HashMap<Hex, (Hex, u32)> here is so one can easily use a helper function to find the shortest path to any tile within the field of movement.
pub fn get_path(target: &Hex, parents: &HashMap<Hex, (Hex, u32)>) -> Vec<Hex> {
    let mut rev = vec![target.clone()];
    let mut next = target.clone();
    while let Some((parent, _)) = parents.get(&next) {
        rev.push(parent.clone());
        next = parent.clone();
    }
    rev.reverse();
    rev
}

Hey @Godnoken could you check #142 ? I think it fixes the problems

Hey bud,
I'll try to take a look at it within the next few days. I'm currently travelling in the UK :)

Closing for now, feel free to reopen if necessary


[BUG]: AzureFileCopy to VM fails

New issue checklist

[X] I searched for existing GitHub issues
[ ] I read pipeline troubleshooting guide
[ ] I checked how to collect logs

Task name
AzureFilecopy
Task version
5.237.1
Issue Description
The AzureFileCopy command started failing last week after presumably being updated. The error being reported is:
2024-03-21T10:42:55.7596977Z -ExecutionPolicy Unrestricted -Command ".
2024-03-21T10:42:55.7597475Z 'C:\Users\adminuser\AppData\Local\Temp\0a0b6c85-26a3-4c18-803e-537ce07dce30.ps1'"
2024-03-21T10:42:55.7597735Z
2024-03-21T10:42:55.7739358Z VERBOSE: Using default AzCopy arguments for dowloading to VM
2024-03-21T10:42:55.7740270Z
2024-03-21T10:42:55.7787839Z VERBOSE: ##[command] & azcopy copy
2024-03-21T10:42:55.7788454Z
2024-03-21T10:42:56.4219789Z "https://XXXdevconfigstorage.blob.core.windows.net/e576feb6-e8bb-47a2-8806-55dc5fea1c13*****"
2024-03-21T10:42:56.4220051Z
2024-03-21T10:42:56.4220291Z "c:\temp" --recursive --log-level=INFO
2024-03-21T10:42:56.4220397Z
2024-03-21T10:42:56.4220445Z
2024-03-21T10:42:56.4220489Z
2024-03-21T10:42:56.4220763Z failed to parse user input due to error: cannot use wildcards in the path section of the URL except in trailing "/*". If you wish to use * in your URL, manually encode it to %2A
2024-03-21T10:42:56.4220980Z
Apparently the SAS token is being added to the URL but it is missing the ? separator.
We have implemented a workaround as proposed by another user using two tasks, one to upload to the blob and another PowerShell script to copy from the blob to the VM but this has started to fail this week.
The problem with the workaround is that it appears that the azcopy command is not being found on the Hosted Agent so the workaround no longer works for us. The error reported is:
2024-03-25T16:45:00.5157506Z azcopy : The term 'azcopy' is not recognized as the name of a cmdlet, function, script file, or
2024-03-25T16:45:00.5157839Z At line:1 char:1
2024-03-25T16:45:00.5158421Z + & 'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe' -NoLogo ...
2024-03-25T16:45:00.5159704Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2024-03-25T16:45:00.5160428Z     + CategoryInfo          : NotSpecified: (azcopy : The te...cript file, or :String) [], RemoteException
2024-03-25T16:45:00.5160843Z     + FullyQualifiedErrorId : NativeCommandError
2024-03-25T16:45:00.5161132Z
2024-03-25T16:45:00.5161307Z
2024-03-25T16:45:00.5264081Z operable program. Check the spelling of the name, or if a path was included, verify that the path
2024-03-25T16:45:00.5264566Z
2024-03-25T16:45:00.5344297Z is correct and try again.
Currently we can't release any of our software so development and testing teams are blocked.
Environment type (Please select at least one enviroment where you face this issue)

[ ] Self-Hosted
[X] Microsoft Hosted
[ ] VMSS Pool
[ ] Container

Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
Windows Server 2019/2022
Operation system
Windows Server 2019/2022
Relevant log output
Errors are inline above.

Full task logs with system.debug enabled

 [REPLACE THIS WITH YOUR INFORMATION]

Repro steps
An AzureFileCopy task to copy a file onto a VM was working correctly great before last week but started to fail. We noted that the build version of the task changed from 5.236.1 (worked okay) to 5.237.1 (fails).

We are also facing the same issue with the AzureFileCopy task, trying to use workaround with azcopy to copy the builds to the VMs,
Those are also failing as well, all of the deployments have been blocked for the team.

Hi all, the task version is now reverted to 5.236.1. can you please rerun your pipeline and confirm back with the result.  Thanks

Seeing the same error with version 5.236.1.

Re-ran the pipelines

Hi all, the task version is now reverted to 5.236.1. can you please rerun your pipeline and confirm back with the result. Thanks

Re-ran the deployment pipelines, are the Azure file copy task's revert didn't helped with the issue,
We have used the AzureFileCopy version 3,4 and 5 as well
The problem we are facing lies in the Azure PS 11.0.0+ . More specifically Az.Storage 6.0.0+. There was a breaking change in New-AzStorageContainerSASToken -- the return value is no longer prefixed with a question mark. The code of the tasks relies on that, though.
https://learn.microsoft.com/en-us/powershell/azure/release-notes-azureps#azstorage-600
https://learn.microsoft.com/en-us/powershell/azure/migrate-az-11.0.0#azstorage
The task log is not very helpful as it displays the source URI with five asterisks appended, while in reality, a slash, an asterisk, and the SAS token are appended.
The New-AzStorageContainerSASToken cmdlet is called in Generate-AzureStorageContainerSASToken defined in AzureUtilityAz1.0.ps1. That one is called from Copy-FilesToAzureVMsFromStorageContainer defined in Utility.ps1, which is called from AzureFileCopy.ps1 itself. There, it can be seen that Copy-FilesToAzureVMsFromStorageContainer gets the "fileCopyJobScript" scriptblock from AzureFileCopyRemoteJob.ps1. At the bottom, see the uses of containerSasToken: "$containerURL/*$containerSasToken"

Re-ran the pipelines

Hi all, the task version is now reverted to 5.236.1. can you please rerun your pipeline and confirm back with the result. Thanks

Re-ran the deployment pipelines, are the Azure file copy task's revert didn't helped with the issue, We have used the AzureFileCopy version 3,4 and 5 as well The problem we are facing lies in the Azure PS 11.0.0+ . More specifically Az.Storage 6.0.0+. There was a breaking change in New-AzStorageContainerSASToken -- the return value is no longer prefixed with a question mark. The code of the tasks relies on that, though.
https://learn.microsoft.com/en-us/powershell/azure/release-notes-azureps#azstorage-600 https://learn.microsoft.com/en-us/powershell/azure/migrate-az-11.0.0#azstorage
The task log is not very helpful as it displays the source URI with five asterisks appended, while in reality, a slash, an asterisk, and the SAS token are appended.
The New-AzStorageContainerSASToken cmdlet is called in Generate-AzureStorageContainerSASToken defined in AzureUtilityAz1.0.ps1. That one is called from Copy-FilesToAzureVMsFromStorageContainer defined in Utility.ps1, which is called from AzureFileCopy.ps1 itself. There, it can be seen that Copy-FilesToAzureVMsFromStorageContainer gets the "fileCopyJobScript" scriptblock from AzureFileCopyRemoteJob.ps1. At the bottom, see the uses of containerSasToken: "$containerURL/*$containerSasToken"

Great work Sherlock! ;o)

Hi all, the task version is now reverted to 5.236.1. can you please rerun your pipeline and confirm back with the result. Thanks

See gusAntonis reply above. Adding the ? and rolling forward might be the simpler route as both seem dependent on that breaking change in the other module. Just change the line to:
"$containerURL/*?$containerSasToken"

Transfer to the blob appeared to work okay but it's the azcopy to the VM that is raising the error. Confirmed that 5.236.1 was the last successful version run.
Log file for the failure is:
2024-03-26T08:48:40.4356045Z INFO: Logout succeeded. 2024-03-26T08:48:41.0590923Z INFO: AzCopy.exe: A newer version 10.23.0 is available to download 2024-03-26T08:48:41.0591071Z 2024-03-26T08:48:41.0600519Z INFO: AzCopy.exe: A newer version 10.23.0 is available to download 2024-03-26T08:48:41.0601575Z 2024-03-26T08:48:50.0901423Z PSSession created for Machines:'utcbldevsol.uksouth.cloudapp.azure.com' 2024-03-26T08:48:50.4821322Z ================================================ utcbldevsol.uksouth.cloudapp.azure.com ================================================ 2024-03-26T08:48:55.4789167Z ================================================ utcbldevsol.uksouth.cloudapp.azure.com ================================================ 2024-03-26T08:48:55.4879442Z ##[command]& 'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe' -NoLogo -NoProfile -NonInteractive 2024-03-26T08:48:55.4879760Z -ExecutionPolicy Unrestricted -Command ". 2024-03-26T08:48:55.4880058Z 'C:\Users\adminuser\AppData\Local\Temp\17c14967-f0bd-4737-a6b1-a9fa1354e0a1.ps1'" 2024-03-26T08:48:55.4930051Z Directory: C: 2024-03-26T08:48:55.4962444Z Mode LastWriteTime Length Name 2024-03-26T08:48:55.4973726Z ---- ------------- ------ ---- 2024-03-26T08:48:55.4983831Z d----- 3/26/2024 8:48 AM ADO_AzCopyV10 2024-03-26T08:48:55.4994545Z VERBOSE: Preparing to expand... 2024-03-26T08:48:55.5004869Z VERBOSE: Created 'C:\ADO_AzCopyV10\AzCopy\AzCopy.exe'. 2024-03-26T08:49:00.4981637Z ================================================ utcbldevsol.uksouth.cloudapp.azure.com ================================================ 2024-03-26T08:49:00.5007325Z failed to parse user input due to error: cannot use wildcards in the path section of the URL except in trailing "/*". If you wish to use * in your URL, manually encode it to %2A

Can confirm the same behaviour for our pipeline as well,
Attaching logs for reference (removed the names for files and machines),
##[debug]Leaving Invoke-RemoteScript.
##[debug]Parallel file copy: Invoke-RemoteScript threw exception
##[debug]Microsoft.PowerShell.Commands.WriteErrorException: Non-Zero exit code: '1' for ComputerName: 'VIMBC001'
##[debug]Processed: ##vso[task.logissue type=error;code={"Task_Internal_Error":"CopyingToAzureVMFailed"};]
##[debug][Azure Call]Deleting container: 074e42-9fdd-a9289a670df7 in storage account: pstas001
##[debug]Removed container '07a402-9fdd-a9289670df7' successfully.
##[debug][Azure Call]Deleted container: 07a4e42-9fdd-a9289a670df7 in storage account: stosa001
##[debug]Completed Azure File Copy Task for Azure VMs Destination
##[debug]Leaving D:\a\_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\4.237.1\AzureFileCopy.ps1.
##[debug]Trying to disconnect from Azure and clear context at process scope
##[debug]Cannot verify the Microsoft .NET Framework version 4.7.2 because it is not included in the list of permitted versions.
##[debug]Populating RepositorySourceLocation property for module Az.Accounts.
##[debug]Loading module from path 'C:\Modules\az_11.3.1\Az.Accounts\2.16.0\Az.Accounts.psm1'.
 This has severely impacted the deployment pipelines.

All deployments have been obstructed guy, hope we understand the urgency here.

Can we get an update on this issue please? It is causing problems with many users and it seems that a simple fix of adding the ? between the url and sas token will resolve the problem. I look forward to an update.

Hi @AndrewBates666 @gusAntonis , I have created a PR https://github.com/microsoft/azure-pipelines-tasks/pull/19700 for the proposed changes.
I will try to get in touch with task owners to get to approved and merged ASAP.
Meanwhile, sine i am unable to repro the issue from my end, I might need anyone of you to assist here, to follow the below simple procedure to test if it fixes the issue. If the changes do work, same can be followed to unblock your case here.
Local task test
Have code changes ready (checkout to this branch), navigate to task root path ie C:...\azure-pipelines-tasks
Note: tfx build works only on node 8 and 10, so make sure to use node 8/10 while testing the task
Step 1: run "npm i"
Step 2: Build:

node make.js build --task AzureFileCopyV5

Step 3: Install tfx:

npm install -g tfx-cli

Step 4: Login:

tfx login
Service URL : [your task test org url]
PAT :

Step 5: Upload:

tfx build tasks upload --task-path C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\ AzureFileCopyV5

New task version will be uploaded to your org, please test the changes by running the pipeline.
Step 6: Delete: // optional and can be done after all the testing is completed.

tfx build tasks delete --task-id {           }

Hi @AndrewBates666 @gusAntonis , I have created a PR #19700 for the proposed changes. I will try to get in touch with task owners to get to approved and merged ASAP.
Meanwhile, since i am unable to repro the issue from my end, I might need anyone of you to assist here, to follow the below simple procedure to test if it fixes the issue. If the changes do work, same can be followed to unblock your case here immediately without need to wait for regular task deployment (which might take upto weeks)
Local task test Have code changes ready (checkout to this branch), navigate to task root path ie C:...\azure-pipelines-tasks Note: tfx build works only on node 8 and 10, so make sure to use node 8/10 while testing the task
Step 1: run "npm i"
Step 2: Build:

node make.js build --task AzureFileCopyV5

Step 3: Install tfx:

npm install -g tfx-cli

Step 4: Login:

tfx login
Service URL : [your task test org url]
PAT :

Step 5: Upload:

tfx build tasks upload --task-path C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\ AzureFileCopyV5

New task version will be uploaded to your org, please test the changes by running the pipeline.
Step 6: Delete: // optional and can be done after all the testing is completed.

tfx build tasks delete --task-id {           }

I have reviewed the PR and testing the same in our env as well.

Hi @AndrewBates666 @gusAntonis , I have created a PR #19700 for the proposed changes. I will try to get in touch with task owners to get to approved and merged ASAP.
Meanwhile, since i am unable to repro the issue from my end, I might need anyone of you to assist here, to follow the below simple procedure to test if it fixes the issue. If the changes do work, same can be followed to unblock your case here immediately without need to wait for regular task deployment (which might take upto weeks)
Local task test Have code changes ready (checkout to this branch), navigate to task root path ie C:...\azure-pipelines-tasks Note: tfx build works only on node 8 and 10, so make sure to use node 8/10 while testing the task
Step 1: run "npm i"
Step 2: Build:

node make.js build --task AzureFileCopyV5

Step 3: Install tfx:

npm install -g tfx-cli

Step 4: Login:

tfx login
Service URL : [your task test org url]
PAT :

Step 5: Upload:

tfx build tasks upload --task-path C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\ AzureFileCopyV5

New task version will be uploaded to your org, please test the changes by running the pipeline.
Step 6: Delete: // optional and can be done after all the testing is completed.

tfx build tasks delete --task-id {           }

I have reviewed the PR and testing the same in our env as well.

We can only test in the Hosted Agents so let us know when it will be released and we will run tests on it. Best A

Hi @AndrewBates666 @gusAntonis , I have created a PR #19700 for the proposed changes. I will try to get in touch with task owners to get to approved and merged ASAP.
Meanwhile, since i am unable to repro the issue from my end, I might need anyone of you to assist here, to follow the below simple procedure to test if it fixes the issue. If the changes do work, same can be followed to unblock your case here immediately without need to wait for regular task deployment (which might take upto weeks)
Local task test Have code changes ready (checkout to this branch), navigate to task root path ie C:...\azure-pipelines-tasks Note: tfx build works only on node 8 and 10, so make sure to use node 8/10 while testing the task
Step 1: run "npm i"
Step 2: Build:

node make.js build --task AzureFileCopyV5

Step 3: Install tfx:

npm install -g tfx-cli

Step 4: Login:

tfx login
Service URL : [your task test org url]
PAT :

Step 5: Upload:

tfx build tasks upload --task-path C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\ AzureFileCopyV5

New task version will be uploaded to your org, please test the changes by running the pipeline.
Step 6: Delete: // optional and can be done after all the testing is completed.

tfx build tasks delete --task-id {           }

I have reviewed the PR and testing the same in our env as well.

We can only test in the Hosted Agents so let us know when it will be released and we will run tests on it. Best A

once @gusAntonis confirms the changes to be working fine, I will proceed to get in touch with relevant task owners to be  get it merged.

Hi @AndrewBates666 @gusAntonis , I have created a PR #19700 for the proposed changes. I will try to get in touch with task owners to get to approved and merged ASAP.
Meanwhile, since i am unable to repro the issue from my end, I might need anyone of you to assist here, to follow the below simple procedure to test if it fixes the issue. If the changes do work, same can be followed to unblock your case here immediately without need to wait for regular task deployment (which might take upto weeks)
Local task test Have code changes ready (checkout to this branch), navigate to task root path ie C:...\azure-pipelines-tasks Note: tfx build works only on node 8 and 10, so make sure to use node 8/10 while testing the task
Step 1: run "npm i"
Step 2: Build:

node make.js build --task AzureFileCopyV5

Step 3: Install tfx:

npm install -g tfx-cli

Step 4: Login:

tfx login
Service URL : [your task test org url]
PAT :

Step 5: Upload:

tfx build tasks upload --task-path C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\ AzureFileCopyV5

New task version will be uploaded to your org, please test the changes by running the pipeline.
Step 6: Delete: // optional and can be done after all the testing is completed.

tfx build tasks delete --task-id {           }

Hi @AndrewBates666 @gusAntonis , I have created a PR #19700 for the proposed changes. I will try to get in touch with task owners to get to approved and merged ASAP.
Meanwhile, since i am unable to repro the issue from my end, I might need anyone of you to assist here, to follow the below simple procedure to test if it fixes the issue. If the changes do work, same can be followed to unblock your case here immediately without need to wait for regular task deployment (which might take upto weeks)
Local task test Have code changes ready (checkout to this branch), navigate to task root path ie C:...\azure-pipelines-tasks Note: tfx build works only on node 8 and 10, so make sure to use node 8/10 while testing the task
Step 1: run "npm i"
Step 2: Build:

node make.js build --task AzureFileCopyV5

Step 3: Install tfx:

npm install -g tfx-cli

Step 4: Login:

tfx login
Service URL : [your task test org url]
PAT :

Step 5: Upload:

tfx build tasks upload --task-path C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\ AzureFileCopyV5

New task version will be uploaded to your org, please test the changes by running the pipeline.
Step 6: Delete: // optional and can be done after all the testing is completed.

tfx build tasks delete --task-id {           }

I have reviewed the PR and testing the same in our env as well.

We can only test in the Hosted Agents so let us know when it will be released and we will run tests on it. Best A

once @gusAntonis confirms the changes to be working fine, I will proceed to get in touch with relevant task owners to be get it merged.

My agents are also azure hosted, i can give this a try to use a self hosted agent as well,
But can't we use the updated task from the org. on the azure hosted agents as well.

@v-mohithgc Hi, versioning of the tasks seems odd to me. When you complete an update rather than adding a new task version you seem to update all versions. This means that if we see a failure we don't have any option to use another version. Logically I'd expect new versions to be added so AzureFileCopy@7 which has the new code in. We've seen several failures with AzureFileCopy over the past few months and it breaks our pipelines and we don't have a way to revert unless you do so; we've lost hours of dev/test through the breakage. Just a thought...

Hi @AndrewBates666 @gusAntonis , I have created a PR #19700 for the proposed changes. I will try to get in touch with task owners to get it approved and merged ASAP.
Meanwhile, since i am unable to repro the issue from my end, I might need anyone of you to assist here, to follow the below simple procedure to test if it fixes the issue. If the changes do work, same can be followed to unblock your case here immediately without need to wait for regular task deployment (which might take upto weeks)
Local task test Have code changes ready (checkout to this branch), navigate to task root path ie C:...\azure-pipelines-tasks Note: tfx build works only on node 8 and 10, so make sure to use node 8/10 while testing the task
Step 1: run "npm i"
Step 2: Build:

node make.js build --task AzureFileCopyV5

Step 3: Install tfx:

npm install -g tfx-cli

Step 4: Login:

tfx login
Service URL : [your task test org url]
PAT :

Step 5: Upload:

tfx build tasks upload --task-path C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\ AzureFileCopyV5

New task version will be uploaded to your org, please test the changes by running the pipeline.
Step 6: Delete: // optional and can be done after all the testing is completed.

tfx build tasks delete --task-id {           }

So we have few issues here,
First the path specified in the upload path needs to be corrected, another \ needs to be added before _build,
And i am not able to find the build task at the specified directory :
C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\

Hi @AndrewBates666 @gusAntonis , I have created a PR #19700 for the proposed changes. I will try to get in touch with task owners to get it approved and merged ASAP.
Meanwhile, since i am unable to repro the issue from my end, I might need anyone of you to assist here, to follow the below simple procedure to test if it fixes the issue. If the changes do work, same can be followed to unblock your case here immediately without need to wait for regular task deployment (which might take upto weeks)
Local task test Have code changes ready (checkout to this branch), navigate to task root path ie C:...\azure-pipelines-tasks Note: tfx build works only on node 8 and 10, so make sure to use node 8/10 while testing the task
Step 1: run "npm i"
Step 2: Build:

node make.js build --task AzureFileCopyV5

Step 3: Install tfx:

npm install -g tfx-cli

Step 4: Login:

tfx login
Service URL : [your task test org url]
PAT :

Step 5: Upload:

tfx build tasks upload --task-path C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\ AzureFileCopyV5

New task version will be uploaded to your org, please test the changes by running the pipeline.
Step 6: Delete: // optional and can be done after all the testing is completed.

tfx build tasks delete --task-id {           }

So we have few issues here, First the path specified in the upload path needs to be corrected, another \ needs to be added before _build,
And i am not able to find the build task at the specified directory : C:\AzurePipelineTask\azure-pipelines-tasks_build\Tasks\

yes its _build... for some reason GH is not allowing that

@v-mohithgc
I can confirm that your changes are correct, we used the AzureFileCopy version 5 and issues on the failure on that stage are resolved,
We need to merge this PR asap guys.

I see that this has been merged a week ago, however it has not been released yet (we have 5.237.11 btw.)
When can we expect a release of the fix as all deployments are currently dead?

I see that this has been merged a week ago, however it has not been released yet (we have 5.237.11 btw.)
When can we expect a release of the fix as all deployments are currently dead?

I have been informed that the new task deployment cycle was started yesterday, and latest version (i.e 5.237.12) is already out on few regions. It might take the next 2-4 working days to be rolled out on all the regions.

I see that this has been merged a week ago, however it has not been released yet (we have 5.237.11 btw.)
When can we expect a release of the fix as all deployments are currently dead?

I have been informed that the new task deployment cycle was started yesterday, and latest version (i.e 5.237.12) is already out on few regions. It might take the next 2-4 working days to be rolled out on all the regions.

Thanks for the update @v-mohithgc,
Really appreciate the good work you guys are doing.
I would like to know more about the release cycles and the process how the tasks get updated, any suggestions.

We still have version 5.237.11 in our deployments. We use Azure Hosted agents in region West Europe. When is the fix in version 5.237.12 available in West Europe?


dropAllForeignKeyConstraints performance/errors in liquibase 3.10.3, works perfect in liquibase 3.4.2

Environment
Liquibase Version: works fine on 3.4.2, broken on 3.10.3
Liquibase Integration & Version: CLI
Database Vendor & Version: MySql 5.6
Operating System Type & Version: centos 7
Description
In 3.10.3 a dropAllForeignKeyConstraints change is incredibly slow (168087ms vs 55ms) and produces errors:
14:52:17.216 INFO  [liquibase.util.SqlUtil]: Unknown default value: value 'XXX4' type SET('XXX1', 'XXX2', 'XXX3', 'XXX4') (SET('XXX1', 'XXX2', 'XXX3', 'XXX4')).   We will add this to our list of issues to process. If you have a suggested PR for a fix that would be great.

It looks like its still an issue on liquibase 4.15. We need to investigate further to be sure. There is a PR that possible fix this. Looking more into it after the PR is validated.

Hi @FBurguer  - PR https://github.com/liquibase/liquibase/pull/2155 fixes it. I'm moving this issue to development to link it with the PR.


noise scaled mas

What do you think of adding noise scaled mas and duration discriminator?

Like vits2?

Yep

Should be alright.


TypeScript Migration - Test-Suite.js

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Part of #43778

This PR migrates the TestSuite.js file to TS.
Why in draft
I couldn't figure out the xs property we pass to the div at line 48. Can anybody help me understand what that is for, so that I can complete the migration to TS?

https://github.com/freeCodeCamp/freeCodeCamp/pull/43790
This file already has a PR


Jesse: Watch glaciercs.com

Jesse requests the watch of the watch_keyword glaciercs\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

glaciercs\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Ignore some metadata modificaiton events

We ignore metadata modificaiton events for the profiles directory
tree as it is a symlink forest that is used to keep track of
channels and nix will unconditionally update the metadata of each
link in this forest.
These ModifyKind::Metadata events are spurious and they can easily
cause a rebuild-loop when the shell.nix files  does not pin it's
version of nixpkgs or other channels.
When a Nix channel is updated we will recieve a data modification
event, so ignoring these metadata modifications will not impact
lorri's ability to correctly watch for changes.
Fixes #176

This seems to be a pragmatic solution to the issue!
Two comments:

Please run cargo fmt, this should fix the CI failure. You can run CI locally using the ci command provided by the top-level shell.nix to check if everything is in order.
Please add a comment explaining why metadata events in that particular directory are ignored, perhaps even including a link to https://github.com/NixOS/nix/blob/629b9b0049363e091b76b7f60a8357d9f94733cc/src/libstore/local-store.cc#L74-L80 to be really precise.

Yeah, I forgot to run cargo fmt, my bad. I'll put most of the commit message into a comment above the code in question.

@curiousleo I have updated the commit to fix the formatting and include a comment explaining why we ignore the profiles dir and linking to the code, as I like being really precise.

include a comment explaining why we ignore the profiles dir and linking to the code

That sounds good! I can't see that comment in the current state of the PR though. Perhaps that commit still needs to be pushed?

I'm about to sign off for the next couple of weeks. Don't take any lack of activity on this PR personally, please :)

Yep. Sorry about that. I see it on github now.

Gotta get comment spellcheck setup. Thanks for the fixes.

Looking absolutely fabulous! Thanks for this patch.


Scrollbars shown in alerts even when not necessary

Ionic version:  (check one with "x")
[ ] 1.x
[ x] 2.x
I'm submitting a ...  (check one with "x")
[ x] bug report
[ ] feature request
[ ] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
Current behavior:

Testing in desktop (the docs), I'm seeing scroll bars in the alert UI when it's not necessary:

Expected behavior:
I would expect the scroll bars to only show up when necessary. I used DevTools to remove most of the text, curious to see if maybe it was just a bit too much, but it didn't remove the scrollbars.
Steps to reproduce:
View the docs/demo in Chrome.

I'm seeing it elsewhere as too. To be clear, this is all in the desktop, not the mobile emulator/device, but it seems new.

Hello, thanks for opening an issue with us! This is on a windows 10 machine im guessing? Do you see the same behavior in edge?

Wow, actually it is worse in Edge:


D_Loss and G_loss gives Nan value after 10 epochs on horse2zebra dataset.

i run DRIT code on horse2zebra dataset after 10 epochs it gives nan for g_loss and d_loss i use original gan loss function.

May I ask how you solved this problem？


Punctuation

Added some Punctuations

Please find an issue you want to fix and only send PR for open issues.


PHP Failed to Restart after Upgrade

Something's wrong with 3.1.7. I just updated and fired the stack upgrade command.
At the end of it, It started to restart the stack and then it seems like it tried to restart php and not php5-fpm like with the standard ee stack restart (Which works fine).
So it ended up showing status as Failed.
I'm on Ubuntu btw.
P.S Updated on 2 of my droplets and both show the same thing.

Duplicate of #550 .
Hello We fixed this bug in master Branch, soon it will be released with new version of EasyEngine.


Explain 'secure by default'

Can you please help to explain the case to help others:
https://bitbucket.org/snakeyaml/snakeyaml/issues/569/cve-2022-41854-is-still-exploitable-on

I think I did, but by posing questions. Let me know if that's not sufficient


data-method throw exception "Uncaught expected string value for 'container' option; got object"

There is not new code, it was working like a charm, but when we updated all packages in composer it becomes with errors.
Other pjax works well, but with a link data-method we got error.
I found out, if I add 'pjax-container' => '#' . $_pjaxId, to my link, it works well. Is its compatibility were broken and we have to add everywhere this param?
In composer "minimum-stability": "stable",
What steps will reproduce the problem?
<?php Pjax::begin([
    'enablePushState' => false,
    'options' => [
        'class' => 'book_action',
    ],
    'scrollTo' => false,
    'id' => $_pjaxId,
])?>
<!-- another html -->
<?= Html::a(
    UserBook::STATUS_LIST[UserBook::STATUS_FINISHED],
    ['set-book-status', 'id' => $model->primaryKey, 'status' => UserBook::STATUS_FINISHED],
    [
        'class' => 'pull-right btn-xs btn btn-bg-status-' . UserBook::STATUS_FINISHED,
        'data' => [
            'method' => 'post',
            'pjax' => true,
        ],
    ]
)?>
<!-- another html -->
<?php Pjax::end() ?>

What is the expected result?
correctly sended POST request using ajax
What do you get instead?
"Uncaught expected string value for 'container' option; got object" in console section of browser
Additional info

Q
A

Yii version
2.0.12

PHP version
7.0.24

Operating system
Ubuntu 14.04.5

Duplicate of https://github.com/yiisoft/yii2/pull/14906


microsoft/mssql-server-linux exits immediately on Docker for Windows

Running
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=yourStrong(!)Password' microsoft/mssql-server-linux:latest
on Docker for Windows (hyper-v) outputs
Aborted
straight away, and the container stops

Did you change your memory settings to at least 4 GB of RAM for the Moby VM?

Yes, 8gb is allocated to it. Note this works perfectly when switching to the VirtualBox based Docker Toolbox on Windows
PS C:\Users\user> docker info Containers: 2  Running: 1  Paused: 0  Stopped: 1 Images: 9 Server Version: 17.11.0-ce-rc4 Storage Driver: overlay2  Backing Filesystem: extfs  Supports d_type: true  Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins:  Volume: local  Network: bridge host ipvlan macvlan null overlay  Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 992280e8e265f491f7a624ab82f3e238be086e49 runc version: 0351df1c5a66838d0c392b4ac4cf9450de844e2d init version: 949e6fa Security Options:  seccomp   Profile: default Kernel Version: 4.9.60-linuxkit-aufs Operating System: Docker for Windows OSType: linux Architecture: x86_64 CPUs: 2 Total Memory: 7.769GiB Name: linuxkit-00155d4b0102 ID: LWTB:FKS3:F7WN:4MSR:5WBT:VQCJ:LQMH:4GC4:7DKD:ZNQP:66IF:N3MO Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): true  File Descriptors: 31  Goroutines: 48  System Time: 2017-11-20T22:10:20.3114299Z  EventsListeners: 1 Registry: https://index.docker.io/v1/ Experimental: true Insecure Registries:  <IP_ADDRESS>/8 Live Restore Enabled: false

Try running without -d and see if there is anything interesting in the output.

Running without -d, and with -it
PS C:\Users\user>docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=yourStrong(!)Password' -it microsoft/mssql-server-linux:latest

Aborted

I have confirmed that this container does work on Docker for Windows (hyper-v) on another system, so it looks like it's something in my environment. The strange part is this seems to be the only container which doesn't work.

With my case, I found that the problem was the password I set too simple, 123456.
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=123456' -p 2433:1433  -v -d mcr.microsoft.com/mssql/server
Solution: used complex password. This cmd is okay.
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=XYZxyz!@#$1234' -p 2433:1433  -v -d mcr.microsoft.com/mssql/server


Call splitlines before passing output to get_process_per_host.

Ensure splitlines is called before passing command output to get_process_per_host. This ensures that we are correctly checking if the server is started.

oh, it was only a few places? excellent. can you change the name of get_process_per_host to suggest that you have to have splitlines?
looks good, though make sure to run make lint

Yeah, there were only 16 uses of get_process_per_host in the product tests, and most of them were already correct. make lint comes up clean.
Per offline conversation, I'm going to make get_process_per_host determine if it has been passed a string by checking hasattr(output_lines, 'iter') and raising an Error if that returns false. Strings are iterable, just like lists, but they don't have an 'iter' attribute, so we can distinguish them from lists that way. We're in agreement that this isn't terribly pythonic, but it does make it harder to reintroduce bugs.

looks good

@ebd2 you pushed this right? if so, you should close the PR

@cawallin I did. Sorry about that.


Feat: Creating a custom 404 notfound page for the website

Describe the feature
Creating a custom 404 notfound page for the website
Add ScreenShots
Creating a custom 404 notfound page for the website
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I'm a VSoC'24 contributor
[X] I want to work on this issue

solved via #656


Testing Suite and Continuous Integration Pipeline for WorldBrain

Modules To Be Tested(not in order)

[ ] Activity Logger
[ ] Blacklist
[ ] Common UI(single snapshot test)
[ ] Imports
[ ] Local Page
[ ] Options
[ ] Overview
[ ] Page Analysis
[ ] Page Storage
[ ] Popup
[ ] Scheduled Task
[ ] Search
[ ] Util

Testing issue at Webmemex for reference.
Since I see the project is already setup to run with jest, I am assuming that is the selected test suite for this project.

How should the testing suite be setup?

Should we have all the tests beside the modules?(eg sr/activity-logger/index.test.js)
Should we create a separate test folder containing all the tests?(rg mocks/activity_logger/index.test.js)

I think we should go the first way as it has been setup at webmemex. This would allow us to easily manage the tests and move them around. It would also allow us to have an implementation of the test just beside the module which would allow us to make change to the tests easier.
Not to mention the work it would go to manage the same folder structure in two places(src as well as tests).
@poltak @oliversauter Any opinions?

Should we have all the tests beside the modules?(eg sr/activity-logger/index.test.js)

100% in favour of this option. No reason to create a new mirror dir tree.


How the app should work

1.) User visits site (e.g. http://example.com/
2.) Can log in using Google's oAuth2
3.) Use Google Calendar API to get/delete events

Backend: Django (Python 3.6)
Frontend: React

Added some fields for the Event model in: app/app/events/models.py

Was able to get serialization to work!
HTTP 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

[
    {
        "event_name": "first pow wow!",
        "organizer": "",
        "location": "",
        "pub_date": "2017-07-23T13:44:07.150904Z",
        "event_date": "2017-07-23T13:44:07.150914Z"
    },
    {
        "event_name": "second pow wow!!",
        "organizer": "",
        "location": "",
        "pub_date": "2017-07-23T13:44:41.709583Z",
        "event_date": "2017-07-23T13:44:41.709590Z"
    }
]

@tronicdata You might want to start reading from here.
If you have any questions, feel free to ask!


Minted compatibility patch

LaTeX: workaround to ensure compatibility with minted
This reverts already merged #1735 (after a commit from #1736 to fix it)
and applies the first envisioned method discussed in #1734.
The reason is that the #1735-#1736 method only partially repairs the
minted compatibility.  Minted queries from Pygments the stylesheet
with a command prefix equal to the style name, which may contain
(at least, so far) characters such as - and _, which are not normally
allowed in LaTeX macros.  So it modifies the meaning of - and _ before
telling LaTeX to input the Pygments provided stylesheet.  Restoring
the normal meaning of - from inside the stylesheet must be carefully
localized: at top and bottom of stylesheets some macros will use in
their names the - and there the - must be the weird minted one, not
the normal one.
The #1735-#1736 method thus does fix compatibility with minted but only
for those style names not using a -, but it creates another issue if
used with e.g. style "paraiso-dark" which has a - character in its name.
"De guerre lasse", I feel it is simpler to use the somewhat strange
very localized hotfix of prefixing - by \string at the one spot where
we need it to be its normal self.  Hence this PR.
Fix #1734.
Relates: https://github.com/gpoore/minted/issues/294
I was not familiar with minted LaTeX code.  It has taken me some time to understand what is involved (and on this occasion I discovered that with LaTeX of spring 2021, loading  a file results in a log trace in about 10000 (!!) macro calls to the LaTeX3 syntax in the minted context so it is nowadays extremely laborious to examine a LaTeX problem via \tracingmacros1.).
The good thing in minted approach is that it uses weird command prefix only to recover the stylesheet. Regarding actual mark-up for highlighting code it uses \PYG default prefix. So the catcode issues are, fortunately, localized to only actually loading the stylesheet for a given stylename.
Although minted will probably get an upgrade, many people will be for some time with its current version. So, despite this being entirely a design problem on minted side, it must be fixed on Pygments side. It is very ugly to add this \string like I do, but this works and solves the problem, as identified so far.

Thanks! I can't tell that I fully understand what is going on here :) I've applied #1735 and #1737 to both master and the release branch, and I understand your PRs correctly that's all that is needed. Can you please confirm before we prepare another release?

Can you please confirm before we prepare another release?

I checked the diff for pygments/formatters/latex.py with 2.8.0 on 2.8.x and master, and it correctly indicates replacement of -\fboxrule by \string -\fboxrule.
So yes, this should fix the issue (also for styles such as paraiso-dark) in a way robust against upstream minted changes. The \string is core TeX syntax to convert the following control sequence to a name, or to convert the following character to have "catcode 12", which is the normal for digits, +, - , ..., and they must be this way for usage in dimension assignments. Fortunately even with calc package loaded which complicates \setlength, the parsing done by calc is gentle enough for \string action to not be inhibited.
I tested with this (which contains non-sense to try to trigger the \PYG{err} which in some styles does a border; this is not a full test but I hope it is confirmation enough)
\documentclass{article}
\usepackage{minted}

\begin{document}

\begin{minted}{http}
abc
\end{minted}

\begin{minted}[style=paraiso-dark]{http}
abc-xyz: -22
\end{minted}

\begin{minted}[style=algol_nu]{http}
abc-xyz: -22
\end{minted}

\begin{minted}[style=paraiso-dark]{python}
-x = def a && b
\end{minted}

\end{document}

Thanks a lot for checking & the detailed explanation, much appreciated!


Feedback for “Q&A”

What's wrong with staying on this screen

Open your browser and visit http://<IP_ADDRESS>:8080


Add installation via package name

Whelp I'm dumb. Anyway, seems you're ok with the change

danke!

:+1:


Add rosdep key for python3 thriftpy

Adds python3-thriftpy-pip to rosdep/python.yaml file
ThriftPy is a pure python implementation of Apache Thrift in a pythonic way.
pip: https://pypi.org/project/thriftpy/
doc: https://thriftpy.readthedocs.io/en/latest/

I added thriftpy from Pypip repository because it is not available in Ubuntu 16.04 in order to install ros dependency for a kinetic package. If it not possible, i will proceed to change the rule.


Bug In Button By Lakhan06

This is a(n):

[x] Update to an existing Animated Button
[ ] Any Error
[ ] Proposal to the Repository
[ ] Documentation / README.md changes

Details:

CSS & JS files were not properly linked in the button provided by Lakhan06. It was a small error and has been fixed. Just need to make a Pull Request.

Sure Bro,
I'll Keep This In Mind.


feat(config): config through rc file, has precedence over package.json

Current coverage is 100% (diff: 100%)

Merging #62 into master will not change coverage

@@           master   #62   diff @@
===================================
  Files           3     3
  Lines          85    94     +9
  Methods         0     0
  Messages        0     0
  Branches        0     0
===================================
+ Hits           85    94     +9
  Misses          0     0
  Partials        0     0

Powered by Codecov. Last update e60ec37...ab945f5

This is a great feature! Thanks!


Update Word template to match Latex template.

This new Word template has been generated by using Latex-PDF on 2020-11-26. I hope it is up to date. Otherwise we have to introduce small changes in Word or redo the process via up-to-date Latex-PDF.

Thanks @GallLeo, I think it is good as it is and should be merged 👍


Error running tests for RobotName

While running the tests for RobotName, I keep getting this error.
Am I supposed to define a struct with Robot in my code?
If so please mention in the Readme file or the test file.
Getting this error:
./robot_name_test.go:11:12: *Robot is not a type

Answering here too... Yes, you have to define the Robot struct yourself. :)


Bump CASE and UCO tracking pointers to current HEAD states

This PR also tracks CASE-unstable (currently pointing at develop) in the master branch of CASE-Examples.


using the fig.yml

Hi, I try to understand how docker-index work via this project and try to run it on my env.
  If I don't have a dns how can I start the registry and index containers as it seems they depends on each other's hostname?

The fig.yml is for development and testing only. If you want to use the index/registry using fig you'll need to use DNS locally on your system to be able to get it to work. It is possible to do it without fig, but you'll have to start the registry and index separately and without HTTPS you have to pass some additional values to the docker client to get it to talk to an insecure index.
You can always run https://github.com/ekristen/dns-proxy locally to get better control over local DNS or use your /etc/hosts file.

Yep, installed dnsmasq and got things work, thanks


Prefix functions with the name of the file they are in

Just an idea to prevent overlapping.
For example in the file ioh-zfs, is it an idea to prefix these functions with zfs in the name?
For example:
__zfs_get() {
}

This might prevent future function overlapping when things get bigger...
Let me know if you'd like to have this and I'll finish up my changes...

Moving discussion to #181


pydoc.locate cannot locate modules in local dir

Sometimes pydoc.locate fails to locate modules in local dir. Probably because local directory changes to something else.
HotFix:
PYTHONPATH=.:$PYTHONPATH catalyst-dl run -C config.yml

https://github.com/python/cpython/blob/main/Lib/pydoc.py#L1721 -> https://github.com/python/cpython/blob/af9ee57b96cb872df6574e36027cc753417605f9/Lib/importlib/_bootstrap.py#L1167


feat: support loading site url from env var

What kind of change does this PR introduce?
closes https://github.com/supabase/supabase/issues/23296
What is the new behavior?
Allows env substitution for auth.site_url
site_url = "env(MY_ENV)"

Additional context
Add any other context or screenshots.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

2 of 4 (50.0%)  changed or added relevant lines in 1 file are covered.
7 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.03%) to 60.182%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/utils/config.go
2
4
50.0%

Files with Coverage Reduction
New Missed Lines
%

internal/storage/rm/rm.go
2
89.53%

internal/gen/keys/keys.go
5
11.76%

Totals

Change from base Build<PHONE_NUMBER>:
-0.03%

Covered Lines:
6889

Relevant Lines:
11447

💛  - Coveralls


[SPARK-17142][SQL] Complex query triggers binding error in HashAggregateExec

What changes were proposed in this pull request?
In ReorderAssociativeOperator rule, we extract foldable expressions with Add/Multiply arithmetics, and replace with eval literal. For example, (a + 1) + (b + 2) is optimized to (a + b + 3) by this rule.
For aggregate operator, output expressions should be derived from groupingExpressions, current implemenation of ReorderAssociativeOperator rule may break this promise. A instance could be:
SELECT
  ((t1.a + 1) + (t2.a + 2)) AS out_col
FROM
  testdata2 AS t1
INNER JOIN
  testdata2 AS t2
ON
  (t1.a = t2.a)
GROUP BY (t1.a + 1), (t2.a + 2)

((t1.a + 1) + (t2.a + 2)) is optimized to (t1.a + t2.a + 3), which could not be derived from ExpressionSet((t1.a +1), (t2.a + 2)).
Maybe we should improve the rule of ReorderAssociativeOperator by adding a GroupingExpressionSet to keep Aggregate.groupingExpressions, and respect these expressions during the optimize stage.
How was this patch tested?
Add new test case in ReorderAssociativeOperatorSuite.

Test build #64774 has started for PR 14917 at commit dc3b1b2.

Test build #64774 has finished for PR 14917 at commit dc3b1b2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64774/
Test PASSed.

Merged build finished. Test PASSed.

Cool! Thanks for picking this up. cc @JoshRosen
Could you make the description a bit more readable by using markdown formatting (mostly adding 1 or 3 backticks here and there)?

I will try to get to this one ASAP.

@hvanhovell I've updated the description following your advice, thank you for your time!

ping @hvanhovell

@hvanhovell Could you review this PR when you have some time please? Thank you!

LGTM. Merging to master. This is a good find. Thanks!

We should actually make sure that Aggregate is not resolved in these cases.

@jiangxb1987 could also open a backport against 2.0? Thanks!

NVM.

@hvanhovell Thank you! This bug was imported in spark-2.1.0, and in spark-2.0 we don't have the problem.

@hvanhovell Do you mean we should check other optimize rules to ensure that Aggregate operator is not resolved in these cases?

No, I am saying that it might be a good idea to incorporate this rule into Aggregate.resolved. This might make that method to complex.

Sure!Will do it soon!

@hvanhovell In CheckAnalysis we have ensured all aggregateExpressions can be derived from groupingExpressions (refer to function checkValidAggregateExpression), but in optimize stage the rule ReorderAssociativeOperator broke this. I found the Aggregate.resolved is only checked in analyze stage so we failed to detect this case. So, maybe incorporate this rule into Aggregate.resolved brings little help to avoid similar problem. Should we check analysis again after we have performed all optimize rules? Thank you!


Run Ingestion source

run mysql Ingestion source  report errors：
ERROR c.l.m.s.e.update.BulkListener:44 - Failed to feed bulk request. Number of events: 1 Took time ms: -1 Message: failure in bulk execution:
[0]: index [datahubexecutionrequestindex_v2], type [_doc], id [urn%3Ali%3AdataHubExecutionRequest%3A4b688061-4cab-4d84-98b8-d4c824451ce3], message [[datahubexecutionrequestindex_v2/4PHQJv2UR-uCpeYjZwoD_g][[datahubexecutionrequestindex_v2][0]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][urn%3Ali%3AdataHubExecutionRequest%3A4b688061-4cab-4d84-98b8-d4c824451ce3]: document missing]]]

Hi @rmzf9192 ! did you run your mysql ingestion through the CLI or the UI for managed ingestion? also can you confirm that your elastic search pod is up, running, and healthy?

I am experiencing the same issue with ingestion from BigQuery:
[I/O dispatcher 1] ERROR c.l.m.s.e.update.BulkListener:44 - Failed to feed bulk request. Number of events: 4 Took time ms: -1 Message: failure in bulk execution:
[0]: index [datahubexecutionrequestindex_v2], type [_doc], id [urn%3Ali%3AdataHubExecutionRequest%3A69dbfb9d-2dde-4ba3-9d65-bfb086c48b68], message [[datahubexecutionrequestindex_v2/qIjaoF9uSGyhjOsqhNZc2w][[datahubexecutionrequestindex_v2][0]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][urn%3Ali%3AdataHubExecutionRequest%3A69dbfb9d-2dde-4ba3-9d65-bfb086c48b68]: document missing]]]

Datahub v0.9.5. deployed to GKE via Helm

The issue has been resolved in my deployment but I am not sure it's the result of what I did.
Specifically, I run the Datahub prerequisites (elastic/kafka/zookeeper/schema-registry) in a k8s namespace separate from Datahub. Before, I used hostnames like elasticsearch-master.<namespace>. After updating these to FQDNs, i.e., elasticsearch-master.<namespace>.svc.cluster.local, and deploying the Datahub pods again, ingestion started working.

Yeah this can happen when elasticsearch tries to do an upsert on a document that it just deleted (as one example) for when we create entities that were just deleted. We also issued a fix that is known to cause this with failure to properly manage runIds. I'd check out a slack thread here if you're curious for more info: https://datahubspace.slack.com/archives/CUMUWQU66/p1671088121654909

closing this issue for now - please re-open if this issue persists for anyone!

Hi @chriscollins3456, i've just ran into this(most likely) with a datahub-helm deployment, also on GKE, should be the latest version - the img versions are
datahub-actions:v0.0.15 linkedin/datahub-frontend-react:v0.13.0 linkedin/datahub-gms:v0.13.0
it's a completely fresh install - could you please share more about the issue, so i can maybe fix it on my deployment? i'm unable to access the slack thread. cheers!


Type hints doesn't work as expected, because they aren't present in dask source code.

What happened:
I'm trying to make a function which accepts dask.dataframe. Type hints doesn't work as expected however. If I switch back to pandas, then type hints works ok.
What you expected to happen:
No red underlines in vscode indicating errors.
Minimal Complete Verifiable Example:
import dask.dataframe as dd

def func1(
    df: dd.DataFrame
) -> dd.DataFrame:

    df = df.set_index("aaa")

    df = df.rename(columns=lambda x: str(x).strip())

    return df

Anything else we need to know?:
I use vscode and:

It seems that dask (unlike pandas) doesn't have type hints defined, and they are generated by pylance...
Environment:

Dask version: 2022.3.0
Python version: 3.10.4
Operating System: W10
Install method (conda, pip, source): conda

Hi @karolzlot, welcome to Dask, thanks for opening this issue! I'm not able to reproduce this (I'm on a macOS system, but I matched your Python and Dask versions). Could you please share more information about the extensions you're using and the Python Language Server (I'm assuming it's Pylance, but I can't reproduce using Jedi either). It may be a Windows thing?
It also seems like this is more of a usage question than a bug report or feature request. We encourage people with such questions to ask them at the Dask Discourse. Would you mind opening a discussion topic about this over there? If you do so, feel free to close this issue and we will continue the conversation on discourse!

@pavithraes
Yes, I'm using pylance.
I'm not sure why you can't reproduce it.
You need to enable "Use Library Code For Types" setting and then you should be able to see those red underlines.
It's not about usage, it's a feature request. I don't see type hints in source code and this issue is about adding them.

This is a known issue that is being worked on, see https://github.com/dask/dask/pull/8706

+1 for integrating type hints with dask. dask.delayed seems like an obvious target for this; beyond better integration with the increasingly typed python ecosystem, you could parse function annotations to get the information used in the nout keyword argument

Hi @karolzlot, welcome to Dask, thanks for opening this issue! I'm not able to reproduce this (I'm on a macOS system, but I matched your Python and Dask versions). Could you please share more information about the extensions you're using and the Python Language Server (I'm assuming it's Pylance, but I can't reproduce using Jedi either). It may be a Windows thing?
It also seems like this is more of a usage question than a bug report or feature request. We encourage people with such questions to ask them at the Dask Discourse. Would you mind opening a discussion topic about this over there? If you do so, feel free to close this issue and we will continue the conversation on discourse!

Try using vscode:
extensions: Pylance (Microsoft), Mypi Type Checker (Microsoft)
config: Type Checking Mode: strict

Is there a recommendation for how to work within VSCode since there is no intellisense or typing? Or has anyone recently gotten VSCode intellisense to work recently with Dask? Makes it difficult to know what is able to be called without running into runtime errors.
I am a bit confused since there seems to be mypy support https://github.com/dask/dask/pull/8706 but there is no intellisense. Not sure if I am missing something.
Even if I import from the module or make sure pylance uses library code it does not work.
Corresponding pylance issue says https://github.com/microsoft/pylance-release/issues/4801#issuecomment-1708900529

¿Existe alguna recomendación sobre cómo trabajar dentro de VSCode ya que no hay intellisense o typing? ¿O alguien ha conseguido recientemente VSCode intellisense para trabajar recientemente con Dask? Hace que sea difícil saber cómo se puede llamar sin encontrarse con errores de tiempo de ejecución.
Estoy un poco confundido ya que parece haber soporte mypy #8706 pero no hay inteligencia. No estoy seguro si me falta algo.
Incluso si importo desde el módulo o me aseguro de que pylance use código de biblioteca, no funciona.
El problema de pilance correspondiente dice microsoft/pylance-release#4801 (comentario)

You can try this configuration until they release an update:
.vscode/setting.json
{
    "python.analysis.typeCheckingMode": "standard",
    "python.analysis.typeEvaluation.analyzeUnannotatedFunctions": false
}

Use absolute imports for functions imported from dask
from dask.dataframe.core import DataFrame

def func1(df: DataFrame) -> DataFrame:
    df = df.set_index("aaa")
    df = df.rename(columns=lambda x: str(x).strip())
    return df


Minor edits

Hello @Fxe ,
Here are minor edits.

Existing periodic table and MW modules are leveraged.
A few instances of |= are replaced with .update(), which supports older Python versions while performing the same function of concatenating sets.
An unused argument is flagged.
Ambiguous variables are clarified.
An exception file is created to store custom exception classes that are used in various packages.

Andrew

The setup.py, README, and test_data should have been added to the last PR but were somehow missed in my PR.

The setup.py adds imports that are necessary for the new content and adds URLs that will automatically render in the PyPI index.
The test_data includes local import of the the ecoli core model -- since cobra.tests throws a ModuleNotFoundError -- and is bundled with the other ModelSEEDpy content for use with the package in Notebooks or other packages as an example model.

The PR is passing.
Andrew

test data does not go into main
readme should not point to your url
why we need cython as requirement?

Hello @Fxe ,

test_data was relocated to tests
The README paths are corrected to the main repository
cython is used in a number of my forthcoming packages.

cython is used in a number of my forthcoming packages.

That does not explain why you need it

Cython is used to compile Python scripts into C, which can be imported into a Python file and read by the Python interpreter: e.g. PkgWithCython.c is a compile Python package that can be imported normally via import PkgWithCython but it has C speeds. Here is the original publication.
A few of my community and constraint packages are very computational extensive, and thus use Cython to optimize performance without deviating from standard Python compatibility.

I known what cython is for and it is a problem (same issue that pyeda brings)
leave it out for now, and if actually required then we have move it to another repo like modelseedpy-community

What is the problem? Why can cython compiled packages for the various operating systems not be created and stored in the same directory, where the interpreter would find the appropriate format for the user's operating system?

that is something to be tested unless you already have a working demo

I justed commented the cython requirement for the PR, since none of the current packages in MSpy require it.
I will share a demo of cython with various operating systems during this evening.

I discovered that cython does not substantially expedite Optlang operations, so I may not use cython. All references to cython are removed. The PR is passing.
Andrew


website only displays directory list

website only displays directory list

Hello @topcat30
Did you have webserver (eg. apache or nginx), mod_rewrite eabled and php installed and enabled?

@kirilkirkov  Yes, Same issue with me. I have php 7.2 running on namecheap hosting.

@kirilkirkov Yes, Same issue with me. I have php 7.2 running on namecheap hosting.

my domain is jawadadil.com with ssh enabled.
I'm giving this in base url. You can try visiting the URL. it will show you what I mean.
$config['base_url'] = defined('https://jawadadil.com/')?BASE_URL:'';

@adiljay05
When i open your website, i didn't see index.php file. This file will start your shop ready to use. Make sure to add the .htaccess file and index.php file in the root directory of the project. You can also download the platform directly from github as .zip file and then upload it into your server and extract the .zip file there. This will list all needed files.

For the domain - https://jawadadil.com/codeigniter/ you need to edit the $config['base_url'] to  $config['base_url'] = 'https://jawadadil.com/codeigniter/';

Also check the github tutorial for the installation steps - https://github.com/kirilkirkov/Ecommerce-CodeIgniter-Bootstrap , there you can find how to install the online shop to directory and how to install it in the root directory  : )


Use frame timestamps to calculate elapsed time instead of tic & toc

In order to make the re-runs of recorded videos discretely reproducible and easily comparable, the number of seconds of elapsed time since experiment start should be the same whether you're doing a live cam run or running a previously saved video.
Background:
tic and toc were the original method of analysis, but that could not be used to process a saved video because the processing of a saved video is faster than real time.  When the code to add saving of a video was implemented, effort was taken to do it the same way the script was originally coded, with the ability to essentially adjust the frame rate using a delay between captures that could be edited by the user.  Thus, the frame rate in the saved video was made to be manual/100%.  In order to get the time each frame was captured, a separate timestamp file was created because the available formats to save the video (with compression) did not include a timestamp, but rather relied on a static frame rate to determine speed for playback.  To then determine the elapsed number of seconds for the output files during a rerun, the difference of the first & current timestamps is used.  While close, the elapsed times understandably turn out to be slightly different than the tic & toc values from the original live cam run.
Strategy:
The timestamps should more accurately reflect the time the image was captured because of intervening code that is executed between the toc event and the actual capture of the frame from the camera.  So, this issue is intended to be to make the values consistent between the original run and a rerun by using the timestamp from the frames to calculate the number of elapsed seconds in both cases: the live cam run and the re-run with saved video file.
This will ensure that the same number of frames are included between data analysis recordings in the output data files.

New code in place.  Works well.  The elapsed seconds are exactly the same between the live and recorded runs.


Fix hidden focus indicator by not support focus-visible browser

resolve: #156
Browsers that do not support :focus-visible will disable all grouped selectors that contain :focus-visible.

[x] Button
[x] IconButton
[x] LinkButton
[x] Checkbox
[x] DropDown
[x] Radio
[x] TextArea
[x] TextField
[x] ToggleSwitch

See also https://developer.mozilla.org/en-US/docs/Web/CSS/:focus-visible#selectively_showing_the_focus_indicator
custom-button {
  display: inline-block;
  margin: 10px;
}

custom-button:focus {
  /* Provide a fallback style for browsers
     that don't support :focus-visible */
  outline: none;
  background: lightgrey;
}

custom-button:focus:not(:focus-visible) {
  /* Remove the focus indicator on mouse-focus for browsers
     that do support :focus-visible */
  background: transparent;
}

custom-button:focus-visible {
  /* Draw a very noticeable focus style for
     keyboard-focus on browsers that do support
     :focus-visible */
  outline: 4px dashed darkorange;
  background: transparent;
}

@herablog @Dai7Igarashi Please review 🚀

@herablog @Dai7Igarashi Please review 🚀

Radio is not fixed.

Radio is not fixed.

@masuP9 Awesome, thanks. I tested in Safari (and all browsers we are supporting). I will merge and release if @Dai7Igarashi has been checked.

@masuP9 Awesome, thanks. I tested in Safari (and all browsers we are supporting). I will merge and release if @Dai7Igarashi has been checked.


Update BasicTokenSender.sol

Well, there's no need of initializing some 'uint256' variable to 0 because by default all 'uint' variables have a value equal to 0

Closing due to inactivity!


Does not update package-lock.json

Might consider updating the versions for package-lock.json or yarn.lock if they exist.
This could be accomplished with a quick check:

if "package-lock.json" exists, run npm i
if "yarn.lock" exists, run yarn

This way versions of locked dependencies won't be out of sync with published versions.

This isn't automated indeed. For now you can also add the file to the pkgFiles config array (the default is ["package.json"]). Does that work for you?

That works. I hadn't realized there was a configuration option.

Good to hear, @ShMcK.

Can you share your .release-it.json and command? I am using the following but it doesn't seem to be using my config file or adding package-lock.json to the commit.
{
  "non-interactive": true,
  "pkgFiles": ["package.json", "package-lock.json"],
  "verbose": true,
  "dist": {
    "repo": "git@github.com:<myCompany>/<myRepoName>.git",
    "pkgFiles": ["package.json", "package-lock.json"]
  }
}

"release": "release-it -n --no-requireUpstream --no-requireCleanWorkingDir --src.tagName='v%s'"
I tried adding -c and my config file path but it doesn't work. This is running on CI so it's a little hard to debug. I also tried it with just pkgFiles, with just dist/pkgFiles, still doesn't seem like it's using the file config file, or maybe I'm doing something else wrong..

@dinomiike,

Is the config file named .release-it.json?
Is .release-it.json in the root of the project next to package.json?
Is release-it executed from the root of the project?

@webpro thanks for the reply! I had some weird caching issue in my CI that wasn't seeing my updated command. I was able to find a fix for it in my CI settings. Using pkgFiles does exactly what we need. Thank you for creating and maintaining this tool!


link to reporting-bugs in readme, instead of repeating it

The section is prominent enough as presented in our README.md; duplicating it in less visible documents will provide little utility. Merging.


Set up Tiled project

This commit adds a template for levels that can be copied and pasted to make new levels.

Closes #38


SSL Error on publishing az cli universal package

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az artifacts universal publish Extension Name: azure-devops. Version: 0.18.0.
Errors:
Failed to update Universal Packages tooling.
 Error occurred in request., ProxyError: HTTPSConnectionPool(host='vsblob.dev.azure.com', port=443): Max retries exceeded with url: /goutamasarmamylavarapu/_apis/clienttools/ArtifactTool/release?osName=Windows&arch=AMD64&distroName=msys_nt-10.0-18363&distroVersion=3.0.7 (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x04524880>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed')))

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az artifacts universal publish --organization {} --project={} --scope {} --feed {} --name {} --version {} --description {} --path {}

Expected Behavior
Should upload the file provided in the command
Environment Summary
Windows-10-10.0.18362-SP0
Python 3.8.9
Installer: MSI

azure-cli 2.30.0

Extensions:
azure-devops 0.18.0

Additional Context

route to service team


[Fix] 14 Add Catalog Metadata Returns No Error

changed return type of SetMetadataAsync from Task to Task<ActionResult>

Interestingly the method appears to not ProtectCatalogAsync even enter the lambda-function when SetMetadataAsync's return type is Task.

I think the following code block has multiple issues (lines 494 - 505):
var response = ProtectCatalogAsync<object>(catalogId, ensureReadable: true, ensureWritable: false, async catalogContainer =>
{
    var canEdit = AuthUtilities.IsCatalogWritable(catalogId, catalogContainer.Metadata, User);

    if (!canEdit)
        return StatusCode(StatusCodes.Status403Forbidden, $"The current user is not permitted to modify the catalog {catalogId}.");

    await catalogContainer.UpdateMetadataAsync(metadata);

    return new object();

}, cancellationToken);

ensureReadable should be false
ensureWritable should be true
The subsequent lines containing canEdit can be removed because ensureWritable is now true

I think I caused this mess when I introduced the ensureReadable and ensureWritable without properly updating this method

The rest looks fine :-)

I fixed the mentioned issues above (because they are unrelated to this issue) and rebased your changes to new newest commit. I am now running on the new .NET 9 release (its much faster now to compile again :-)) and I cannot reproduce the original issue. So if you cannot reproduce it as well, we might just close this P/R.

Sorry for the chaos, I changed the method implementation to ensure the catalog ID is correct. During that change I had the need to change the return type to Task<ActionResult<object>> which is exactly what you tried to to in this P/R. Because now changes were left now, GitHub automatically closed the P/R.


feat: remove h2 from all tests

What this PR changes/adds
removes all usages of H2 from all test code.
Why it does that
Executing some tests against H2, and others against Postgres has the inherent risk of causing inconsistencies, therefor it was decided (#1839) to remove H2 entirely.
Further notes

test scenarios are defined in the abstract base class for the tests
H2 is still used in the common-sql and transaction modules, as they are truly independent of database technology and simply deal with JDBC.

Linked Issue(s)
Closes #1754
Checklist

[x] added appropriate tests?
[x] performed checkstyle check locally?
[ ] added/updated copyright headers?
[x] documented public classes/methods?
[ ] added/updated relevant documentation?
[x] assigned appropriate label? (exclude from changelog with label no-changelog)
[x] formatted title correctly? (take a look at the CONTRIBUTING and styleguide for details)

Codecov Report

Merging #1841 (62dec9f) into main (9841a4e) will decrease coverage by 7.11%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #1841      +/-   ##
============================================
- Coverage     67.48%   60.36%   -7.12%
- Complexity        0     3163    +3163
============================================
  Files           778      778
  Lines         16602    16602
  Branches       1084     1084
============================================
- Hits          11204    10022    -1182
- Misses         4930     6163    +1233
+ Partials        468      417      -51

Impacted Files
Coverage Δ

...taspaceconnector/sql/assetindex/SqlAssetIndex.java
0.00% <ø> (-80.38%)
:arrow_down:

...nnector/sql/assetindex/schema/AssetStatements.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ion/store/schema/ContractDefinitionStatements.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ore/schema/postgres/ContractDefinitionMapping.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ore/schema/postgres/SelectorExpressionMapping.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ql/assetindex/schema/BaseSqlDialectStatements.java
0.00% <0.00%> (-97.96%)
:arrow_down:

...inition/store/schema/BaseSqlDialectStatements.java
0.00% <0.00%> (-96.56%)
:arrow_down:

...process/store/schema/BaseSqlDialectStatements.java
10.00% <0.00%> (-90.00%)
:arrow_down:

...tor/sql/policy/store/SqlPolicyDefinitionStore.java
0.00% <0.00%> (-89.37%)
:arrow_down:

...ispatcher/sender/MultipartContractOfferSender.java
0.00% <0.00%> (-88.89%)
:arrow_down:

... and 31 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Just a note: at this point these abstract *StoreTest could be used to test in memory and cosmos as well? They seems pretty agnostic and at the end the behavior we want out of the different implementation is the same.

that's correct, then we could dub it a "test specification" and probably the correct location for it would probably be in the same module as the *store itself, right?

Just a note: at this point these abstract *StoreTest could be used to test in memory and cosmos as well? They seems pretty agnostic and at the end the behavior we want out of the different implementation is the same.

that's a fine idea, then we could dub it a "test specification" and probably the correct location for it would probably be in the same module as the *store itself, right? I'll look into that.

I would say so, so everyone that wants to provide an implementation will just need to extend that class and make the implementation work accordingly

Just a note: at this point these abstract *StoreTest could be used to test in memory and cosmos as well? They seems pretty agnostic and at the end the behavior we want out of the different implementation is the same.

that's a fine idea, then we could dub it a "test specification" and probably the correct location for it would probably be in the same module as the *store itself, right? I'll look into that.

I would say so, so everyone that wants to provide an implementation will just need to extend that class and make the implementation work accordingly

Created https://github.com/eclipse-dataspaceconnector/DataSpaceConnector/issues/1845, because this will bring with quite a few changes:

somehow consolidate AssetIndex, AssetLoader and DataAddressResolver
move the abstract test classes into the testFixtures directory of the SPI module
add a dependency onto that
potentially fix quite a few tests
...


doc: Proposal for Samples QA

Description of the change:
Proposal for we improve and ensure the QA of Samples

/lgtm


Adding --yes confirmation flag for velero commands

Have you though about adding this for create/delete commands?
eg.

velero backup create would just output how the backup will be created (prints that --snapshot-volumes is set to true and other important stuff that may cause backup to last longer or do some unwanted actions) - something like 'dry-run'.

Adding --yes to the command would actually execute it.

I think this practice would increase the caution about sensitive commands. It is used in kops and terraform as well.

We do have --confirm for delete commands, but not for creation. I can appreciate the concern, though I'm reticent to change this behavior at this stage because it's changing our primary interface for users. If we did make such a change, it would be a major release such as 2.0.

Seems reasonable 👍


binary/text readers return different type for padded int

Derived from ion-tests/iontestdata/good/equivs/paddedInts.10n.  Binary reader returns an IonLongInt object, while the text reader returns a number.  (maybe this is ok?)
let ion = require("../../dist/commonjs/es6/Ion.js");

let bytes = new Uint8Array([
    0xe0, 0x01, 0x00, 0xea,
    0x2e, 0x90,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7f,
]);
getInt(bytes);

let tw = ion.makeTextWriter();
tw.writeInt(127);
tw.close();
getInt(tw.getBytes());

function getInt(src) {
    let r = ion.makeReader(src);
    r.next();
    let v = r.value();
    console.log(v + ', ' + typeof(v));
}

produces:
127, object         // an IonLongInt
127, number

Verify it is returning a JSBI object from both now.

I've verified that the above code receives a JSBI from both the binary and text reader's value() method (by adding console.log(v) after the shown console.log() statement.  Closing.


go.*,vendor: Bump to k8s v1.20.x

Update the k8s-related dependencies from 1.18.x to 1.20.x.

Hmm looks like openshift/client-go needs to be may need to be bumped as well as I'm seeing missing context parameters and IIRC that was required in 1.18 and not a recent 1.19 or 1.20 development.
/hold

Looks like we're back in business with the openshift/client-go dependency.
/hold cancel

/retest

This is a known error in the upgrade test. I need to dive into why this is happening at some point:
=== CONT  TestMeteringUpgrades/HDFS-OLM-Upgrade/pre-upgrade-testReportingProducesData/node-memory-utilization-runonce
    report.go:124:
        	Error Trace:	report.go:124
        	            				wait.go:211
        	            				wait.go:541
        	            				wait.go:427
        	            				wait.go:421
        	            				report.go:115
        	            				report_dynamic_input_data_test.go:163
        	Error:      	Not equal:
        	            	expected: 200
        	            	actual  : 504
        	Test:       	TestMeteringUpgrades/HDFS-OLM-Upgrade/pre-upgrade-testReportingProducesData/node-memory-utilization-runonce
        	Messages:   	http response status code should be ok

Link to the test artifacts: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/kube-reporting_metering-operator/1497/pull-ci-kube-reporting-metering-operator-master-metering-upgrade-aws/1361537148079050752

/override ci/prow/metering-upgrade-aws


Unify loadFromXml behavior

All loadFromXml function should handled with MediaPackageSerializer.
There are three loadFromXml functions implemented in MediaPackageBuilderImpl.java. Two of them (loadFromXml(String xml) and loadFromXml(Node xml)) will handle media package with registered MediaPackageSerializer. Yet, the third one loadFromXml(InputStream is) handle media package without registered MediaPackageSerializer.
This PR unify the behavior of loadFromXml function to make them all handle media packages with registered MediaPackageSerializer.
This change make a test case in SearchServiceImplTest.java fail. Because it use loadFromXml(InputStream is) to get the resource file. When it use DefaultMediaPackageSerialzerImpl to handle media packages, the url of the resource is translated to an absolute path which can not be accessed by getClass().getResourceAsStream(). So, I modify the test case, use FileInputStream to handle the absolute path.
Your pull request should…

[ ] have a concise title
[ ] close an accompanying issue if one exists
[ ] be against the correct branch
[ ] include migration scripts and documentation, if appropriate
[ ] pass automated tests
[ ] have a clean commit history
[ ] have proper commit messages (title and body) for all commits

I'm curious for the reasoning here.
Right now, two of the three cases use
MediaPackageImpl.valueOf(InputStream);

Yet, you chose to use the third case which – apart from two test usages – right now is used only at a single place in all of Opencast's code in handling OAI-PMH since that one is dealing with XML structures internally anyway. So, why go with the most complex but least tested and mostly unused case? I was actually leaning the other way and wanted to remove loadFromXml(Node xml).

Here is the reason in my case.

I use LTI tool to integrate Opencast to Canvas LMS. And, use PresignedUrlMediaPackageSerializer to transfer the presigned S3 URL (for videos and images) to client.
LTI tool use SearchRestService.getEpisode to get the episodes' information. And following is the call stack

        at org.opencastproject.mediapackage.MediaPackageBuilderImpl.loadFromXml(MediaPackageBuilderImpl.java:106)
        at org.opencastproject.mediapackage.MediaPackageBuilderImpl.loadFromXml(MediaPackageBuilderImpl.java:138)
        at org.opencastproject.search.impl.solr.SolrRequester$1.getMediaPackage(SolrRequester.java:204)
        at org.opencastproject.search.api.SearchResultItemImpl.fill(SearchResultItemImpl.java:710)
        at org.opencastproject.search.impl.solr.SolrRequester.createSearchResult(SolrRequester.java:177)
        at org.opencastproject.search.impl.solr.SolrRequester.getForRead(SolrRequester.java:836)
        at org.opencastproject.search.impl.SearchServiceImpl.getByQuery(SearchServiceImpl.java:505)
        at org.opencastproject.search.endpoint.SearchRestService.getEpisode(SearchRestService.java:499)

So, if loadFromXml does not handle media packages with PresignedUrlMediaPackageSerializer, the client will receive an unsigned URL (or, a raw URL) which cannot be opened.

I'm reviewing this code right now, because also I modified the exact same part in #3058. From my tests, your changes in the test script help me as well.

@Gao-Jun Since i merged #3058 I checked your solution vs @mliradelc's.  They look (basically) the same, but his is properly rewriting the URLs.  I'm going to close this off, feel free to refile with an updated patch if you think we've missed something!


implemented secrets for prombench and updated the readme

closes https://github.com/prometheus/prombench/issues/37
Signed-off-by: Krasi Georgiev<EMAIL_ADDRESS>

lgtm


Added command: Rebalance

Added Rebalance Command
Added incoming and outgoing args to:
QueryRoutes
FindRoutes
FindPaths
FindPath

Requires https://github.com/lightningnetwork/lnd/pull/1751 to be in first.  (All changes in 1751 are included here.)

Solves: https://github.com/lightningnetwork/lnd/issues/1603

I tried to compile it but I get an error
(38/43) Wrote golang.org/x/net@ae89d30ce0c63142b652837da33d782e2b0a9b25
(39/43) Wrote golang.org/x/crypto@49796115aa4b964c318aad4f3084fdb41e9aa067
(40/43) Wrote github.com/ltcsuite/ltcd@cdab10132e8c6e4a3ffd112dba54791946d28906
(41/43) Wrote github.com/btcsuite/btcd@f899737d7f2764dc13e4d01ff00108ec58f766a9
(42/43) Wrote google.golang.org/genproto@df60624c1e9b9d2973e889c7a1cff73155da81c4
(43/43) Wrote<EMAIL_ADDRESS> Building debug lnd and lncli.
go build -v -tags=debug -o lnd-debug -ldflags "-X main.Commit=08673651129a15aff3c46b5cb080d45fd4990bcc" github.com/lightningnetwork/lnd
github.com/lightningnetwork/lnd/chainntnfs
github.com/lightningnetwork/lnd/keychain
# github.com/lightningnetwork/lnd/keychain
../../lightningnetwork/lnd/keychain/btcwallet.go:181:25: addr.DerivationInfo undefined (type waddrmgr.ManagedPubKeyAddress has no field or method DerivationInfo)
github.com/lightningnetwork/lnd/ticker
github.com/lightningnetwork/lnd/lnrpc
github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify
github.com/lightningnetwork/lnd/chainntnfs/btcdnotify
github.com/lightningnetwork/lnd/chainntnfs/neutrinonotify
# github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify
../../lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:90:21: undefined: chain.BitcoindConn
../../lightningnetwork/lnd/chainntnfs/bitcoindnotify/driver.go:18:29: undefined: chain.BitcoindConn
Makefile:132: recipe for target 'build' failed
make: *** [build] Error 2

It compiles just fine on Travis.
Those errors aren't related to the code changes in this PR.  Looks like you don't have the correct dependencies?

I get it directly from the branch Rebalance on your lnd fork, I just ran make and make install before, on the same box I had running lnd f0f5e11b826e020c11c37343bcbaf9725627378b without any problem, in another directory, not related with your fork
What can it be? I would like to try this feature

The make file on my branch doesn't point to it's own branch.  It points to LightningNetwork.
I'm unfamiliar with GIT, and couldn't get the my local copy of my fork to work right, because it kept trying to use LND's generated proto's instead of mine.  So, what I do is work on a local copy of LND, compile/test, then just move the changes to my branch and upload.
All I can suggest is: get a copy of LND that at the same commit level as my branch, and copy over the changed files.
Probably a horrid way of doing it... but it's the best (only) solution I can think of.

Rebalancing can be done by external scripts together with SendToRoute


Fixes #615 : Clears warnings for SdlProxyBase

Fixes #615

[x] Suppressed some warnings that were unavoidable due to the nature of it being a library project.
[x] Updated some code style.
[x] Removed unnecessary code and members.
[x] Fixed all javadoc issues including missing entries

Codecov Report

Merging #617 into develop will increase coverage by 0.11%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             develop     #617      +/-   ##
=============================================
+ Coverage      40.42%   40.54%   +0.11%
  Complexity      2399     2399
=============================================
  Files            331      331
  Lines          14567    14517      -50
  Branches        1538     1525      -13
=============================================
- Hits            5889     5886       -3
+ Misses          8433     8387      -46
+ Partials         245      244       -1

Impacted Files
Coverage Δ
Complexity Δ

...n/java/com/smartdevicelink/proxy/SdlProxyBase.java
6.09% <ø> (ø)
6 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39c48a2...e78686d. Read the comment docs.


Caching dynamically compressed files

Describe the feature you'd like to add to nginx
Storing compressed files returned by on-the-fly brotli filter in cache path.
Describe the problem this feature solves
This is one of the remaining areas where nginx falls short and can’t match Apache httpd’s offering at present. Whereas Apache caches and serves compressed files, nginx only saves the uncompressed response and requires on-the-fly compression for each subsequent request (or else manual compression beforehand).
This also makes SSL_sendfile() much less effective, having to be disabled whenever such repeated compression happens.
Additional context
Previous request: https://forum.nginx.org/read.php?11,283440,283440

Actually this can be achieved by NGINX alone.
You can use proxy_cache (set up a "front-end" caching and "backend" PHP/compression NGINX configurations) and then configure your "front-end" proxy cache key to be based on normalized Accept-Encoding header. Roughly (untested, consider as more of a pseudo-code to begin with):
# front-end with caching
server {
    …
    # listen on actual server ports
    location / {
        proxy_pass http://backend;
        proxy_cache myZone;
        gzip off;
        brotli off;
        # normalized_http_accept_encoding has to be created via `map` to ensure `Accept-Encoding:` values are normalized
        # to just the three possible variants like `Accept-Encoding: gzip`, `Accept-Encoding: br` and an empty one.
        proxy_cache_key "$scheme$request_method$host$request_uri $normalized_http_accept_encoding";
        # for consistency with normalized encoding, instruct backend to use it
        proxy_set_header Accept-Encoding "$normalized_http_accept_encoding";
    }
}
# backend with PHP and compression
server {
    …
    # listen on designated private ports
    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php7.3-fpm.sock;
        ...
    }
    gzip on;
    brotli on;
}

This of course, will introduce a bit of latency for additional communication between NGINX "instances", although it may be negligible on the same server and provided the benefits outweighs it.
Introducing direct support for this in NGINX is highly unlikely as the order of body filters like gzip is more or less set to be after fastcgi cache already did its job. Plus an issue with normalization of Accept-Encoding header as mentioned in the linked discussion would make it never work out of the box anyway.


Include the whole initial line in the exception message when failing …

…to create a message from it.
 To make it easier to debug we should include the whole initial line in this case in the exception message.
Modifications:
Wrap thrown IllegalArgumentException in another IllegalArgumentException and include the full intial line in the message.
Result:
Easier to debug HTTP decoders.

I think it would be good to use a specific exception for this and not have the initialLine value part of the message that will be printed by the exception when toString() is called. If the Netty user does not take proper care, this can result in being a log injection vector.

That may be a good idea… that said it should be a sub-class of IllegalArgumentException just in case people depend on the type of it.
WDYT ?

On 15. Oct 2018, at 17:54, Julien Viet<EMAIL_ADDRESS>wrote:
I think it would be good to use a specific exception for this and not have the initialLine value part of the message that will be printed by the exception when toString() is called. If the Netty user does not take proper care, this can result in being a log injection vector.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub https://github.com/netty/netty/pull/8383#issuecomment-429910553, or mute the thread https://github.com/notifications/unsubscribe-auth/AAa0QnngpuEgs3fDuSthPhX8uQpAvjslks5ulK--gaJpZM4Xb9CT.

no problem with me, the main issue is to have the initial line in the toString() message

Whatever the decision is, I think it remains important to signal this exception so the user can handle it specially and not print the message.

@netty-bot test this please

Let me just close this for now...


Create data container that relates musicXML, EasyScoreMessage, and Vexflow objects

Once MusicXML is rendered as Vexflow objects, it is difficult to relate them back to the musicXML they were derived from. We need a way to let the caller track this or have a opinionated container for relating these objects.

This will be really good.

@jaredjj3 let me know when this is stable and I should go for the next missing feature.

It won't be stable for a while. The way you can start moving on this now is:

define the VexmlMessage types you need
update LegacyProducer as needed to produce the types you defined
update whatever rendering machinery you need

The LegacyProducer is still the producer that's hooked up to the dev app and the new producer system I'm implementing is completely decoupled from it. So you're good to make changes as you need.

After refactoring, I don't think this makes sense to do in this library. Relating vexflow objects to MusicXML would mean we'd have to unravel some of the abstractions that were done to make rendering easier.
I think vexml should remain solely as a rendering library and editing capabilities should be done outside of it.