Build side cannot be reverted

Hi,
We found for hash join in some cases the build side had much larger size than the stream side, which would limit the join performance. For example, a left outer join is with right build side but the right side has larger datasize than the left side. In this case, if we can build the left side, the performance could be improved. But the issue is not all engines support both left and right build side for left outer join. Looks like a different logic is needed to support left outer join with left build side to have the correct output.
For now, if we just change left outer join to be right outer join (which builds left), correct result cannot be got unless we revert the left and right tables at the same time. But in this case, the build side would still be larger than the stream side and the performance issue cannot be resolved.
We also found in Velox if we reverted the build side for left outer join, the result was not correct. But for full outer join, correct result can be got with build side reverted. This is the Velox test I'm testing on if you want to take a look: Velox test.
@oerling @mbasmanova Glad to hear your opinoin on this issue, thanks!

cc @FelixYBW

The wrong seq we noted from Gluten, in all cases below, right table is targe table. We can modify spark to get the correct build table, question is how Velox can support this.
Q4, stage 67     LeftSemi, BuildRight
Q13, stage 362   LeftOuter, BuildRight
Q21, stage 522   LeftSemi, BuildRight
Q22, stage 553  LeftAnti, BuildRight

@rui-mo Rui, thank you for reaching out. I expect a LEFT OUTER JOIN b to be equivalent to b RIGHT OUTER JOIN a. Hence, if b is smaller, the optimizer can plan the left join. Otherwise, the right join. Maybe there is a bug in the right join implementation in Velox. I'll take a look at the tests you linked.

@rui-mo A quick look at the leftJoinBuildSideRevert test suggests that the test is comparing a LEFT b with b LEFT a. These are not expected to be the same. Let's change the test to compare a LEFT b with b RIGHT a.

a LEFT OUTER JOIN b to be equivalent to b RIGHT OUTER JOIN a

Hi Masha, maybe I did not decribe the issue accurately. We agree above sentence but the build and probe tables should be reverted, right? For example, left outer with a as build table and b as probe table equals to right outer with b as build table and a as probe table. I think this case works well in Velox.
But our issue is about left outer with b as build table and a as probe table. You can see the build side is reverted but the join type is still the same. In this case, if we still want the correct result, the join logic is different.

@rui-mo In Velox, right side of the join is always the build side. Hence, a LEFT b creates hash table from b, while b RIGHT a creates hash table from a. Since there are equivalent, the optimizer can pick which one to plan based on the sizes of a and b. If b is small, then a LEFT b. Otherwise, b RIGHT a.

@rui-mo A quick look at the leftJoinBuildSideRevert test suggests that the test is comparing a LEFT b with b LEFT a. These are not expected to be the same. Let's change the test to compare a LEFT b with b RIGHT a.

I guess that is because build right is default behavior. Ideally, there may be build side options. For left outer join, build left or right can have the same output.

For left outer join, build left or right can have the same output.

True, but since a LEFT b == b RIGHT a, there is no need to provide an option of the left join with "left side being the build side".

@rui-mo I modified the leftJoinBuildSideRevert test to change the join type for the second query to kRight and the test passed.

The optimizer can pick which one to plan based on the sizes of a and b. If b is small, then a LEFT b. Otherwise, b RIGHT a.

Actually the issue we met was that the optimizer planned an a LEFT b logic, but b is larger. So we want to change the build side to a, if we change it to be b Right a, then because right outer builds left, b is still the build side. Therefore, with this change the build side is still the larger side.

because right outer builds left, b is still the build side

Why do you say this? In Velox, b RIGHT a uses table a to build the hash table.

In Velox, right side of the join is always the build side. Hence, a LEFT b creates hash table from b, while b RIGHT a creates hash table from a.

Thank you Masha! I think this is the key point I was missing. Based on this, it would be feasible to change the build table by justing switching the tables and the join type. I think we could close this issue now, how do you think so? @FelixYBW

@rui-mo Happy to hear the issue is resolved. I'm going to close, but feel free to reopen if there are any follow-up questions.

Can leftsemi and leftanti work as well?

Semi and anti joins are tricker. Velox has LEFT and RIGHT SEMI joins, but only one NULL AWARE ANTI JOIN. @Yuhta is working on adding regular ANTI JOIN. We may need to look into adding support for both LEFT and RIGHT anti joins.

Do you mean one null aware semi join?

@FelixYBW
There is only one kind of SEMI join. Having both kLeftSemi and kRightSemi allows us to choose which table to load into a hash table.
However, there are 2 different ANTI joins: NULL AWARE (NOT IN) and regular (NOT EXISTS). This is explained in more details in https://facebookincubator.github.io/velox/develop/anti-join.html
Currently we only support NULL AWARE ANTI join with subquery being loaded into a hash table. We are working on adding support for regular anti join with subquery being loaded into a hash table. On top of that, there might be a need to implement both joins in a way that allow for the outer query to be loaded into a hash table. That work hasn't been planned yet.

Thank you, Masha. Then we can fix Q4, Q13 and Q21.
Anti join isn't urgent now. Anti join in Q22 is a very small one.


nil_panic prevents inject/reduce from raising an exception

DRGTK arithmetic operators couldn't possibly be any more fake:
-> [1,2].inject(:+)
* EXCEPTION: * ERROR:
** method_missing: +
** instance:       1
** on class:       Fixnum
** with modules:   [Integral, NumericDeprecated, ValueType, Comparable, Kernel]
** with values:    1
** given args:     [[2]]
** exception:      can't define singleton

Log above is without nil_panic.
With nil_panic required — no exception is thrown and Array#Inject/Array#Reduce silently returns nil.
This is not desired behavior.
Block overload still works with and without nil_panic.
Examples:
[4, 20].reduce{|sum, summand| sum + summand}
array.inject{|product, factor| product + factor}

I think I remember reading that AriThMetic OpeRaToRs Are A Lie in mruby

Hrrgm…
Not like I'm picking this back up but where was I with it..?
Think I remember some changes to DRGTK that made this issue obsolete but don't quote me on it.


feat: introduce validation for deprecated resource apiVersions [WIP]

WIP...
This PR fixes #356.
Changes

Introduced K8S002 for validating resources apiVersion as part of kubernetes-schema validator.
Since kubernetes-schema code was really focused on a single rule it required some adjustments in plugin internals. I tried to keep it minimal, but it feels like it might need more refactoring and better separation in the future.
I used custom JSON object to store deprecation data.

As I mentioned in https://github.com/kubeshop/monokle-core/issues/356#issuecomment-1569848792, maybe it can be a part of existing schemas somehow.
Even if not, keeping it as a part of the bundle would require releasing new plugin version each time new K8s version is released (every 6 months AFAIR). OTOH it's quite convenient and doesn't require querying for external files.
Also I think it's worth keeping all the metadata so we can guide users on how to update deprecated apiVersion.

Fixes

None.

Checklist

[x] tested locally
[ ] added new dependencies
[ ] updated the docs
[x] added a test

Well, this was fun to gather all the deprecations information. As I mentioned in https://github.com/kubeshop/monokle-core/issues/356#issuecomment-1569848792, the deprecation guide includes info only on removed APIs. So to gather deprecation info I wenth through K8s versions changelogs - https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG and...:

Most of the information was in Deprecated section but there were cases where it was in different places. So I ended up got with Ctrl+F on deprecat keyword to find all.
In some versions there was also information on removals not included in "deprecation guide". Not sure why such inconsistencies exists but I included it to.
Sometime the same info was duplicated in couple of versions (like v1.16, v1.17a dn v1.18) all mentioning the same deprecations/removal of "apps/v1beta2". In such case I would go with lowest version.
Sometime the info was quite short so I have to lookup in other places for kinds it relates too.
I also found https://kubernetes.io/docs/reference/scheduling/config/ which mentions deprecations of scheduling API not mentioned in other places 🤔🤷♂

At this stage the data we have seems to be quite complete. This https://github.com/kubeshop/monokle-core/pull/372/commits/f43021533383bd26ad764e9334ae2125bb5718ee commit has raw data right after I gathered everything. And the next one https://github.com/kubeshop/monokle-core/pull/372/commits/bd497d706cf76a9f5ed1fda364abebfe61b5f6bd is end result, cleaned up, sorted and deduplicated.


fix: issue of "fetch is not defined" when using docker

Because fetch does not exist in early version of node (like 14.x)
Then if running up by docker compose, we see errors below,
Application Error
ReferenceError: fetch is not defined
    at Note (/opt/notes-app/src/Note.js:36:24)
    at attemptResolveElement (/opt/notes-app/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack-server.node.unbundled.development.js:1597:18)
    at resolveModelToJSON (/opt/notes-app/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack-server.node.unbundled.development.js:1883:21)
    at Object.toJSON (/opt/notes-app/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack-server.node.unbundled.development.js:1421:14)
    at stringify (<anonymous>)
    at processModelChunk (/opt/notes-app/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack-server.node.unbundled.development.js:258:14)
    at retryTask (/opt/notes-app/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack-server.node.unbundled.development.js:2166:26)
    at performWork (/opt/notes-app/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack-server.node.unbundled.development.js:2213:7)
    at async_hooks.js:313:14
    at AsyncResource.runInAsyncScope (async_hooks.js:197:9)
    at AsyncLocalStorage.run (async_hooks.js:311:35)
    at Immediate.<anonymous> (/opt/notes-app/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack-server.node.unbundled.development.js:2313:29)
    at processImmediate (internal/timers.js:461:21)
    at process.callbackTrampoline (internal/async_hooks.js:131:14)

thanks but we assume latest node anyway so i’ll close this and merge the other one


Remove-AzureADServiceAppRoleAssignment is not working as expected / is still using old API

Hello,
trying to use this cmdlet I noticed that it isn´t doing what it´s supposed to in our environment. Instead of removing the service app role assignment, it just removes the granted consent, the permission however remains on the app and users that have permission for example can still give new consent. Is this intended? And if yes, what is the correct command to remove the app role assignment instead?
I also noticed that it is still using graph.windows.net in the background which is not only deprecated for two years, it´s also end of life in about 4 months so this might be related? I am sure that is not the only command that still uses it so I am wondering what the future of this module is?
Best,
Mario

Hello @magichappens89 , this cmdlet removes the assignment of a certain app role from either a user, group or service principal. The role itself remains defined on the resource but the assignment to either a user, group or service principal is removed and you reassign it afresh. Let me know if this is what you also want to remove.
You are right. This cmdlet still calls the Azure AD Graph endpoint. We are currently working on official communication on the way forward with the deprecation. We however recommend that you start migrating to Microsoft Graph PowerShell which is where all efforts are focused in order to achieve parity with the Azure AD PowerShell.

Hello @msewaweru ,
I think it actually manages service principal assignment. For the assignment of users and groups there are other cmdlets. What I want is that the app permissions assignment is removed (as indicated by the cmdlet name). However it just removes admin consent if there was any.

Based on the architecture I guess this is rather something that needs to be done on the app registration but the module lacks cmdlets for this.
It seems that there is no command of archiving this, also not in the Microsoft Graph PowerShell module. Hope you can advise how to actually remove it.

@psignoret , could you help with this query.
@magichappens89 , there could be an extension for users to continue using Azure AD PowerShell for sometime and this communication will be going out to you as soon as it is finalized.

@msewaweru, there could? Any ETA when this communication will be finalized? I am a bit confused. I mean the Azure AD PowerShell was "the" module to administer Azure AD with PowerShell. A lot of the commands stop working in 4 months because of the end of life I guess. And there is still yet no communication or whatsoever how to migrate etc?

@magichappens89, the ETA for his communication should be March and that will be to say that the cmdlets will not stop working in 4 months. The plan is to give customers more time to migrate by providing the migration tools and guidance to Microsoft Graph PowerShell.

@msewaweru but how the modules that use Azure AD graph can still work after the API will be end of life?

@magichappens89
Regarding the API:
I suggest you consider the API used by this module to be mostly an implementation detail. If Microsoft says the module is still supported, then it doesn't really matter if the API the module relies on is supported for use by your own applications. That said, you should be actively working on switching to Microsoft Graph PowerShell, and you should not write any new scripts using Azure AD PowerShell.
Regarding the Remove-AzureADServiceAppRoleAssignment cmdlet:
This cmdlet removes app role assignments, but it sounds like this isn't what you expect it to do. Can you clarify what you expected this cmdlet to do?
An app role assignment is an association between a service principal representing an app or a service (the "resource"), and a user, a group, or a service principal for another app or service (the "principal"), representing authorization granted for the principal to access the resource. App roles which can be assigned to service principals, are sometimes called "application permissions".
The table from your screenshot (from Azure portal > Azure AD > App registrations > API permissions) includes two bits of information:

The list of API permissions (delegated permissions and app roles) the application is configured to require. This is just a "wishlist" of the permissions you'd like the application to have, primarily used to inform consent experiences. If you want to remove a configured permission, you would update the RequiredResourceAccess property of the Application object (i.e. the app registration). Here's an example:
# Remove required permissions for Microsoft Graph from required API permission, keep the rest
$myAppId = "{my-app-id}"
$graphAppId = "00000003-0000-0000-c000-000000000000"
$app = Get-MgApplication -Filter "appId eq '$($myAppId)'"
$newRra = @(,$app.RequiredResourceAccess | ? { $_.ResourceAppId -ne $graphAppId })
Set-MgApplication -ApplicationId $app.Id -RequiredResourceAccess $newRra

Keep in mind, that removing a permission from this list does not revoke a permission grant. If you remove a required permission which was granted, it will still be granted.

The list of API permissions which have been granted to the application. These are the tenant-wide delegated permission grants and app role assignments which have actually been granted to the application. To remove a delegated permission grant, you would use Remove-MgOAuth2PermissionGrant. To remove an app role assignment you would use Remove-MgServicePrincipalAppRoleAssignedTo. Here's an example:
# Remove all Microsoft Graph app role assignments granted to this app
$myAppId = "{my-app-id}"
$graphAppId = "00000003-0000-0000-c000-000000000000"
$sp = Get-MgServicePrincipal -Filter "appId eq '$($myAppId)'"
$graph = Get-MgServicePrincipal -Filter "appId eq '$($graphAppId)'"
Get-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $sp.Id -Filter "resourceId eq $($graph.Id)" | % {
    Remove-MgServicePrincipalAppRoleAssignedTo -ServicePrincipalId $graph.Id -AppRoleAssignmentId $_.Id
}

The examples above are using Microsoft Graph PowerShell because that's what you should be switching to. Here are the equivalent examples, using Azure AD PowerShell:
# Remove required permissions for Microsoft Graph from required API permission, keep the rest
$myAppId = "{my-app-id}"
$graphAppId = "00000003-0000-0000-c000-000000000000"
$app = Get-AzureADApplication -Filter "appId eq '$($myAppId)'"
$newRra = @(,$app.RequiredResourceAccess | ? { $_.ResourceAppId -ne $graphAppId })
Set-AzureADApplication -ObjectId $app.ObjectId -RequiredResourceAccess $newRra

# Remove all Microsoft Graph app role assignments granted for this app
$myAppId = "{my-app-id}"
$graphAppId = "00000003-0000-0000-c000-000000000000"
$sp = Get-AzureADServicePrincipal -Filter "appId eq '$($myAppId)'"
$graph = Get-AzureADServicePrincipal -Filter "appId eq '$($graphAppId)'"
Get-AzureADServiceAppRoleAssignedTo -ObjectId $sp.ObjectId | ? { $_.ResourceId -eq $graph.ObjectId } | % {
    Remove-AzureADServiceAppRoleAssignment -ObjectId $graph.ObjectId -AppRoleAssignmentId $_.ObjectId
}

Note: Use of Get-AzureADServiceAppRoleAssignedTo in the example above assumes you connected with the default use of Connect-AzureAD, without connecting with a customer-owned app or service.

@psignoret thank you for that detailed answer. That confirms the workaround I assumed working as well. I was just trying to find a command to remove the application permission without changing the required resource access object and Remove-AzureADServiceAppRoleAssignment cmdlet seemed the right choice for me. However, running this only removes the admin consent granted for that assignment.
Before

After

I understood that to remove the actual resource permission I need Set-AzureADApplication. I will take a look at Microsoft Graph PowerShell module as well. Thank you.

@psignoret I just tried out the Microsoft Graph PowerShell commands you provided. Remove all Microsoft Graph app role assignments granted to this app works for me. Although I found "assignments" in this context confusing as the assignments are the actual permissions for me in this context.
Set-MgApplication did not work as there is no cmdlet with that name. Also tried it with Update-MgApplication  but that runs into an error.
Cannot process argument transformation on parameter 'RequiredResourceAccess'. Cannot convert the  "Microsoft.Graph.PowerShell.Models.IMicrosoftGraphRequiredResourceAccess[]" value of type "Microsoft.Graph.PowerShell.Models.IMicrosoftGraphRequiredResourceAccess[]" to type "Microsoft.Graph.PowerShell.Models.IMicrosoftGraphRequiredResourceAccess".

unsubscribe
On Tue, Feb 22, 2022 at 4:21 AM Eunice Waweru @.***>
wrote:

Hello @magichappens89 https://github.com/magichappens89 , this cmdlet
removes the assignment of a certain app role from either a user, group or
service principal. The role itself remains defined on the resource but the
assignment to either a user, group or service principal is removed and you
reassign it afresh. Let me know if this is what you also want to remove.
You are right. This cmdlet still calls the Azure AD Graph endpoint. We are
currently working on official communication on the way forward with the
deprecation. We however recommend that you start migrating to Microsoft
Graph PowerShell
https://docs.microsoft.com/en-us/powershell/microsoftgraph/overview?view=graph-powershell-1.0
which is where all efforts are focused in order to achieve parity with the
Azure AD PowerShell.
—
Reply to this email directly, view it on GitHub
https://github.com/Azure/azure-docs-powershell-azuread/issues/782#issuecomment-1047737224,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEX3LVF6PBDXCDSHAIGVSITU4N5UNANCNFSM5O7BU5MQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

@Jackson-Woods , are you able to help with this query?

@magic-happenz, please see the latest updates on the deprecations plans here.


Add VCollectionHeader component

Fixes

Fixes #2729 by @obulat
Description

This PR adds the component that displays the collection icon, its title, external link (if available) and the number of results in the collection.
This component is not used anywhere yet, so you can only see it in the Storybook.
Testing Instructions

Check out the rendered Story. It should match the Figma designs ( creator desktop and mobile, source desktop and mobile, tag desktop and mobile).
Clicking on the external link button sends an analytics event, but since the build is "production", you cannot see the event details logged in the console, only the warning that Plausible was disabled.
Checklist

[ ] My pull request has a descriptive title (not a vague title likeUpdate index.md).
[ ] My pull request targets the default branch of the repository (main) or a parent feature branch.
[ ] My commit messages follow best practices.
[ ] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[ ] I tried running the project locally and verified that there are no visible errors.
[ ] I ran the DAG documentation generator (if applicable).

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

10000 images from <source name>

This wording also doesn't solve the problem of i18n: some sources need the, some don't. We should try to find a way to not have the source name part of the sentence. Something like "Source name: 10000 images"?

It looks great. Just two comments:

For the three variants, the horizontal layout in md is correct, but h1 should follow the same style as in xs / sm
In Source, the results sentence should say 10000 images provided by this source instead of 10000 images from <source name>. This was thought to avoid i18n conflicts.

I updated the h1 style to change after the lg breakpoint. As for the result sentences, those should be passed as props by the parent component, so we should update them in the PR that creates the page for the additional search views.

@fcoveram, I reverted the breakpoint change for the h1 style (it's md: now instead of lg:) and added the top margin for breakpoints above md. I also updated the snapshot tests so that you can see all of the breakpoints, not only 2.
As for the texts, I updated the creator results label according to the suggestion, but they will also be updated in the PR that fixes #2774

@obulat I was waiting for a final verdict on the sub-heading templating here: https://github.com/WordPress/openverse/pull/2734#discussion_r1302279123
Can you confirm the plan? I'll review on Monday if this PR is ready to go, but then I think we'll need a new issue to record the changes/complexities of that sub-heading.

Can you confirm the plan? I'll review on Monday if this PR is ready to go, but then I think we'll need a new issue to record the changes/complexities of that sub-heading.

Thank you, @sarayourfriend . The plan is to move the discussion of the changes/complexities of the sub-heading to #2774 that will create the subheading, translate it and pass as a prop to VCollectionHeader.

Thanks for the suggestion @sarayourfriend; that section kept me iterating different ideas a lot. Unfortunately, one spacing solution doesn't work for horizontal and vertical layouts.
But in the future, we can try showing the view name next to the icon on xs to break down that imbalance. Here is a mockup.


New release for byteorder dependency

Hi!  It would be great if a new release could be published for the byteorder dependency bump that already happened.  This crate is the only reason we get a duplicate byteorder in our dependencies.
thanks!

Oh, sure, I can do that! Thanks for the info!

I just released 0.1.6. Hope that helps!

thanks!


Using "TimeGenerated 15m" and Evaluation Period of 15 minutes does not generate alerts

There seems to be a problem with the logic used in the examples, or at least it's a bad example using an evaluation period of 15 minutes with 3 consecutive failures.
TimeGenerated 15m in the queries, creates a single data point every 15 minutes.
Consecutive breaches of 3 in a 15 minute period will never happen with only 1 data point.
Even consecutive breaches of 1 in a 15 minute period seems to not work, I guess because there is very little overlap.
To get these examples to work, I reduced the TimeGenerated value to 5m and set consective breaches to 2.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 76418778-d604-5863-8580-d1dff231d7c4
Version Independent ID: 01beca87-9978-6d85-4ee4-6ddc1fc8a884
Content: Alerts from Azure Monitor for VMs - Azure Monitor
Content Source: articles/azure-monitor/insights/vminsights-alerts.md
Service: azure-monitor
Sub-service: ****
GitHub Login: @bwren
Microsoft Alias: bwren

@DataBitz  Thanks for the question! We are investigating and will update you shortly.

@DataBitz I have tried to reproduce the issue in multiple different environments and the observation is not always particular TimeGenerated value and consecutive breaches work because it would depend on how often the InsightsMetrics table is fetching the data for Origin vm.azm.ms. However, assigning @bwren for visibility our your feedback, review it and to help provide additional insights or consider enhancing the document with a note or more clarification around it. Thanks.

#label:"CreateAzMonWIFeb2021"

Thank you for your contribution to the Azure Monitor documentation.  We have created a workitem for this issue in our internal backlog database.  We will prioritize it and make any necessary changes as we are able.
#please-close


Fix crash caused by uninitialized variable

Problem
What is being fixed?

Fix crash in DeviceController::Shutdown()
Fixes #7188 chip-device-ctrl always crashes in DeviceController::Shutdown()

Change overview
This PR fixes crash caused by uninitialized member variable. The crash happened in DeviceController::Shutdown().
Testing
How was this tested? (at least one bullet point required)

launch chip-device-ctrl
commission accessory e.g. connect -ble 3840 12345678 12344321
optionally enter any other commands
issue exit command

Result without the fix: crash with segmentation fault on the following place in src/controller/CHIPDeviceController .cpp DeviceController::Shutdown()
chip::Platform::Delete(mDefaultIMDelegate);
Result with the fix: application tears down correctly and exits.

@bluebin14 - could you try making a change and re-updating this PR? Restyler seems stuck.
maybe the change could be to add more nullptr initializations as in my comment.

@bluebin14 Needs rebase to tip to fix the Darwin failure...


Image.Create() creates an image with size 0

Godot version
4.0.beta4.official.e6751549c
System information
Windows 10, NVIDIA GeForce RTX 3060
Issue description
I create an image and use it as texture. In v4.0.beta3.official.01ae26d31 it worked perfect, and in the latest Beta it produces a image with 0/0 dimension.
These are the results of the code:
--- Debugging process started ---
Godot Engine v4.0.beta3.official.01ae26d31 - https://godotengine.org
Vulkan API 1.2.0 - Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 3060
Hight: 1080 Width: 1920
--- Debugging process stopped ---
--- Debugging process started ---
Godot Engine v4.0.beta4.official.e6751549c - https://godotengine.org
Vulkan API 1.2.0 - Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 3060
Hight: 0 Width: 0
--- Debugging process stopped ---
Steps to reproduce
func _ready():
var w = 1920
var h = 1080
var img = Image.new()
img.create(w, h, false, Image.FORMAT_RGBA8)
print("Hight: ", img.get_height(), " Width: ", img.get_width())
Minimal reproduction project
No response

Image.create() is static in 4.0:
var img = Image.create(w, h, false, Image.FORMAT_RGBA8)

Closing per above comment.


[Storage] Fix browser tests in storage-blob - feature/storage

Would be nice to add a brief description of the issue which can help others understand this change.


Implement dying for players

Players can die. This happens from one of two things:

The health value reaching zero from attacks by other players;
The hunger value reaching a threshold (hunger increases a bit each turn).

Players can already die as a direct consequence of implementing hunger mechanics in #8. Closing this as outdated.


How to change runtime node.js version in next.js SSR

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting questions.
[X] I have removed any sensitive information from my code snippets and submission.

App Id
d3akk1nl18rszx
AWS Region
us-east-1
Amplify Hosting feature
Build settings, Custom build image
Question
I changed the Node.js version in the build settings for both Amazon Linux 2 and a custom build image. In both cases, I successfully changed the build Node.js version. However, when I checked the logs in CloudWatch, it showed that the Lambda function was running with Node.js 16 instead of 18, which I had set in the build settings.
How can I change the runtime version to 18 for Lambda?

I am facing the same issue with NextJS 14. Next JS 14 drops support for Node 16 and requires at least Node v18.17 - https://nextjs.org/blog/next-14
This means Amplify can not run Nextjs 14 until the lambda environment is upgraded.
Is there a timeline on when Node 18.17 will be supported? Thanks!

It's being worked on according to: https://github.com/aws-amplify/amplify-hosting/issues/3773

Hi @ArianHamdi and @gangeshwark , please see my comment here https://github.com/aws-amplify/amplify-hosting/issues/3773#issuecomment-1792971715 . Closing this to track the ongoing conversation in one place.


Npm install failed (Failed at the<EMAIL_ADDRESS>install script 'node install.js')

Hi anyone experience this?
Downloading http://chromedriver.storage.googleapis.com/2.22/chromedriver_linux64.zip
Saving to /tmp/chromedriver/chromedriver_linux64.zip
Receiving...
Received 781K...
Received 1562K...
events.js:141
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at exports._errnoException (util.js:873:11)
    at TCP.onread (net.js:557:26)
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated<EMAIL_ADDRESS>graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to<EMAIL_ADDRESS>as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.
npm WARN optional dep failed, continuing<EMAIL_ADDRESS>
><EMAIL_ADDRESS>postinstall /home/ferri/Developments/domains/vue-bootstrap/node_modules/cross-spawn/node_modules/spawn-sync
> node postinstall

><EMAIL_ADDRESS>install /home/ferri/Developments/domains/vue-bootstrap/node_modules/phantomjs-prebuilt
> node install.js

Considering PhantomJS found at /usr/local/bin/phantomjs
Found PhantomJS at /usr/local/bin/phantomjs ...verifying
Writing location.js file
PhantomJS is already installed on PATH at /usr/local/bin/phantomjs
npm ERR! Linux 3.19.0-65-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! node v4.4.7
npm ERR! npm  v2.15.8
npm ERR! code ELIFECYCLE

npm ERR<EMAIL_ADDRESS>install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script 'node install.js'.
npm ERR! This is most likely a problem with the chromedriver package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node install.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs chromedriver
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR!     npm owner ls chromedriver
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/ferri/Developments/domains/vue-bootstrap/npm-debug.log

Thanks

Solution:
after upgrading npm with npm install -g npm, then run npm install, then everything is fine..


Add support for logarithmic singularity (fixed-point) quadratures

See http://www.pmf.ni.ac.rs/pmf/publikacije/filomat/2016/30-4/30-4-28.pdf for details
I have an implementation using mpmath, but it feels like I'm being very inefficient about it.

An interesting suggestion. I'll probably look at it at some point.

Let me know if you need any reference code or anything. Like I said, mine feels very inefficient, but I'm not familiar with generating quadratures, so maybe there's an obvious easy way to improve the approach I have.


Include saving model in docs/tutorial

Can an example of serializing models and datasets to disk be included in the tutorials and/or docs? Additional help on using ONNX would be helpful too.

Sure, we can add examples of serialization to the docs. At the moment serialization is using standard java.io.Serializable mechanisms, you can see examples of loading models & datasets here and saving them here from the demo programs that come with Tribuo. Note that you should use a serialization allow list to guard against some deserialization attacks, there's a note in our security docs.
The best examples for the ONNX loaders at the moment are in the unit tests. We'll add a tutorial that goes through the third party model loading process.


Multiple middlewares and subapps

Not sure if this one has been addressed but would like to share the problem I'm encountering.
It's a bit long to list everything out here but I will try my best. Alternatively, there is a demo for the said issue which can be found here: https://glitch.com/edit/#!/indigo-sneeze.
// app.js
const app = polka();

// middleware before subApp, but only `apiCore()` runs and `verifyToken` being skipped.
// It's like either one can be run at the same time.
polka.use('/api', verifyToken(), apiCore());

// verify-token.js

function verifyToken() {
  return (req, res, next) {
    ... // Verify token middleware
  }
}

// api-core.js
function apiCore() {
  return async (req, res) => {
    return polka() // subApp consists of multiple route handlers
      .post('/create', create())
      .post('/view', view());
  };
}

Hey! Good timing haha, I just fixed this tonight, along with a couple related issues.
I'll commit and release the changes tomorrow morning. Thanks!

Thank god (and @lukeed)! 😄 Then I will wait for the new release. Thank you so much! 🎉

Lol not a problem.
In the meantime, you can move your verifyToken into your sub-app directly. The next release you can take it back out.
// api-core.js
module.exports = function apiCore() {
  // return async (req, res) => { REMOVE THIS LINE
    return polka()
      .use(verifyToken)
      .post('/create', create())
      .post('/view', view());
  // }; AND THIS
}

You should return the sub-app directly, always. Otherwise you are recreating the app every time.

Ok, bed time 👋

It worked. Thanks for the patch. 👍


[JFG-865] PDF reporting through JaaS REST API

:+1:

:+1:

Please notice, that from now on (after the merge request is merged) there are 2 ways to get ReportingService

ApplicationContext (in Master and standalnoe Reporter)
ReportingService (in Jaas)

I do believe that it will be great to use one source of the reportingService in future.


Xcode 10 build failure - location of textDidChangeNotification() in UITextView+Validator.swift

I'm seeing the same thing: 'textDidChangeNotification' has been renamed to 'NSNotification.Name.UITextViewTextDidChange'

I think you're using an old version. See https://github.com/adamwaite/Validator/issues/107.

Pod 3.1.1

I see this with pod 3.1.1 and 3.1.0.
Version 3.0.2 is alright.

I can build, run and test on Xcode 10 fine with 3.1.1.

Yeah I'm getting this with Xcode 10 too in one of my projects. Through CocoaPods.
It's very weird though. I tried to create a reproducer with just Validator in the Podfile, but the error doesn't occur.

I get this issue with 3.1.1

@adamwaite this still happens to me on the latest version and looking at the code in 3.1.1 the old style textDidChangeNotification notification names are still there...

Fixed in https://github.com/adamwaite/Validator/commit/4ef95eb33af5624647f9f0a4bf457916e0fabfca

(I think). This one is strange.


Parameter encoding for all requests

Description
This enables encoding parameters in urls correctly for ALL request types, including array parameters, null parameters, empty parameters.
Also allows for #546 without any extra configuration. Try to put the parameters in the body, but if there's already a body, put it in the URL. Still allows for "pre-encoded" urls.
Fixes #546
Type of change
Check all that apply

[ ] Bug fix (a non-breaking change which fixes an issue)
[X] New feature (a non-breaking change which adds functionality)
[X] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[X] Refactoring (a change which changes the current internal or external interface)
[X] This change requires a documentation update

How Has This Been Tested?
New tests have been added to test this behaviour
Checklist:

[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation, if necessary
[X] My changes generate no new compiler warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[ ] Inspect the bytecode viewer, including reasoning why

Based on #550 . Merge that first.

Do you like?
On Tue, 4 Dec 2018, 05:50 Kittinun Vantasin<EMAIL_ADDRESS>wrote:

@kittinunf commented on this pull request.
In
fuel/src/main/kotlin/com/github/kittinunf/fuel/core/interceptors/ParameterEncoder.kt
https://github.com/kittinunf/Fuel/pull/553#discussion_r238529036:

+object ParameterEncoder : FoldableRequestInterceptor {

override fun invoke(next: RequestTransformer): RequestTransformer {

   return inner@{ request ->

       val contentType = request[Headers.CONTENT_TYPE].lastOrNull()

       // Expect the parameters to be already encoded in the body

       if (contentType?.startsWith("multipart/form-data") == true) {

           return@inner next(request)

       }

       // If it can be added to the body

       if (request.body.isEmpty() && allowParametersInBody(request.method)) {

           if (contentType.isNullOrBlank() || contentType.startsWith("application/x-www-form-urlencoded")) {

               return@inner next(

                   request

                       .header(Headers.CONTENT_TYPE, "application/x-www-form-urlencoded")

AHHH I see what it is now.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/kittinunf/Fuel/pull/553#discussion_r238529036, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB35WAN68lCEfhgYQuUZ-j54IWiwt5wvks5u1f79gaJpZM4Y4PYc
.


Add support of Dimension Attributes.

See :

http://www.openmobilealliance.org/release/LightweightM2M/V1_1_1-20190617-A/HTML-Version/OMA-TS-LightweightM2M_Core-V1_1_1-20190617-A.html#5-1-2-0-512-Attributes-Classification
http://www.openmobilealliance.org/release/LightweightM2M/V1_1_1-20190617-A/HTML-Version/OMA-TS-LightweightM2M_Core-V1_1_1-20190617-A.html#6-3-2-0-632-Discover-Operation

This is now implemented by  #1578 and integrated in master. This will be available in next release (2.0.0-M15)


move proplot to function-level import

proplot changes the plot aesthetics on import and the changes aren't always desirable for maps IMO. So we should be conservative about when its imported and only do so when somebody uses tsplot (i.e. the import should be moved inside the method)

@AnGWar26


Array.Sort method call situationally ignores string-comparer argument

Note:

The problem occurs if the the array elements are invisibly [psobject]-wrapped, as happens when you create an array via a Write-Output call, for instance.

As such, it is another manifestation of #5579.

Array.Sort() is just one manifestation: any .NET method call with an array / object[]-typed array argument that relies on seeing the true array element types rather than their incidental [psobject] wrappers will behave unexpectedly from the user's perspective.
Steps to reproduce
# OK, with array constructed *by an expression*.
$a = "b", "ä", "a"; [Array]::Sort($a, [StringComparer]::Ordinal)
$a -join ' ' | Should -Be "a b ä"  # OK - non-ASCII character ä comes LAST in ordinal sorting.

# !! BROKEN, with array constructed via Write-Output, resulting in
# !! individually [psobject]-wrapped strings.
$a = Write-Output "b", "ä", "a"; [Array]::Sort($a, [StringComparer]::Ordinal)
$a -join ' ' | Should -Be "a b ä"  # !! BROKEN - outputs "a ä b" instead

Expected behavior
Both tests should succeed, as it shouldn't matter how the array is constructed.
Actual behavior
The 2nd test fails:
Expected strings to be the same, but they were different. String lengths are both 5. Strings differ at index 2.
Expected: 'a b ä' But was:  'a ä b'

That is, the string-comparer argument was effectively ignored, because the .Sort() method saw psobject instances, not strings.
Environment data
PowerShell Core 7.2.0-preview.3

This may just need to be documented.  I'm not sure what the binder can really do here other than iterate object[] arrays looking for PSObject's to replace with the underlying object. Plus nested arrays, other complex objects like hashtable, etc.
This is also one of the few situations where autonull can leak into a dotnet method.

Thanks, @SeeminglyScience. I agree that trying to fix the problem in the binder is not the right place (and that documentation is called for, for old PowerShell versions alone).
The larger question is if we can eliminate the [psobject] wrapping that automatically happens when compiled cmdlets (only) call .WriteObject() - I don't know if these wrappers are ever needed, but as this example and the other ones in  #5579 show, they certainly cause subtle bugs - and no one should be relying on them, given that they're implementation details.
Compiled cmdlet:
Add-Type @'
using System.Management.Automation;
[Cmdlet("Get", "Foo")]
public class GetFooCmdlet : PSCmdlet {
  protected override void ProcessRecord() {
    WriteObject(42);
  }
}
'@ -PassThru | % Assembly | Import-Module

(Get-Foo) -is [psobject] # -> $true - the [int] instance was [psobject] wrapped

PowerShell code:
# ---
function Get-Foo { 42 }

(Get-Foo) -is [psobject] # -> $false - NO [psobject] wrapper


chore: bdd-spring-1586536031 to 0.0.1

chore: Promote bdd-spring-1586536031 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Performance improvements

Thanks, @pzbyszynski!
Released in 1.2.5 version. If you are using DTTableViewManager or DTCollectionViewManager, you can just update them to 3.1.4 version.

Great, thx ;)


add licencing info about using images/logos

We see more and more people asking for permission to use our logos/charts in their articles about dvc. Looks like we could provide licencing info on dvc.org ourselves, so that people don't have to ask us explicitly.

which license is used to share the logos/ charts?

there is no specific license. It something like this https://github.com/logos . As for the images - use them with a reference to the website.

we need a specific article for this as GitHub has?
or just to add a section in the existing docs.

@ryokugyu I'm not sure. what's your take on it? Could you check other open source projects, or projects like Kong, Prisma.io, etc.

@shcheklein some have a seperate section for logos like you mentioned prisma.io has.
but, kong has not.
imho, let's add a section titled as Logos under /doc/understanding-dvc/resources section and share logos/images. also, including how to embed the DVC logo on a web page.

@ryokugyu could you share some links?
I don't like the idea of mixing it with Resources. They are different.

Ubuntu: https://design.ubuntu.com/downloads/
OSI : https://opensource.org/logo-usage-guidelines
GitHub: https://github.com/logos
They all have seperate pages for it.

related https://github.com/iterative/iterative.ai/issues/290


Finer grained cache hash input

This PR fixes the issue of the caching feature having too broad of parameters that are being used for generating the caching value.
The current implementation uses the entire environment variable dictionary which includes test-level environment variables (e.g. PYTEST_CURRENT_TEST). This results in the cache being not used if a subsequent pytest test session were to be called with a different test that still requires the same tftest instance. To solve this, an internal _env attribute is set within __init__ that defines the env vars that the tftest instance was initiated with. This attribute will be used instead of the env attribute.
In addition, the file contents within tfdir and any tf_var_file or extra_files method arguments are used for generating the caching value. This means that if the Terraform configuration or any tfvars.json files were to change, the cache will be ignored.

Actually, I just realized this is introducing a dependency on checksumdir, making the module less portable. Can you find a way of using the standard library to achieve the same? I'm holding back releasing for now.


docs: new pattern templates

This PR updates the existing single pattern template with two types of pattern templates--one for single variant, and one for multiple variants.
The pattern templates themselves have undergone significant revision to capture what we learned through both the pattern development process and the component usage guidance development process.

Looks good. Might want to add line breaks between the anchor links in the code snippet.

Thanks, @alisonjoseph. I did this by adding two spaces at the end of each line. It did the trick!


Section header isn't moving with the tableview

when adding section headers and using the Plain style for the tableview the header will remain on the location where it starts before you make the parallax header smaller.
 I tried to implement a custom tableView class, as in the example by reference, but it did not help.

when adding section headers and using the Plain style for the tableview the header will remain on the location where it starts before you make the parallax header smaller.

Here is a gist with the swift implementation: https://gist.github.com/ajacquelin/965e08322fdf60bd0cc82b4b9f9f135a

@ajacquelin saved me a couple of hours development time , worked on grouped table too


AssertionError: Only axis-aligned geoboxes are currently supported

I'm relatively new to odc-stac and Digital Earth Africa, and trying out the stac client and odc-stac package to access sentinel-1.    I started with the digital earth africa sentinel2 example code and that worked fine, but when I tried adapting it for sentinel-1, I'm getting AssertionError: Only axis-aligned geoboxes are currently supported, despite trying both projected and lon/lat specification.
I'm guessing this is user error rather than a bug, so suggestions welcome!  :)

It's clearly a bug in odc-stac or possibly datacube. Data sources with non-axis aligned geo references aren't that common so don't get tested as much.
I think this assert is over strict, also not very cheap to run, but it was added 3 years ago...
https://github.com/opendatacube/datacube-core/commit/ca61414b9004e1026a0aabf3cb20ec300877b1b8
Only some operations are not supported for non-axis aligned geoboxes, but odc-stac is just using it as a container to move geo referencing information around.

While this is highlighted an important problem, the real issue is with the projection information in STAC data.
https://explorer.digitalearth.africa/stac/collections/s1_rtc/items/46bbaf19-62fd-58c0-9538-9a48c2aec7cc
    37.0,      0.0002,      0.0,
    13.0,      0.0,      -0.0002,
    0.0,      0.0,      1.0

The order of parameter returned is all messed up, I guess it's using GDAL convention, while stac extension uses row-major, actual files are axis aligned, their transform is:
    0.0002,    0,    37,
    0,    -0.0002,    13,
    0,    0,    1

so, it's a data provider error.
@alexgleith  :arrow_up: metadata for s1_rtc seems to be broken, I don't think it's to stac conversion issue, looks like it's like that in yaml too.

https://explorer.digitalearth.africa/products/s1_rtc/datasets/46bbaf19-62fd-58c0-9538-9a48c2aec7cc

Hey @rsignell-usgs thanks for this report.
This is the original metadata, as supplied: https://deafrica-sentinel-1.s3.af-south-1.amazonaws.com/s1_rtc/N12E037/2020/09/02/03F841/s1_rtc_03F841_N12E037_2020_09_02_metadata.json
It has the same order. I'll get Marko from Sinergise to look at the order of the transform parameters.

Can I confirm that if we do change the axis order in the transform to what GDAL uses, it'll fix it @Kirill888?

So expected order for proj extension is
sx, 0, tx,
0, sy, ty,
0, 0, 1

flattened: sx,0,tx,0,sy,ty
What appears to be in the metadata is: tx,sx,0,ty,0,sy which I think is gdal geotransform order.
which is different from it is expected in stac
https://github.com/stac-extensions/projection#projtransform

Can I confirm that if we do change the axis order in the transform to what GDAL uses, it'll fix it @Kirill888?

@alexgleith not ~to~ but AWAY from what GDAL uses and into matrix form instead, then serialized in row major order as a flat 9 element array
sx,0,tx,0,sy,ty,0,0,1

rio info gives me: "transform": [0.0002, 0.0, 37.0, 0.0, -0.0002, 13.0, 0.0, 0.0, 1.0]
Whereas the STAC is [37, 0.0002, 0, 13, 0, -0.0002]
I don't know which is correct, though.

https://github.com/stac-extensions/projection/issues/9

Same issue in stacstack it seems:
https://github.com/gjoseph92/stackstac/issues/132

@rsignell-usgs you can work-around this issue by disabling proj extension for s1_rtc collection with the following config:
config = {
    "s1_rtc": {
        "ignore_proj": True
    }
}

Then you should be able to load the data, but you will have to supply crs= and resolution= parameters as these can not be auto-guessed when proj extension is disabled or not available. But you already do that in your example.

FYI this has been acknowledged by the Sentinel-1 suppliers and they are issuing a fix.

I believe this has been fixed, for example, here's a newly produced scene: https://explorer.digitalearth.africa/stac/collections/s1_rtc/items/0a461211-1d56-5bb0-8baf-bb9917d8f77f
The transform is:
"proj:transform": [
0.0002,
0,
50,
0,
-0.0002,
12,
0,
0,
1
],


Call a microservice running on Fargate using API Gateway integration

*Issue #200
*Description of changes: This sample project demonstrates how to use Step Functions to make a call to API Gateway in order to interact with a service on AWS Fargate, and also to check whether the call succeeded.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Please move the Step functions definition to a separate file, referencing in line on the template.yaml is not best practice


formatTitle function is resizing and repositioning the title box

I have configured a Slide Title on the Title and Content layout - it is positioned around slide graphics.
The formatTitle() function is flattening the title box which causes the content placeholder to move up the slide.
Ideally, if a title box is found - the dimensions should not be modified.
local fix was to comment out assignments to title.top/left/width

def formatTitle(presentation, slide, titleText, titleFontSize):
    # Find title
    title = findTitleShape(slide)

    if titleText == " ":
        deleteSimpleShape(title)

        return (marginBase, None, "<No title>")

    # Do not resize or reposition the template title
    #title.top = marginBase
    #title.left = marginBase
    #title.width = presentation.slide_width - marginBase * 2

    # Figure out how many lines title will need (ignoring overflow)
    #titleLineCount = len(titleText.split("<br/>"))

    # Add the title text to the first paragraph
    flattenedTitle = addFormattedText(title.text_frame.paragraphs[0], titleText)

Thanks for this. I would like to create a metadata item to govern this. Then you needn't keep patching the code...
Philosophically what I'm trying to do with this code is automate the tidying up of a presentation. In the docs I describe the "origin story" - reworking a surly, higgledy piggledy presentation. Part of that was fonts ' colours and part of that was automating the placement of items. (This latter becomes even more important when there are multiple non-title elements on a slide.)
I have a question: Is it a title / section slide you want a "don't reposition" semantic for? Or is it all slide titles - including ones with content?
I'm not even sure my title repositioning code gets it quite right anyway so am more than happy to reopen this bit of the code. Might be a few days, though.
I assume you're OK with metadata to control md2pptx. Otherwise we have a much bigger problem. :-( :-)

Ideally, Title and Content placeholders do not move and are not resized.  Though just stopping the Title box modification fixed my content-related issues as well.
metadata controls make sense to me -- thank you!

OK. Not going to do keeping content fixed as there are so many use cases where that would be wrong. For example, where you have 2 tables on one slide - and you're probably not going to make a template slide like that.
Question: Is it ALL titles you want nailed in place? Or just Title Slide and Section Slide?
And glad metadata works for you. I assume this need not be "dynamic". The same setting applies for the whole presentation. (Not hard to offer dynamic but it seems overkill - unless someone tells me it's not.)

Yes, all titles.  My use-case doesn't require it to be dynamic. I want to preserve the title placement from all of my template layouts.

All titles is simpler. :-) I don't need to code "what slide type is this?" I'm considering syntax:
AdjustTitles: False

for your use case. Default would be True. Of course this is all case insensitive.

Went with AdjustTitles: no - in keeping with other boolean metadata items.

Pushed. Documentation to follow.

Docs done and pushed. I'd like to fix Issue 84 and any bugs from this before releasing 2.2.1 and closing this.

fixed in v2.2.1.


fix: correctly modify page payload for init mode

Description
This PR fixes #24
Nuxt Core Runtime Nitro Renderer has now 2 different functions renderPayloadScript and renderPayloadJsonScript.
When using renderPayloadJsonScript, isPageSSR returns false when it should be true and  'init' mode is not working.
Linked Issues
#24

Nice catch! Thanks for the help :green_heart:


iOS: Table based layouts look bad

I think this CSS should fix it.  I'm leaving it here to add once I get some test data.  I'm currently waiting for the newsletters to start pouring in.
.nnw-overflow table table { 	margin-bottom: 0; 	border: none; }


Error: read ENOTCONN

Hi @jontewks , I'm having issues with this buildpack on Heroku.
Things I've tried:

making sure to include args: ['--no-sandbox', '--disable-setuid-sandbox'] in the puppeteer.launch command
use the default headless: true in the puppeteer.launch command
clear the build cache, remove, then re-add the buildpack and redeploy

But no luck :( Looks like other people have experienced the same/similar error:

https://stackoverflow.com/questions/67494667/same-question-as-one-asked-a-few-days-ago-why-is-this-puppeteer-code-not-workin
https://stackoverflow.com/questions/64173252/pupeeteer-heroku-buildpack-exiting-the-lunch-statement-on-heroku

Versions:

"puppeteer": "^13.7.0"
"node": "16.13.1"
"npm": "8.1.2"
"yarn": "1.22.5"
"typescript": "^4.5.4"

The error I'm getting looks like...
2022-08-27T19:59:43.518282+00:00 app[worker.1]: node:events:368
2022-08-27T19:59:43.518284+00:00 app[worker.1]: throw er; // Unhandled 'error' event
2022-08-27T19:59:43.518287+00:00 app[worker.1]: ^
2022-08-27T19:59:43.518288+00:00 app[worker.1]:
2022-08-27T19:59:43.518289+00:00 app[worker.1]: Error: read ENOTCONN
2022-08-27T19:59:43.518289+00:00 app[worker.1]: at tryReadStart (node:net:592:20)
2022-08-27T19:59:43.518289+00:00 app[worker.1]: at Socket._read (node:net:603:5)
2022-08-27T19:59:43.518290+00:00 app[worker.1]: at Socket.Readable.read (node:internal/streams/readable:487:10)
2022-08-27T19:59:43.518291+00:00 app[worker.1]: at Socket.read (node:net:644:39)
2022-08-27T19:59:43.518291+00:00 app[worker.1]: at new Socket (node:net:401:12)
2022-08-27T19:59:43.518292+00:00 app[worker.1]: at Object.Socket (node:net:284:41)
2022-08-27T19:59:43.518292+00:00 app[worker.1]: at createSocket (node:internal/child_process:327:14)
2022-08-27T19:59:43.518293+00:00 app[worker.1]: at ChildProcess.spawn (node:internal/child_process:444:23)
2022-08-27T19:59:43.518293+00:00 app[worker.1]: at Object.spawn (node:child_process:698:9)
2022-08-27T19:59:43.518294+00:00 app[worker.1]: at BrowserRunner.start (/app/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:95:34)
2022-08-27T19:59:43.518295+00:00 app[worker.1]: at ChromeLauncher.launch (/app/node_modules/puppeteer/lib/cjs/puppeteer/node/Launcher.js:113:16)
2022-08-27T19:59:43.518295+00:00 app[worker.1]: Emitted 'error' event on Interface instance at:
2022-08-27T19:59:43.518295+00:00 app[worker.1]: at Socket.onerror (node:readline:265:10)
2022-08-27T19:59:43.518296+00:00 app[worker.1]: at Socket.emit (node:events:390:28)
2022-08-27T19:59:43.518296+00:00 app[worker.1]: at Socket.emit (node:domain:475:12)
2022-08-27T19:59:43.518296+00:00 app[worker.1]: at emitErrorNT (node:internal/streams/destroy:157:8)
2022-08-27T19:59:43.518297+00:00 app[worker.1]: at emitErrorCloseNT (node:internal/streams/destroy:122:3)
2022-08-27T19:59:43.518297+00:00 app[worker.1]: at processTicksAndRejections (node:internal/process/task_queues:83:21) {
2022-08-27T19:59:43.518297+00:00 app[worker.1]: errno: -107,
2022-08-27T19:59:43.518298+00:00 app[worker.1]: code: 'ENOTCONN',
2022-08-27T19:59:43.518298+00:00 app[worker.1]: syscall: 'read'
2022-08-27T19:59:43.518298+00:00 app[worker.1]: }
2022-08-27T19:59:43.538852+00:00 app[worker.1]: error Command failed with exit code 1.
2022-08-27T19:59:43.538962+00:00 app[worker.1]: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
2022-08-27T19:59:43.561462+00:00 app[worker.1]: error Command failed.
2022-08-27T19:59:43.561490+00:00 app[worker.1]: Exit code: 1
2022-08-27T19:59:43.561492+00:00 app[worker.1]: Command: /app/.heroku/node/bin/node
2022-08-27T19:59:43.561493+00:00 app[worker.1]: Arguments: /app/.yarn/releases/yarn-1.22.5.cjs worker-start
2022-08-27T19:59:43.561493+00:00 app[worker.1]: Directory: /app/packages/processing
2022-08-27T19:59:43.561493+00:00 app[worker.1]: Output:
2022-08-27T19:59:43.561494+00:00 app[worker.1]:
2022-08-27T19:59:43.561783+00:00 app[worker.1]: info Visit https://yarnpkg.com/en/docs/cli/workspace for documentation about this command.
2022-08-27T19:59:43.760317+00:00 heroku[worker.1]: Process exited with status 1
2022-08-27T19:59:43.831973+00:00 heroku[worker.1]: State changed from up to crashed

Any suggestions for this? It would be really nice to be able to use this to run puppeteer on Heroku. Thanks!

@jontewks I also should add, I tried:

Upgrading to the latest Puppeteer version, 16.2.0
Remove '--disable-setuid-sandbox' from the args on puppeteer.launch

Some additional important details about the errors I'm seeing:

This error seems to only happen periodically -- the majority of pages seem to be crawled just fine
This error causes the Heroku proc to crash and have to restart
I can't seem to catch this error, even by try/catch-ing the entirety of the function all Puppeteer invocations happen in -- if this weren't the case, the error would be workable.

Here is my current code:
export const setupPuppeteer = async (): Promise<any> => {
  const browser = await puppeteer.launch({
    headless: true,
    args: ['--no-sandbox', '--disable-setuid-sandbox']
  });

  const page = await browser.newPage();

  await page.setUserAgent(
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36'
  );

  await page.setViewport({ width: 1920, height: 1080 });

  await page.setDefaultNavigationTimeout(15000);
  await page.setDefaultTimeout(15000);

  return [page, browser];
}

Can you help me make this code more reliable?

Hey @cakemountain
I don't think I can help out in this scenario. This buildpack only adds libraries to Heroku's runtime to allow Chromium to function. As far as I'm aware it doesn't have an effect on how Chromium runs after its started. I think the Puppeteer repo itself would be a better place to get help with it's use. There are mentions of it with Puppeteer outside of Heroku as well, so I wouldn't think this was a Heroku specific thing: https://github.com/pedroslopez/whatsapp-web.js/issues/1133


Canvas Modulate should use light mask

Canvas modulate can be thought of as a light that applies uniformly to everything in 2D. It would be helpful, if you could apply it to some nodes, but not others based on the light mask.

Totally agree!

@xXGOTNOFANSXx Please don't bump issues without contributing significant new information. Use the :+1: reaction button on the first post instead.


Aiohttp with nginx

The recommended way to deploy aiohttp is with nginx. In the documentation it clearly states that using gunicorn is slower than the recommended nginx solution.
This code runs hapily in my machine, I am unsure if the actions would work propertly.
I am not sure if other frameworks would profit from nginx. Most of them are wsgi/asgi based, aiohttp is not, so maybe is the only one.

I guess this does not add much actually, I am unsure how many workers are being used by gunicorn anyways. I am happy to make changes, but as it is this might not add much.


Setup comprehensive CI

We need to make sure that images built with rules_oci work with common runtimes out there. This will give us confidence that our rules such as oci_tarball oci_image can built artifacts that are loadable and usable.
We should have a CI set up for

darwin/arm64 with multipass
darwin/arm64 with podman
linux/amd64 with docker
linux/amd64 with podman

fixed by #33 and #30


Begin adding pre-trained models

This pull request beings to process of adding pre-trained models.

[x] Make sure pre-trained models are included when Saber is installed
[x] Make sure the web-service and jupyter notebooks work
[x] Make it such that the user can easily specify pre-trained models shipped with Saber (#39)

TODO

[ ] Train and save more pre-trained models (likely move this to its own pull-request)
[ ] Update the jupyter notebooks with documentation

Pull Request Test Coverage Report for Build 151

13 of 38 (34.21%)  changed or added relevant lines in 5 files are covered.
8 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.3%) to 74.603%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

saber/sequence_processor.py
1
5
20.0%

saber/utils/app_utils.py
7
14
50.0%

saber/app.py
0
14
0.0%

Files with Coverage Reduction
New Missed Lines
%

saber/sequence_processor.py
2
67.34%

saber/utils/app_utils.py
3
47.76%

saber/app.py
3
0.0%

Totals

Change from base Build 150:
-0.3%

Covered Lines:
1222

Relevant Lines:
1638

💛  - Coveralls


What is dbt?

I didn't know what dbt was until I found https://docs.getdbt.com/docs/introduction after reading the README.md in this repo. I think a short description that explains what dbt is, and more about how this tool uses dbt, could be a useful update to the readme.

Thanks @SgtPooki! I can see how confusing is this right now. Will add a small section explaining the different tools that Datadex uses.

Fixed in https://github.com/davidgasquez/datadex/commit/e63d634a448459a488b149fc8132deeac819dc7f! Thanks again @SgtPooki for the issue.


Direct3D 11 Feature Level Selection

Hello, I was looking into the C99 bindings for the past days and I was wondering how should D3D 11 backend feature level be set; or does bgfx detect gpu caps at runtime and selects the feature level automatically?

bgfx automatically makes use of feature levels where appropriate. If you want to target lower feature levels with shaders you'll need to build them against the appropriate shader profile though.

Thanks! Now let's assume I'm going to render a couple of skinned identical objects; OpenGL3ish way of rendering of them is to setup an uniform/texture buffer object to feed the bone transformation data, and finally use instanced drawing.
How should I write the rendering path (or paths?) using BGFX, so it can run on both old DX9/OGL2 hardware (no UBO/TBO and instancing support) and modern hardware effectively?

bgfx automatically abstracts things like uniforms and does the most efficient thing depending on the user's hardware. The whole point of the library is that you don't worry about the underlying system.

That's amazing, thanks again for help :)


Does it auto connect if service disconnects?

Team,
In majority cases, the port forward service auto disconnects after few hours or days. Does this application application automatically reconnect in such cases.
For eg. we have jenkins hosted in kubernetes. To connect windows slave agents with jenkins we have to enable port forwarding in production. After few days, connection automatically disconnects which needs a custom coding of looping mechanism either using powershell or bat script or any custom console application or task scheduler.
Does kubefwd application resolves this problem and auto reconnects with the services. Please help.
Thank you

@smartaquarius10 kubefwd does not monitor the forwarded connection. kubefwd is designed for the temporary forwarding of services to aid in local development.


Add virtual processing option

Integrates https://github.com/isce-framework/isce2/pull/319 into this plugin

I have been running the sample_run.sh with the newly added --process-virtually option as seen below:
isce2_topsapp --username <username> \
              --password <password> \
              --reference-scenes S1B_IW_SLC__1SDV_20210723T014947_20210723T015014_027915_0354B4_B3A9 \
              --secondary-scenes S1B_IW_SLC__1SDV_20210711T014922_20210711T014949_027740_034F80_859D \
                                 S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404 \
                                 S1B_IW_SLC__1SDV_20210711T015011_20210711T015038_027740_034F80_376C \
              --process-virtually \
             > topsapp_img.out 2> topsapp_img.err

Currently the process fails at burstifg step:
TopsApp Steps:  62%|██████▎   | 15/24 [3:12:49<1:55:41, 771.29s/it]
Traceback (most recent call last):
  File "/home/havazli/miniconda3/envs/topsapp_env/bin/isce2_topsapp", line 33, in <module>
    sys.exit(load_entry_point('isce2-topsapp', 'console_scripts', 'isce2_topsapp')())
  File "/u/kriek1/havazli/DockerizedTopsApp/isce2_topsapp/__main__.py", line 117, in main
    topsapp_processing(reference_slc_zips=loc_data['ref_paths'],
  File "/u/kriek1/havazli/DockerizedTopsApp/isce2_topsapp/topsapp_proc.py", line 76, in topsapp_processing
    raise ValueError(f'TopsApp failed at step: {step}')
ValueError: TopsApp failed at step: burstifg

Another error I receive multiple times, which doesn't kill the process, is access to a specific dataset:
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw1-slc-vv-20210711t014948-20210711t015013-027740-034f80-004.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw1-slc-vv-20210711t014948-20210711t015013-027740-034f80-004.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw1-slc-vv-20210711t014948-20210711t015013-027740-034f80-004.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw1-slc-vv-20210711t014948-20210711t015013-027740-034f80-004.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw1-slc-vv-20210711t014948-20210711t015013-027740-034f80-004.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw1-slc-vv-20210711t014948-20210711t015013-027740-034f80-004.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw1-slc-vv-20210711t014948-20210711t015013-027740-034f80-004.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw1-slc-vv-20210711t014948-20210711t015013-027740-034f80-004.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw2-slc-vv-20210711t014947-20210711t015012-027740-034f80-005.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.zip/S1B_IW_SLC__1SDV_20210711T014947_20210711T015013_027740_034F80_D404.SAFE/measurement/s1b-iw3-slc-vv-20210711t014947-20210711t015013-027740-034f80-006.tiff' does not exist in the file system, and is not recognized as a supported dataset name.
TopsApp Steps:  62%|██████▎   | 15/24 [3:08:04<4:25:35, 1770.56s/it]ERROR 4: `/vsizip/vsicurl/https://datapool.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210723T014947_20210723T015014_027915_0354B4_B3A9.zip/S1B_IW_SLC__1SDV_20210723T014947_20210723T015014_027915_0354B4_B3A9.SAFE/measurement/s1b-iw1-slc-vv-20210723t014949-20210723t015014-027915-0354b4-004.tiff' does not exist in th
e file system, and is not recognized as a supported dataset name.

The process runs over 3 hours and errors out. Majority of the time is spent on fineresamp step. Fine resampling spends 5 minutes on each burst (3 subswaths X 9 bursts X 5 minutes = 2 hours 15 minutes).
My environment is a new installation of the repo and the versions of the usual suspects are:
  gdal                3.4.3-py310hce6f0df_0
  isce2              2.6.0-py310h13e036c_0
  python           3.10.4-h9a8a25e_0_cpython

@sssangha  - do you know how long fine resampling normally takes? i.e. can you get it from one of the hyp3 jobs you have submitted? That would be a helpful point of comparison. Or @ehavazli do you have this time stored elsewhere?
@ehavazli - as best as possible, as you work on this, please explain why there is so much difference between the above performance and the report below
We successfully included virtual data processing into ISCE2 (version 2.5.3 and above) Sentinel-1 interferogram processing, i.e. the ability for the Sentinel-1 InSAR processor to more efficiently access data in the cloud and perform initial reading completely within an instance. Our initial tests have shown ~20% increase in processing time and ~25% decrease in utilized disk space on the file system for processing collocated with the data.

We are less interested in disk space right now and more just on wall-clock time.

So far I tried downgrading python and gdal packages and haven't been able to bring the virtual process to a completion. Currently I am working on locating the issue causing the virtual processing to fail at burstifg step. I believe this may be caused by a bug in the way the measurement file is called within the code.
@cmarshak The discrepancy is due to the differences in testing environments. Initial development and testing was done on OSL using 2 SLCs staged as:
https://jhk-data-east.s3.amazonaws.com/SLC/SA/S1A_IW_SLC__1SDV_20210501T230628_20210501T230655_037701_0472DA_54F3.zip
https://jhk-data-east.s3.amazonaws.com/SLC/SA/S1A_IW_SLC__1SDV_20210525T230629_20210525T230656_038051_047DAA_A759.zip

which were previously in:
https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SDV_20210525T230629_20210525T230656_038051_047DAA_A759.zip
https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SDV_20210501T230628_20210501T230655_037701_0472DA_54F3.zip

This was done after contacting @jhkennedy and asking him to stage specific SLCs on the east coast.
If I should look at the bigger picture here; downloading each SLC takes ~30 seconds (2.5 minutes total for the sample_run.sh) while staging virtual data files takes ~20 seconds in total. Even though the process isn't brought to a completion at the end of the virtual processing, majority of the runtime is spent on fineresample step which takes 2 hours 15 minutes. During local processing (downloading SLC's and processing on the server) fineresample step takes ~10 minutes. So we are losing way more time than we are gaining with virtual processing. I don't think the virtual processing option is a promising feature for the purpose of saving time during GUNW production. However, a fair comparison will be possible after resolving the bug which causes the virtual processing to fail.

We are going to close this until further notice because the wall-clock time is not going to improve with ISCE2 virtual data processing. At best 2% (1-2 minutes of 90+ minute processing time accounted for downloading) is the most we have to gain in wall-clock time, which has to be measured with workforce investment in making ISCE2 optimized for cloud reads (not to mention SLC data is not easily subset so may not be possible). To be clear, cannot shave all of 2% off wall-clock time, just some small fraction of it.


Div's positioning

Is there any reason why the elements are added using an absolute position? It shouldn't affect SEO, am I wrong?

Yes, absolute position or maintaining the regular html flow should have same seo effects. I know that html should follow the flow, but just to be on safer side and have same UI I used absolute position and width for it.
I might be wrong Do you have any suggestion about it? I am not very much into web/html.


Module not found: Can't resolve 'react-internet-speed-meter/dist/index.css'

Module not found: Can't resolve 'react-internet-speed-meter/dist/index.css'

Hey sorry for the late response.
If you are facing the same issue
Can you please provide some more details?
Adding screenshot/s will be helpful.

Tried replicating it but with given versions it is running on my local machine.
More data can help here.
What you can do here.

Try updating react-internet-meter version

if possible update react-scripts.

I got the same error
installed version : "react": "^17.0.2", "react-internet-meter": "^1.1.1",
Solution for anyone still facing this
Change
from: import 'react-internet-speed-meter/dist/index.css'
to:  import 'react-internet-meter/dist/index.css'
For me the folder created inside node_modules is react-internet-meter.


Document how to provide options for .NET SDK's models

I couldn't find any documentation on how to provide options for analyzers/tokenizers/filters using the .NET SDK, I had to resort to reading the SDK's source code. The documentation should have at least one example on this topic, ideally one that lines up with the raw JSON example here: https://docs.microsoft.com/en-us/azure/search/search-analyzers#custom-analyzer-example

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d45ce92c-4793-b875-7c5e-596fea89443f
Version Independent ID: c351f93b-e3a2-f737-bb63-9bf0fb2b7dc2
Content: Create an index in code using the .NET API - Azure Search
Content Source: articles/search/search-create-index-dotnet.md
Service: search
GitHub Login: @brjohnstmsft
Microsoft Alias: brjohnst

Hi @loic-sharma, thanks for bringing this to our attention. Your feedback has been shared with the content owner for further review.

@loic-sharma Did you manage to find the SDK reference docs? I'm wondering if we need to make those more discoverable in addition to adding examples of analyzer usage.
Also, the article on analyzers seems like a better place to put SDK examples focused on analyzers rather than the more generic tutorial. Would you agree?

Did you manage to find the SDK reference docs? I'm wondering if we need to make those more discoverable in addition to adding examples of analyzer usage.

The reference is great to better understand types, but poor for discoverability. I don't think many people will think of drilling down on the reference's "Search", "Client", and "Microsoft.Azure.Search.Models" sections. I think it'd be great if the documentation linked to these types' reference.

Also, the article on analyzers seems like a better place to put SDK examples focused on analyzers rather than the more generic tutorial. Would you agree?

I agree that the generic tutorial should be short and sweet, and not bogged down by advanced features. I think that adding this to the article on analyzers would be good, but this page may get large especially if y'all add  SDKs for more languages.
Personally, I would follow Cosmos DB's documentation. They have a "Samples" section with a page for each SDK they support. Each page is broken down by categories like "Document examples", "Query examples", etc.. Each category has tasks like "Create a document", "Delete a document". Each task links to a sample on GitHub with its relevant API reference. See this page for an example: https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-dotnet-samples

@loic-sharma Thanks for the feedback. We'll look at doing a better job of linking our docs to our SDK samples.

FYI @HeidiSteen

@loic-sharma  I updated the analyzer docs to include C# examples (https://docs.microsoft.com/en-us/azure/search/search-analyzers#c-examples).   Given the addition of C# examples, I would like to close this issue because it's addressed in the docs.
Other areas for improvement, such as adding more links and re-framing our samples, are in progress right now, but I would like to answer for that work as separate issues.  If you could file new issues, or just wait until the end of April, we will be done with sample repo work, sample content pages, and API linking.
Thanks!

#please-close

Thanks Heidi @HeidiSteen.  We will now proceed to close this thread.


s3tk scan --object-level-logging error

Ran the s3tk scan with the --object-level-logging flag and received the following error:
Traceback (most recent call last):
  File "/usr/local/bin/s3tk", line 7, in <module>
    s3tk.cli()
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/s3tk/__init__.py", line 440, in scan
    event_selectors = fetch_event_selectors() if object_level_logging else {}
  File "/usr/local/lib/python2.7/dist-packages/s3tk/__init__.py", line 408, in fetch_event_selectors
    if trail['IsMultiRegionTrail']:
KeyError: 'IsMultiRegionTrail'

Thanks @JRodriguez556, should be good now.


es-300-draw-elementes frag-fail doesn't fail

As title
Win 7 x64
GeForce GTX 770/PCIe/SSE2
OpenGL ES 3.2 NVIDIA 378.78

Ouppppps, that's an old bug issue.
The issue here is that it should fail but on NVIDIA drivers it doesn't which is a bug.
Thanks,
Christophe


Name is empty error in withdraw API call

Hello,
I am trying to call withdraw API. All the API key and secret is enabled and i call withdraw api like below:
binance.withdraw('ETH', gdaxAddress, 0.0205, false, function(response) {
console.log(response);
});
"Name" isn't a mandatory param. I still get following error response:
{ msg: 'Name is empty.', success: false }
What could be the issue.
Thanks
Aditya

Thanks for the report!
Hopefully this is fixed in version 0.2.12
Try updating by navigating to your project directory and issuing a composer update
Please let me know if it works or if you still have trouble

The latest version is 0.2.11. I tried reinstalling node-binance-api but it still points to 0.2.11. composer update requires composer.json. Where do i get it from. It should be simple update from npm ?


ARROW-10699: [C++] Fix BitmapUInt64Reader on big endian

s390x CI ok: https://travis-ci.org/github/pitrou/arrow/builds/745424074.
(the remaining test failures are ARROW-10610)


Temp Role Request

What is your username and discriminator?
allbombson#4979
Why did you join Discord Linux?
I joined so i would be able to help other and be helped myself.
Where did you hear about Discord Linux?
Some of the official discord servers.
What is the passphrase that was assigned to you by the bot?
bosom-refrigerator-twine-prior

From Ethan Weegee#9302 :
Looks good.  I've given you a temporary role so that you can enter the passphrase in the welcome channel.


Linear target dataset structure

Thank you for your research and for sharing your code!
I want to train a custom dataset rgb2rgb ex. blured_image 2 focused_image.
From your paper it seams that the I should use the Linear target approach.
How would I go about creating a dataset structure? Should it be as simple as trainA (blured images) trainB (focused images)?
Can you provide your Linear target dataset loading files?
Thank you!

Thanks for your interest!
Yes, it's correct, the linear approach is appropriate. The dataset structure is exactly like that, you don't need anything else except for the custom model in the case for sharpness to apply to blurred images. I believe that some simple sharpening filter will do the trick.
For the linear model, it's suboptimal (see supp.) but you can also simply adapt the cyclic dataset structure for the linear case by providing a sharpening model invariant to sin(\phi) (i.e. which just depends on cos(\phi)), in that case you'll be ready for a quick test really fast. The cyclic FIN will simply collapse to a linear one. In other words, just modify the __apply_colormap function in the dataset with a __sharpening one to apply the correct continuous model.

I'm closing the issue, if you need further assistance don't hesitate to reopen.


Also hint at using the next port directly

Ross gave a wonderful tip:

You can avoid this if you explicitly specify the rdzv backend with localhost:0

So for us, we can tell the user they can specify it to 0, and the next port will be used.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Fabric 1.19

Ported it for a friend.
No idea if you want to manage a fabric port, but I'll gladly offer a fully working version and will do 1.18.2 and 1.19.3, too.

Fabric 1.18 & 1.19.3 are done, too

If you don't want to maintain fabric ports, I will happily offer doing so - with your permission, of course.

no port works, error on entrypoint stage main, on 1.18.x and 1.19.x: Caused by: java.lang.IllegalStateException: Registry is already frozen, on 1.19.3 Caused by: java.lang.IllegalStateException: This registry can't create intrusive holders.
java: eclipse adoptium 17.0.4 amd64
other mods: none, no dependency reported by loader

closing as not planned