Hampel numba implementation

@ecomodeller Also added the Hampel detector to the notebook. Really fast compared to pure pandas.

@ecomodeller Also added the Hampel detector to the notebook. Really fast compared to pure pandas.

@rhaDHI I have no fork and no further changes

From: Rasmus Halvgaard<EMAIL_ADDRESS>Sent: Monday, 25 January 2021 09.23
To: DHI/anomalydetection<EMAIL_ADDRESS>Cc: Anne Katrine V.Falk<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [DHI/anomalydetection] Hampel numba implementation (#9)
@rhaDHI commented on this pull request.

In anomalydetection/hampel.pyhttps://github.com/DHI/anomalydetection/pull/9#discussion_r563533568:

   The window range is from [(i - window_size):(i + window_size)], so window_size is half of the

   window, counted in number of array elements (as opposed to specify a time span, which is not

   supported by this implementation)

threshold: float

   The threshold for marking an outlier. A low threshold "narrows" the band within which values are deemed as

   outliers. n_sigmas

k
"""

time_series_clean = time_series.copy()
outlier_indices = []
is_outlier = [False] * len(time_series)

for t in range(window_size, (len(time_series) - window_size)):

   time_series_window = time_series[(t - window_size):(t + window_size)]

   median_in_window = np.nanmedian(time_series_window)

@akfDHIhttps://github.com/akfDHI Do you have more changes to this PR in your fork? Otherwise I will change to np.nanmedian and close this PR.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/DHI/anomalydetection/pull/9#discussion_r563533568, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIECKWFFGPU3GRAYLLWBCSTS3UTAPANCNFSM4WI5PD7A.

@rhaDHI I have no fork and no further changes

From: Rasmus Halvgaard<EMAIL_ADDRESS>Sent: Monday, 25 January 2021 09.23
To: DHI/anomalydetection<EMAIL_ADDRESS>Cc: Anne Katrine V.Falk<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [DHI/anomalydetection] Hampel numba implementation (#9)
@rhaDHI commented on this pull request.

In anomalydetection/hampel.pyhttps://github.com/DHI/anomalydetection/pull/9#discussion_r563533568:

   The window range is from [(i - window_size):(i + window_size)], so window_size is half of the

   window, counted in number of array elements (as opposed to specify a time span, which is not

   supported by this implementation)

threshold: float

   The threshold for marking an outlier. A low threshold "narrows" the band within which values are deemed as

   outliers. n_sigmas

k
"""

time_series_clean = time_series.copy()
outlier_indices = []
is_outlier = [False] * len(time_series)

for t in range(window_size, (len(time_series) - window_size)):

   time_series_window = time_series[(t - window_size):(t + window_size)]

   median_in_window = np.nanmedian(time_series_window)

@akfDHIhttps://github.com/akfDHI Do you have more changes to this PR in your fork? Otherwise I will change to np.nanmedian and close this PR.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/DHI/anomalydetection/pull/9#discussion_r563533568, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIECKWFFGPU3GRAYLLWBCSTS3UTAPANCNFSM4WI5PD7A.

The non-numba version already handles nan's. Pandas' median has skipna=True by default and all nans are subsequently filled. So we should be fine.

The non-numba version already handles nan's. Pandas' median has skipna=True by default and all nans are subsequently filled. So we should be fine.


Particle velocity is not initialized to zero when new particles start

Tested versions

v4.2.2.stable.mono.official [15073afe3]
v4.3.stable.mono.official [77dcf97d8]

System information
Godot v4.3.stable.mono - Windows 10.0.22631 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 4080 (NVIDIA; <IP_ADDRESS>99) - AMD Ryzen 9 7950X3D 16-Core Processor (32 Threads)
Issue description
Assume a very simple particle shader like this:
shader_type particles;

void start() {
	TRANSFORM = mat4(vec4(1., 0.,0.,0.),vec4(0.,1.,0.,0.), vec4(0.,0.,1.,0.), vec4(float(INDEX),0.,0.,1.));
	VELOCITY = vec3(0,5,0);
}

This will make the particles line up in a row and fall down with 5 px per second. Now when I comment out the VELOCITY line, I would expect newly spawned particles to no longer fall down. However, newly spawned particles keep the velocity until I reload the scene.
https://github.com/user-attachments/assets/b4bde8ac-6373-4e87-bd9d-a8d60360884b
I would expect the particles to be initialized with a zero velocity on start.  This also affects visual shaders but I assume fixing the underlying issue will fix both text-based and visual shaders.
Steps to reproduce

Create a GPUParticles2D node.
Create a particle shader script with the contents above.
Repeat the steps from the video.

Minimal reproduction project (MRP)
N/A

Please upload an MRP to make this easier to test and ensure the exact same steps can be performed, to rule out any other configuration being part of this

I confirm this behavior, here is the minimum MRP for testing
part.zip


bumps go version: 1.14 -> 1.17

Fixed https://github.com/signalsciences/terraform-provider-sigsci/issues/56


feat: deserialize new epoch stakes bank snapshot field

Problem
Epoch stakes are not serialized into bank snapshots with the stake credits_observed field which is highly desired for implementing snapshot recovery of partitioned epoch rewards state.
Summary of Changes
Implements phase 1 of SIMD-0149 by deserializing the new epoch stakes map field from the bank snapshot if it exists. If the map is deserialized, it will be merged with the old epoch stakes field entries.

Codecov Report
Attention: Patch coverage is 73.61111% with 19 lines in your changes are missing coverage. Please review.

Project coverage is 82.7%. Comparing base (e01278e) to head (86f5efb).
Report is 6 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##           master    #1397     +/-   ##
=========================================
- Coverage    82.7%    82.7%   -0.1%
=========================================
  Files         872      871      -1
  Lines      369436   369420     -16
=========================================
- Hits       305779   305723     -56
- Misses      63657    63697     +40

I wasn't planning on backporting but could be nice actually. But we would definitely need the merging logic if we did backport because otherwise old versions would be deserializing but not picking up necessary epoch stakes entries from new snapshots.

rebased to fix conflicts (just imports)

Discussed with @brooksprumo and not planning to backport this.. phase 2 of the new field migration will be implemented for v2.1

Discussed with @brooksprumo and not planning to backport this.. phase 2 of the new field migration will be implemented for v2.1

Sorry for not chiming in, but this is my preferred approach also 🤝

Fixed a bad conflict resolution, can I get another approval?


snack-bar multiline

Bug, feature request, or proposal:
proposal
What is the expected behavior?
According to the material design spec, a snack-bar toast can have 2 lines of text. I want the snack-bar to grow in height to a minimum of 2 lines when it has 2 lines.
What is the current behavior?
The current behavior is that the text is pushed down under the bottom of the page
What are the steps to reproduce?
Providing a Plunker (or similar) is the best way to get the team to see your issue.
Plunker template: http://plnkr.co/edit/o077B6uEiiIgkC0S06dd
What is the use-case or motivation for changing an existing behavior?
Which versions of Angular, Material, OS, browsers are affected?
Angular 2.0.1
Material 2.0.0-alpha.10
Is there anything else we should know?

@josephperrott can you already do this with openFromComponent?

Multi-line snackbars are part of the spec on mobile only.
Unfortunately you would not be able to do this as we define the height explicitly for the snackbar container.

@josephperrott It's a mobile web app I'm writing, if the screen is very narrow you can't put a lot of text on one line. I get that you shouldn't put too much text in a snackbar, but I also get the spec that mobile can have 2 lines.
I currently fixed it with an '!important', I know it's bad practice, but it is contained to only this case.
A solution could be to apply the mobile specs through a media query if the device has a screen width of a certain amount of pixels -> mobile?

Yep, overall that is the plan.  I am currently working on the service that is needed for determining the breakpoints for when multiline should be used.

@josephperrott Oh perfect. Love the work put in to Material2 btw!

Hi, please make example for multi snack-bar, uses function openFromComponent.
I make test component :
`import { Component, OnInit } from '@angular/core';
@Component({
selector: 'app-test-messages',
templateUrl: './test-messages.component.html',
styleUrls: ['./test-messages.component.css']
})
export class TestMessagesComponent implements OnInit {
constructor() { }
ngOnInit() {
}
}I uses component: let config = new MdSnackBarConfig();
config.viewContainerRef = this.viewContainerRef;
let tests = this.snackBar.openFromComponent(TestMessagesComponent, config);`
I have error message:
Error in ./CardLogComponent class CardLogComponent - inline template:35:20 caused by: No component factory found for TestMessagesComponent

@cyberat Add your snackbar component both to ngModule's declarations and entryComponents, it works

The Material Components page is also showing this defect.
Is there a solution available for this problem? Unfortunately, I can't set the height due to the length of the text in multiple languages, so that answer is not solving the problem.


Feature support lammps data files

@junghans I think I have addressed your comments but take a look.


Could not load file or assembly error when loading a class library from a module in beta.3

I have a C# Powershell module compiled with netcoreapp2.0 that uses a C# class also library compiled with netcoreapp2.0.
Since beta.3 when trying to load the class library from the module I am seeing a 'Could not load file or assembly' error. The same code works just fine in beta.2.
I have created an example case in the following repo with an example of the issue: https://github.com/barcexwiki/cannotloadassembly
In the repo, project 'psmodule' is a module that has a single cmdlet: Get-HelloWorld. This cmdlet calls an static method in a class in the 'classlibrary' project and outputs "Hello World from classlibrary!".
Steps to reproduce
D:\>git clone https://github.com/barcexwiki/cannotloadassembly.git
Cloning into 'cannotloadassembly'...
remote: Counting objects: 8, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 8 (delta 0), reused 8 (delta 0), pack-reused 0
Unpacking objects: 100% (8/8), done.
Checking connectivity... done.

D:\>cd cannotloadassembly\psmodule

D:\cannotloadassembly\psmodule>dotnet restore
  Restoring packages for D:\cannotloadassembly\classlibrary\classlibrary.csproj...
  Restoring packages for D:\cannotloadassembly\psmodule\psmodule.csproj...
  Generating MSBuild file D:\cannotloadassembly\classlibrary\obj\classlibrary.csproj.nuget.g.props.
  Generating MSBuild file D:\cannotloadassembly\classlibrary\obj\classlibrary.csproj.nuget.g.targets.
  Restore completed in 296,18 ms for D:\cannotloadassembly\classlibrary\classlibrary.csproj.
  Generating MSBuild file D:\cannotloadassembly\psmodule\obj\psmodule.csproj.nuget.g.props.
  Generating MSBuild file D:\cannotloadassembly\psmodule\obj\psmodule.csproj.nuget.g.targets.
  Restore completed in 2,46 sec for D:\cannotloadassembly\psmodule\psmodule.csproj.

D:\cannotloadassembly\psmodule>dotnet build
Microsoft (R) Build Engine version 15.3.388.41745 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  classlibrary -> D:\cannotloadassembly\classlibrary\bin\Debug\netcoreapp2.0\classlibrary.dll
  psmodule -> D:\cannotloadassembly\psmodule\bin\Debug\netcoreapp2.0\psmodule.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.78

Expected behavior
Loading the module with beta.2 it loads the class library and the Get-HelloWorld cmdlet works.
D:\cannotloadassembly\psmodule>"c:\Program Files\PowerShell\6.0.0-beta.2\powershell.exe"
PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS D:\cannotloadassembly\psmodule> Import-Module D:\cannotloadassembly\psmodule\bin\Debug\netcoreapp2.0\psmodule.dll
PS D:\cannotloadassembly\psmodule> Get-HelloWorld
Hello World from classlibrary!

Actual behavior
Loading the module with beta.3 it loads the class library and the Get-HelloWorld cmdlet fails to load the class library.
D:\cannotloadassembly\psmodule>"c:\Program Files\PowerShell\6.0.0-beta.3\powershell.exe"
PowerShell v6.0.0-beta.3
Copyright (C) Microsoft Corporation. All rights reserved.

PS D:\cannotloadassembly\psmodule> Import-Module D:\cannotloadassembly\psmodule\bin\Debug\netcoreapp2.0\psmodule.dll
PS D:\cannotloadassembly\psmodule> Get-HelloWorld
Get-HelloWorld : Could not load file or assembly 'classlibrary, Version=<IP_ADDRESS>, Culture=neutral,
PublicKeyToken=null'. El sistema no puede encontrar el archivo especificado.
At line:1 char:1
+ Get-HelloWorld
+ ~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-HelloWorld], FileNotFoundException
    + FullyQualifiedErrorId : System.IO.FileNotFoundException,psmodule.GetHelloWorld

Environment data
for Powershell beta.2
> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta
PSEdition                      Core
BuildVersion                   <IP_ADDRESS>
CLRVersion
GitCommitId                    v6.0.0-beta.2
OS                             Microsoft Windows 10.0.15063
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

for Powershell beta.3
> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta
PSEdition                      Core
GitCommitId                    v6.0.0-beta.3
OS                             Microsoft Windows 10.0.15063
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

net version
> dotnet --version
2.0.0-preview2-006497

@daxian-dbw Could you please comment?

Hum! I'm having a similar issue as in Beta.2 can load the Microsoft.SqlServer.Smo assembly but not in Beta.3.  And, is giving a similar error message stating the assembly is not found or not loaded.
Just wondering it is related!
:)

This is due to https://github.com/dotnet/corefx/issues/18791 (For an assembly loaded via 'Assembly.LoadFrom', the assembly it referenced cannot be loaded even if they are in the same folder). This has been addressed by moving PowerShell to .NET Core 2.0.0-preview3 (#4144).
I will double validate using the repro repo https://github.com/barcexwiki/cannotloadassembly.git and reply back.

Verified that the repro works with the latest powershell:
PS:68> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta
PSEdition                      Core
GitCommitId                    v6.0.0-beta.3-30-gece27ff68798f508c755ccb84be5ae78b386ac82
OS                             Microsoft Windows 10.0.15063
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

[F:\tmp\repro\cannotloadassembly\psmodule]
PS:69> Import-Module F:\tmp\repro\cannotloadassembly\psmodule\bin\Debug\netcoreapp2.0\psmodule.dll
[F:\tmp\repro\cannotloadassembly\psmodule]
PS:70> Get-HelloWorld
Hello World from classlibrary!

The beta.4 powershell will be released on 7/13.

@daxian-dbw can we add a test for this?

@SteveL-MSFT Sure, will do.

As expected, it works in beta.4! Thanks!

@daxian-dbw can you post link to PR for added test here?

Thanks for bringing it up. It slipped off my radar ... The PR has been submitted.

@SteveL-MSFT @daxian-dbw
It still failing loading "Microsoft.SqlServer.Smo" assembly in Windows in Beta 4, but it works in Linux So, it's getting there!

@MaximoTrinidad why is it not working on Windows? Did you find the root cause? We would like to know if there is still a problem.

@daxian-dbw I'm getting the same error with smo. (I have SQL Server 2014 express with management studio installed)
See below:


Support 1-1 relationships

Right now Backpack does support 1-1 relationships. It's generally preferred to NOT move columns away from the parent table, because it saves a JOIN, but some people prefer it, because it's more object-oriented.
Since Laravel supports them, Backpack should too.
Here's one solution for it: https://gist.github.com/ChrisThompsonTLDR/c83b21a6f7c9041b680be880c2a21ff0
Next step: figure out best way to do it (syntax) and implement the solution.

I don't understand what's going on 😂

So if you have a model, say Product (id, name, price, discount, author). Instead of storing everything in the "products" table, some prefer to split it up into multiple tables:

products (id, name, author_id)
product_details (product_id, price, discount, description)

It sounds absurd for such a small table, but when one table would have 20-30 columns, it kind of makes sense.
One product has one detail. So it's a 1-1 relationship. Backpack currently has no easy way to handle that, the only way is having mutators for each detail attribute.

Ah okay yes, I understand this now :D

Nice! Almost give up searching on backpack for almost two weeks to find if this is already existed on backpack... haha

Backpack currently has no easy way to handle that, the only way is having mutators for each detail attribute.

Are the mutators you are talking about, eloquent model mutators? or does backpack have its own mutators?
I have a polymorphic relation. Many of my resources have an App\Address. I would like to update the address from the App\User edit page. This issue also includes this situation (polymorphic one to one), correct?

@scottgrayson , yup, we're talking about standard eloquent mutators. And yes, this feature will also fix the polymorphic 1-1). I expect it to be done by the end of the week, as it's quite a bummer I forgot to include the easiest relationship of them all :-)
Cheers!

Hi!
I kind of followed that conversation and have a similar problem where I am not sure of how to solve it...
Imagine a table users (user_id etc) and another one called pictures. Each picture is assigned to a single user and there is no way that a picture is moved to another user. Also there are plenty of pictures for each user, which should be sortable.
So the problem is this - how do I implement a nice, smart way where I as admin can go to the user and then view all pictures assigned to him, and eventually sort them? As I said, there is no use case where the pictures change the user and viewing a list of all pictures does not make any sense...
Any way to solve this? Is this included in the request above?

What is the status of this issue? In december you said it would be there within a week. I am willing to help if you could give an explanation of the current situation.

Hi all, is there any news about this important works??
Thanks!

Bump! I need this!

@tabacitu is there any news? I want to help if possible :)

Any news on this?

Hi, you checkout the pull request for this issue. It's still in review but it's close to being done.

Thank you @tumf87 . Let's close this issue and move the conversation in your PR, if needed.
Cheers!


how about performance comparsion with rapidjson?

Usually, i think rapidjson has the best performance, would you mind comparing your lib with it?
rapidjson
benchmark

I'll add a benchmark at some point, but daw_json_link is over twice as fast as rapidjson see chart
, and glaze is faster than daw_json_link. So, I expect glaze to be multiple times faster.

Wow, amazing, i never tried daw_json_link before, thx for the link.


Add Fluent UI Icons demo page

We need to add a demo page for the Fluent UI Icons.

This issue is duplicated #1412


add API endpoint - /announcements

Announcements
Add API endpoint to manage Portal announcements

Required role of portal-admin to interact with any endpoint

Endpoints

[x] /announcements  (GET, POST)
[x] /announcements/{uuid} (GET, PATCH, DELETE)

Functions

[x] get list of announcements - search by is_active, announcement_type, title (open to anon access)
[x] create a new announcement
[x] get details for a particular announcement (open to anon access)
[x] edit an existing announcement
[x] delete an existing announcement

Fields
--- /announcements
announcement_type:  - string in ["facility", "maintenance"]
button:             - string
content:            - string
display_date:       - string
end_date:           - string
is_active:          - bool
link:               - string
start_date:         - string
title:              - string
uuid:               - string
--- /announcements/{uuid}
announcement_type:  - string in ["facility", "maintenance"]
button:             - string
content:            - string
display_date:       - string
end_date:           - string
is_active:          - bool
link:               - string
start_date:         - string
title:              - string
uuid:               - string

Field requirements by type
Type: facility
announcement_type:  - "facility" (required)
button:             - string (optional - default to NULL)
content:            - string (required)
display_date:       - string (required)
end_date:           - string (optional - default to NULL)
is_active:          - bool (required - default to true)
link:               - string (optional - default to NULL)
start_date:         - string (required)
title:              - string (required)

Type: maintenance
announcement_type:  - "maintenance" (required)
button:             - string (optional - default to NULL)
content:            - string (required)
display_date:       - string (optional - default to NULL)
end_date:           - string (required)
is_active:          - bool (required - default to true)
link:               - string (optional - default to NULL)
start_date:         - string (required)
title:              - string (required)

v1.3.0


[Bug] Synchronization is periodically disabled

🐛 Bug Report

synchronization is periodically disabled, after restarting everything is normalized.
the number of node (miner) blocks exceeds the total height of the network.
Steps to Reproduce

Step 1 #systemctl status aleod-miner
Nov 23 07:02:32 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:02:32.941503Z TRACE snarkos::network::peers: Received 'Ping' from <IP_ADDRESS>:4132
Nov 23 07:05:13 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:05:13.061956Z DEBUG snarkos::rpc::rpc: Received 'getnodestate' RPC request from <IP_ADDRESS>:43276: {"accept": "application/json, text/plain, /", "content-typ
Nov 23 07:05:39 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:05:39.254974Z  INFO snarkos::network::ledger: Disconnecting from <IP_ADDRESS>:4132
Nov 23 07:10:12 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:10:12.813515Z DEBUG snarkos::rpc::rpc: Received 'getnodestate' RPC request from <IP_ADDRESS>:44606: {"accept": "application/json, text/plain, /", "content-typ
Nov 23 07:10:13 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:10:13.663588Z DEBUG snarkos::rpc::rpc: Received 'latestblockheight' RPC request from <IP_ADDRESS>:44708: {"accept": "application/json, text/plain, /", "conten
Nov 23 07:15:12 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:15:12.344656Z DEBUG snarkos::rpc::rpc: Received 'getnodestate' RPC request from <IP_ADDRESS>:46170: {"accept": "application/json, text/plain, /", "content-typ
Nov 23 07:20:13 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:20:13.864962Z DEBUG snarkos::rpc::rpc: Received 'getnodestate' RPC request from <IP_ADDRESS>:47522: {"accept": "application/json, text/plain, /", "content-typ
Nov 23 07:20:14 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:20:14.708686Z DEBUG snarkos::rpc::rpc: Received 'latestblockheight' RPC request from <IP_ADDRESS>:47598: {"accept": "application/json, text/plain, /", "conten
Nov 23 07:21:39 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:21:39.389180Z DEBUG snarkos::rpc::rpc: Received 'getconnectedpeers' RPC request from <IP_ADDRESS>:53162: {"host": "localhost:3032", "user-agent": "curl/7.58.0", "ac
Nov 23 07:21:42 Ubuntu-1804-bionic-64-minimal snarkos[13629]: 2021-11-23T06:21:42.350298Z DEBUG hyper::server::conn::spawn_all: connection error: connection closed before message completed
Step 2 #systemctl restart aleod-miner
step 3 #journalctl -u aleod-miner -f
Nov 23 07:24:10 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:10.127528Z  INFO snarkos::network::peers: Connected to <IP_ADDRESS>:4132
Nov 23 07:24:10 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:10.151727Z TRACE snarkos::network::peers: Received 'PeerResponse' from <IP_ADDRESS>:4132
Nov 23 07:24:11 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:11.214982Z TRACE snarkos::network::peers: Received 'ChallengeRequest-B' from <IP_ADDRESS>:4132
Nov 23 07:24:11 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:11.214997Z TRACE snarkos::network::peers: Sending 'ChallengeResponse-B' to <IP_ADDRESS>:4132
Nov 23 07:24:11 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:11.239875Z DEBUG snarkos::network::peers: Received a connection request from <IP_ADDRESS>:59262
Nov 23 07:24:11 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:11.239887Z TRACE snarkos::network::peers: Sending 'ChallengeRequest-A' to <IP_ADDRESS>:59262
Nov 23 07:24:11 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:11.239906Z TRACE snarkos::network::peers: Dropped prior to challenge request of <IP_ADDRESS>:59262
Nov 23 07:24:11 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:11.280365Z TRACE snarkos::network::peers: Received 'ChallengeResponse-A' from <IP_ADDRESS>:4132
Nov 23 07:24:11 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:11.308150Z  INFO snarkos::network::peers: Connected to <IP_ADDRESS>:4132
Nov 23 07:24:14 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:14.292674Z DEBUG snarkos::network::ledger: Peer <IP_ADDRESS>:4132 is at block 10419 (common_ancestor = 10389)
Nov 23 07:24:19 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:19.798897Z TRACE snarkos::network::peers: Received 'PeerResponse' from <IP_ADDRESS>:4132
Nov 23 07:24:19 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:19.848848Z TRACE snarkos::network::peers: Received 'PeerResponse' from <IP_ADDRESS>:4132
Nov 23 07:24:19 Ubuntu-1804-bionic-64-minimal snarkos[12624]: 2021-11-23T06:24:19.848864Z TRACE snarkos::network::peers: Received 'PeerResponse' from <IP_ADDRESS>:4132

Expected Behavior

(Write what you expected to happen here)
Your Environment

snarkOS Version 2.0.0

rustc 1.56.1

Ubuntu 18.04.6 LTS

Can you please specify the exact nature of the issue you're experiencing? Also, the node has undergone significant updates in the last 2 weeks; is this still a problem?

Closing this issue due to inactivity and since syncing issues have not reported by anyone else in the meantime.


Fix 4.0.3

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

fixes #8
fixes #6
fixes #4
fixes #2

@tischi looks good now!


Want to have record set limit column in aws_route53_zone table

Is your feature request related to a problem? Please describe.
I'm using the steampipe as Grafana datasource for alerting.
Would be helpful if it had the column.
Describe the solution you'd like
I wrote simple PR here
https://github.com/turbot/steampipe-plugin-aws/pull/1969
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Addressed in this PR https://github.com/turbot/steampipe-plugin-aws/pull/1969


Update meson to 0.60.1

rename tool-meson to vcpkg-tool-meson. (deprecate old port or to be more precise use for testing of vcpkg_find_acquire_program(MESON))
make ports explicitly depend on vcpkg-tool-meson if they require meson.
make it so that no future changes to vcpkg_find_acquire_program are necessary if meson gets updated. (Will reduce the number of CI rebuilds.)

I expect ports to break with this because meson updates break stuff even more than cmake updates

@BillyONeal  can you say something about the v8 failure. Is that a baseline failure?

opened https://github.com/mesonbuild/meson/issues/9618 because they regressed my previous glib fix for libintl.

@BillyONeal can you say something about the v8 failure. Is that a baseline failure?
probably baseline see #21620

Indeed; I was forced to engage Twitter Driven Development https://twitter.com/sokac/status/1463182302679220230

I believe Google folks fixed v8

@BillyONeal:
hpx on linux:
-- Could NOT find TCMalloc (missing: TCMALLOC_LIBRARY TCMALLOC_INCLUDE_DIR)
CMake Error at cmake/HPX_Message.cmake:51 (message):
  HPX_WITH_MALLOC was set to tcmalloc, but tcmalloc could not be found.
  Valid options for HPX_WITH_MALLOC are: system, tcmalloc, jemalloc,
  mimalloc, tbbmalloc, and custom
Call Stack (most recent call first):
  cmake/HPX_SetupAllocator.cmake:47 (hpx_error)
  CMakeLists.txt:2166 (include)

looks like a baseline regression?

@BillyONeal: hpx on linux: -- Could NOT find TCMalloc (missing: TCMALLOC_LIBRARY TCMALLOC_INCLUDE_DIR)
CMake Error at cmake/HPX_Message.cmake:51 (message):
  HPX_WITH_MALLOC was set to tcmalloc, but tcmalloc could not be found.
  Valid options for HPX_WITH_MALLOC are: system, tcmalloc, jemalloc,
  mimalloc, tbbmalloc, and custom
Call Stack (most recent call first):
  cmake/HPX_SetupAllocator.cmake:47 (hpx_error)
  CMakeLists.txt:2166 (include)

looks like a baseline regression?

We did just remove hpx from ci.baseline.txt here but that was because it passed in CI (it got PASSING: REMOVE FROM FAIL LIST). Perhaps it's order dependent?

I'm able to repro the hpx failure locally, stand by...

https://github.com/microsoft/vcpkg/pull/21673

the gstreamer regression also seems like baseline:
LINK : fatal error LNK1181: cannot open input file 'm.lib'
, {"name": "flac"
, "version": "1.3.3"
, "compile_args": ["-ID:/installed/x64-windows/include"]
, "link_args": ["D:/installed/x64-windows/lib/FLAC.lib"
, "D:/installed/x64-windows/lib/ogg.lib"
, "m.lib"]}

gst-plugins-good| Dependency flac found: NO unknown version, but need: ['>=1.1.4']
gst-plugins-good| Run-time dependency flac found: YES 1.3.3

so its an issue in libflac. fixed it manually.

The hpx regression should not related to this PR.

The osx failure ran on the one we suspect is unhealthy from https://github.com/microsoft/vcpkg/pull/21676#issuecomment-982010390 but I'm not resubmitting because it looks like there are version DB updates needed. I've removed -06 from the fleet.

hmm the formater in vcpkg-tool changed?

hmm the formater in vcpkg-tool changed?

https://github.com/microsoft/vcpkg-tool/pull/282 may have had an effect, yes

I'm merging this despite the qwt-qt6 failure because that port is currently broken and being backed out https://github.com/microsoft/vcpkg/pull/21825

Thanks for the update!


[INTEL] Removed ScanOpToLLVM.cpp intel backend file

The ScanOpToLLVM.cpp in the intel backend was identical to the upstream triton version, so this PR deletes the file and replaces its use with the common version

Not possible to remove due to use of Intel Specific function getSmemBases() in ReduceScanCommon.h


[MKL-DNN] Enabling Winograd algorithm for AVX 512?

Hi, I am trying MXNet + MKL-DNN on an AVX 512 capable machine.
Looking at the code in mkldnn_convolution.cc, I'm assuming that MXNet always choose a direct algorithm for mkl-dnn convolution (by specifying  mkldnn::algorithm::convolution_direct). But MKL-DNN implements Winograd algo for AVX 512, and it should be much faster than direct algo when applicable.
Why not enable it if there is a support for AVX 512?  Complication are that we need to check cpu feature
and that winograd algo is applicable for only certain filter shape.  Ideally, MKL-DNN should choose the best algo automatically so that MXNet doesn't need to specify what algo to use. @pengzhao-intel @TaoLv

It's nice that you are using MKL-DNN backend :)
Winograd outperforms the direct convolution for small filter size and we have the plan to enable it in the next step.

Thanks, let me keep this thread open to track progress.

@masahi Thanks for asking that. Since winograd may impact the numerical stability and model accuracy, so I don't think it can be chosen by library implictly. Besides, Winograd doesn't always outperform direct convolution in MKL-DNN. It depends on the implementation and kernel size/shape. To make this feature request more clear, could you also explain what does you model look like and what's you expectation from MKL-DNN Winograd?

@TaoLv Good point on numerical accuracy of winograd. Performance - accuracy trade off is an interesting topic. Since you already have int8 convolution (which trades accuracy for performance way more than winograd), it also makes sense to make winograd available so users can opt-in.
I want to test performance of MKL-DNN AVX 512 winograd to compare with my home grown winograd implementation written in TVM.  I can't talk about my architecture, but you can imagine a network where almost all convolutions are 3 x 3, similar to VGG.
I tested my network on both MXNet + MKL-DNN and TVM + NNVM. At the moment, the TVM compiled model with winograd convolution is way faster than MXNet + MKL-DNN with AVX 512 direct convolution. I expect big speed up from MXNet by enabling mkl-dnn winograd.

@TaoLv Good point on numerical accuracy of winograd. Performance - accuracy trade off is an interesting topic. Since you already have int8 convolution (which trades accuracy for performance way more than winograd), it also makes sense to make winograd available so users can opt-in.

I meant MKL-DNN shouldn't "choose the best algo automatically" because of the accuracy issue. At least there should be an environment variable to enable it at framework level. INT8 is also enabled by the user explictly and it need be tuned to exclude some layers for accuracy. If we agree that winograd cannot be applied to all convolution layers, this kind of tuning should also be applied at model level.

I tested my network on both MXNet + MKL-DNN and TVM + NNVM. At the moment, the TVM compiled model with winograd convolution is way faster than MXNet + MKL-DNN with AVX 512 direct convolution. I expect big speed up from MXNet by enabling mkl-dnn winograd.

Do you have any chance to compare TVM and MKL-DNN winograd at kernel level? Maybe you can try benchdnn for that.

Yes, something like an environment variable to enable winograd is also what I had in mind.
Thanks for suggesting benchdnn. I'll try it.

@mxnet-label-bot [Question]


feat: port Data.Set.Pairwise

We have a race condition between two PRs, this and #1176: https://leanprover.zulipchat.com/#narrow/stream/287929-mathlib4/topic/Data.2ESet.2EPairwise

Thanks for making those changes.

bors merge


Refactor role policies, and create multiple "aws_ssoadmin_customer_managed_policy_attachment" statements.

User Story
As mod platform user
When new permissions are requested, adding them to a role saturated in permissions hits a limit, they need to be split
So that we can continue to add permissions and manage them in a better fashion.
Value / Purpose
We can manage permissions in our roles correctly.
Useful Contacts
Edd Proctor
Additional Information
Step 1 -
Review our roles and policy attachments. Are there any that could be reduced, so that we have less to manage, could some be built up of pre existing ones? Please see this doc
https://docs.google.com/document/d/1vYCAtQpHgBBSMXgs7--uTwV7idt3cI3HRTZ1kKrZMCQ/edit?usp=sharing
We are going with option 1, but the graph might help you on this step.
Step 2 - Attach multiple customer policies to a role, as shown in step 2. Easiest step will be to have the common statements attached to each role as another statement, so it can be taken out of our other policy statements.
Please check plans, and think of any impact to users, test in sprinkler.
Definition of Done

[ ] Documentation has been written / updated on roles / policies
[ ] Informed users of the change.
[ ] Another team member has reviewed
[ ] Test changes in sprinkler
[ ] Deployed

The reason this should work - If you're hitting the limit on an IAM role's policy size, you can create multiple IAM policies, each covering a subset of the permissions, and then attach those policies to the role. This way, you can effectively distribute your permissions across multiple policies.

I think the idea on this one was that you would create individual policies for the "common statements" common to all of our current roles and then individual policies that contain the specific delta for each specific role.
So looking at Ed's doc I assume we'd have a policy for each of these...

Common Statement - Contains basic permissions required in almost every role.
Developer - Badly named, but consists of common statements and developer additional statements.
Data Engineering - Build from developer additional and its own. 3 layers
Sandbox - Uses sandbox additional statements, which is built from common statements, bedrock statements, and its own.
Migration - Built from common, developer additional, and its own.
Instance Management -
Reporting Operations -
PowerBi - Common statements plus more
Fleet manager- Common Statements plus more

Then you would attach the various combinations of these policies to the specific roles (each role would get the "common statement" policy).

PR created and awaitiung review deployed in sprinkler and have checked that no access has been lost
https://github.com/ministryofjustice/modernisation-platform/pull/6742


Direct URL constraints are ignored

It seems that uv does not honor constraints that are Direct URLs.
Assuming you have a direct URL for pkga in constraints.txt, uv pip install pkga -c constraints.txt fails saying pkga is not found in the package registry.
Here is a reproducer:
#/bin/bash

cd $(mktemp -d)

mkdir pkga
cat <<EOF > pkga/pyproject.toml
[project]
name="pkga"
version="1.0"
EOF

cat <<EOF > constraints.txt
pkga @ file://$PWD/pkga
EOF

uv venv
source .venv/bin/activate

uv pip install pkga -c constraints.txt

Thanks! Will fix.

Sorry about this, silly oversight in the code. Will go out in the next release, thanks!

That was quick, thanks!


Add Python 3 support for the python scripts

This patch adds a new file called "python_compatibility_utils.py"
which contains the imports and methods that work on both
Python 2 and 3. It exposes the same name for functions and modules,
making it easy to use in other parts of the code.
It also adds brackets to the message from print function as that
is mandatory in Python 3.
Adds init.py file to be able to import from current directory
without adding the current directory to system path.

LGTM


evdev keyboard mapping broken

freerdp_keyboard_init tries to detect the keyboard layout and create a reverse map to get a scancode for a keypress.
when WITH_XKBFILE is not detected or fails, the detection falls back to freerdp_keyboard_init_x11_evdev.
this mapping is broken.
easily tested with a build that does not have libxkbfile-dev installed at compile time.

Yes, we meet the same problem.
freerdp_keyboard_init_x11_evdev -> GetVirtualKeyCodeFromKeycode, parameter should use WINPR_KEYCODE_TYPE_XKB instead of WINPR_KEYCODE_TYPE_EVDEV , can get correct result.


Altaria EX check fix

it.EX is for pokemon ex it.POKEMON_EX is for pokemon EX. Dont you love the naming conventions?

yeah this tripped me up when I started working on ex sets.  I forgot to go back and change the initial sets I worked on


Define nodePackages as an extensible fixed point

We also make the generatePackages utility function overridable with makeOverridable so that users can supply their own overrides following a similar convention to that of other language package sets.
This change should be backwards compatible and we also honor the extensions argument with the caveat that anything provided via overrides will override packages given by the extensions argument to generatePackages.
Usage example
An example of what this change allows us to do, assuming we have generated a package set and that one of the npm packages in this set is named to-array. We also assume that the generated package for to-array is for the 0.1.4 version but what we really want is the 0.1.3 version. Therefore we want to override the to-array package and we also want all instances of that package as a dependency to be updated too.
Thus, we have an extensible overrides mechanism that also resembles what we see in other language-specific package sets:
# We are in the root of the output directory for `nixfromnpm`
$ tree .
.
├── default.nix
├── nodeLib
│   ├── buildNodePackage.nix
│   ├── circular_dependencies.md
│   ├── default.nix
│   ├── fetch.py
│   ├── fetchUrlNamespaced.nix
│   └── tools
│       ├── check-package-json
│       ├── install-binaries
│       └── patch-dependencies
├── nodePackages
│   ├── default.nix
│   ├── smallest
│   │   └── 1.0.1.nix
│   └── to-array
│       └── 0.1.4.nix
└── release.nix

5 directories, 13 files

... where release.nix is:
let

  pkgs = import <nixpkgs> { };
  generatedPackageSet = import ./default.nix { };

in

  generatedPackageSet.override {
    overrides = nodePackagesNew: nodePackagesOld: {
      to-array = nodePackagesOld.to-array.overrideNodePackage (oldAttrs: {
        version = "0.1.3";
        src = pkgs.fetchurl {
          url = "https://registry.npmjs.org/to-array/-/to-array-0.1.3.tgz";
          sha1 = "j54lzfyxxi8flks7507zc5rl6v3aspfl";
        };
      });
    };
  }

... we can see the overridden package is the result of building the nodePackages.to-array attribute:
$ nix-build --no-out-link --attr nodePackages.to-array ./release.nix
...
/nix/store/9zxzpmfm6knamb4k628b75z155kk9nw0-to-array-0.1.3-nodejs-8.9.4

Please note, in this usage example, release.nix was added in order to provide the example but this patch contains no change to nixfromnpm to generate such a file.

CC: @intractable @Gabriel439

I’ll need to look it over in more detail but I like the idea here!

This looks like a positive change. The only thing is If we're deprecating extensions passed in via a list, then we should update the Haskell code which generates the nix expressions for extensions to use the proper logic.

I figured we could do that in a follow-on PR that introduces the breaking change. As it stands, the extensions are still used so this is still backwards compatible with that feature while introducing the change to represent the package set as a fixed point that can be overridden downstream of generation (so, it's a nice rather than abrupt transition).

@adnelson thank you for the merge! We already use nixfromnpm to build pulp which we use to build purescript packages but now we'll be able to use it to build our entire front-end user interface (which is a mix of npm and purescript)!

@adnelson I think I got confused in my branches and testing somewhere; I broke this. I am fixing it right now (and tested). Sorry!

#98 addresses the issue I referenced above.


Discord link doesnt work

The discord link will not work. could you help me out by getting me a link please?

https://discord.gg/cMGhNtZ3tW
Need to fix the ones in the readme!

Thank you!


docs(2.4.0) regenerate admin api docs merged with declarative config

Review
New version of the API docs where db-less-admin-api is merged into admin-api.
Summary
This was done by creating a custom temp branch from the 2.4.0 tag in the kong repo and cherry-picking the changes in the Admin API generation docs.

Please hold from merging this pr until the details are ironed out in #2951

@HCloward I have backported the changes done from #2951 into this branch, in order to also unify the Admin API for db/db-less for the 2.4.0 docs. Please review


restoring trained or pretrained models blocked...

cd $PATH_ROOT/tools/r3det/ && python test_dota.py --test_dir='/PATH/TO/IMAGES/'  --gpus=0
blocked when restore trained model ,(python3.6,tensorflow-gpu==1.13.0,CUDA10.0)
necessary trained model has been downloaded and  put in  $PATH_ROOT/output/trained_weights/ ,and $PATH_ROOT/libs/configs/cfgs.py  has been replaced and updated
help!


Install noobaa CLI as part of dev-env

Allows to install "noobaa" CLI as part of the dev-env, which is needed for the S3 bucket testing and development.
@redhat-cop/mdt

/lgtm

/hold

/LGTM

/retest

ci failures look unrelated

/retest

ci failures look unrelated

Yes, the CI fails on failure-exporter test:

curl failure-exporter-pelorus.apps.ci-op-ywvk4bpc-c8f87.origin-ci-int-aws.dev.rhcloud.com 2>&1 | grep todolist || exit 2

interesting, ran from my local system it works just fine:
pod/todolist-1-fl5kr condition met
+ set +x
committime-exporter   committime-exporter-pelorus.apps.cluster-mpryc1502ocp49m.mpryc1502ocp49m.mg.dog8code.com          committime-exporter   http                                 None
deploytime-exporter   deploytime-exporter-pelorus.apps.cluster-mpryc1502ocp49m.mpryc1502ocp49m.mg.dog8code.com          deploytime-exporter   http                                 None
failure-exporter      failure-exporter-pelorus.apps.cluster-mpryc1502ocp49m.mpryc1502ocp49m.mg.dog8code.com             failure-exporter      http                                 None
commit_timestamp{app="todolist",commit="d756bbe5e65c915bb01d576d9582d6abdc84cf8b",image_sha="sha256:5d1175f428e649f6b1be8c002d6bd8226484938381209e0c1a66513aefe2c1b1",namespace="mongo-persistent"} 1.655302899e+09
deploy_timestamp{app="todolist",image_sha="sha256:5d1175f428e649f6b1be8c002d6bd8226484938381209e0c1a66513aefe2c1b1",namespace="mongo-persistent"} 1.658143907e+09<PHONE_NUMBER>000
failure_creation_timestamp{app="todolist",issue_number="2"} 1.658142948e+09
Cleaning up temporary files
Exit code: 0

/retest


Support multiple antiAffinity statements

What is missing?
We can't currently add affinityTerms using the provided "antiaffinity" function in addons/ because the function doesn't use the +: notation on the keys it creates. The podAntiAffinityTopologyKey key in values also only supports one topology key.
Why do we need it?
We would like to add an antiAffinity rule for zone, as well as hostname so that prometheus, alertmanager, etc. instances are put onto different hosts in different zones (the hostname rule would be a fallback if the zone rule was unable to be carried out).
Environment

kube-prometheus version:
dff8d8c11ccace01468ba7c4bae25784e5113b45

Anything else we need to know?:

I agree it would be good to improve this addon. Do you want to contribute a fix?

Not familiar with jsonnet yet but if you can give some starting points I can look at it :)


Ignore D203 by default

Pydocstyle ignores D203 by default, because it's in conflict with D211, see https://github.com/PyCQA/pydocstyle/pull/137/files.
Pylama does not seem to keep this behaviour, I had to explicitly ignore D203. Should D203 be ignored by default also in Pylama?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

There does not seem to be much interest for this, therefore I am gonna close this in a few days, unless someone has news about it :slightly_smiling_face:


Fixed the event card descriptions links outside card on mobile

issue #5

This can't merge into development because it's a change based on an old commit of the main branch. Your overflow-x-hidden cursor-pointer code needs to go into the app/events/eventList.tsx file on the development branch.
Thank you very much for all of the merges though, given me a lot of cues for how to write the contributor guidelines when I come to make them for this repository. Sorry about the rough experience you've had so far.
If you'd like I can set up a new pull request with your code in it for you? Am happy to take this from here if it's a little to much to revise this again?

@CanopusFalling okay


doc: Update architecture documentation and others

Closes #326

I spoke with @simanadler. We'll postpone the ingestion documentation to a later PR.


Remove declared Python 3.4 support

The major version bump should be a fine time to do this.

Plan to leave the Python version constraint such that v2.0 will allow installation on 3.4, and to leave 3.4 in the tox matrix as long as pytest & other tooling allows.
As soon as testing on 3.4 is no longer practical, or code changes lead to observable 3.4 incompatibility, will need to update python_requires to strictly forbid installation there, prior to pushing a new release to PyPI.

I would appreciate keeping Python 3.4 support as long as possible.
The lack of py27 already causes problems with adoption by tools like seen at https://github.com/kvas-it/pytest-console-scripts/issues/24 .  I would hate to loose more potential uses because of py34 being dropped.  It was the stable v3 release for a very very long time.
I am hoping to also use this for @coala , which still supports Python 3.4, likely until we push out a much delayed release. I'll be happy to assist if Python 3.4 is causing any troubles.
I am quite sure we will find some features not possible with 3.4.  https://github.com/bskinn/stdio-mgr/pull/53 even has problems with Python 3.5.

Nah, I'm good to keep 3.4 officially supported for v2.0...pytest and pip have started complaining about it but seem to work fine, still. Worst case we can just pin a couple of things in the development environment for this release.
I've been continuing to muse on a 2.7 backport...I'd really rather not muddy master with 2/3 compact, but a backport branch would be fine. I figure it's an open question what of the new 2.0 functionality is actually practical in 2.7...though maybe more of it is than I think?

Mmmm...but a separate backport branch makes bringing any new features back to py2 require 2x the effort, always...whereas 2/3 compat on master would be a lot easier.
I guess if there will be a stairstep of supported features along 3.4/3.5/3.6...what harm in tagging 2.7 on the bottom rung?


Using Dataset.map with python dicts as dataset elements passes Tensors without numpy values to the mapping function

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): YES
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: n/a
TensorFlow installed from (source or binary): Pycharm, pip-install
TensorFlow version (use command below): v2.0.0-rc1-51-g2646d23074 2.0.0-rc2
Python version: 3.6
Bazel version (if compiling from source): n/a
GCC/Compiler version (if compiling from source): n/a
CUDA/cuDNN version: n/a
GPU model and memory: n/a

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"
Describe the current behavior
I'm doing a simple pipeline where I load a dataset from a csv file, convert the dataset to a dict structure (where one element of the dataset is a dict with strings as keys and tf.Tensor as values) and then try to use this resulting dataset. The issue is that, when I create the dict-based dataset, I can see that the numpy values are there in the Tensors, BUT when I use Dataset.map to apply a function to the dict-based dataset, I can't access the numpy values inside this function. The numpy values ""disappear"".
Describe the expected behavior
I should be able to see and use the numpy values.
Code to reproduce the issue
IMPORTANT: replace the TODO in the beginning of the code with the path that the csv I provided after the code, below, will have in your system.
import os
import tensorflow as tf

#TODO REPLACE THE LINE BELOW BY ONE THAT DESCRIBES THE PATH OF THE CSV IN YOUR SYSTEM
data_source = os.path.join(".", "Data", "mini.csv")

#Help function to convert tuples of Tensors to dicts, so I have the column names from the csv as keys
def _convert_to_dict(*el):
    base_list = ["var1", "var2", "var3"] #as in the csv header
    dicto = dict()
    for i in range(len(base_list)):
        dicto[base_list[i]] = el[i]
    return dicto

#Simple function which will try to investigate and use the numpy values inside the Tensors in the datapoints
def _use_dict(dicto, key):
    """
    dicto: dict with string keys and Tensors as values
    :param key: a specific string key
    :return:
    """
    print("Printing inspections from _use_dict function")
    print(dicto) #Here we print the dataset element passed to the function, which is a dictionary, and see that no numpy value is associated with the Tensors
    print(dicto[key]) #We look closer, no numpy values
    print(dicto[key].numpy()) #No numpy value, so we get an exception here
    return None

def load_and_preprocess_dataset(): #Main function which will load the dataset from csv and transform to a dict-like structure
    #Load the dataset from CSV
    dataset = tf.data.experimental.CsvDataset(
        header=True,
        filenames=data_source,
        record_defaults=[
            tf.int64,
            tf.string,
            tf.float64,
        ]
    )
    #Convert each element from the dataset from a tuple of Tensors to a dict with keys being the csv header values, and values being the corresponding Tensors
    dataset = dataset.map(map_func=_convert_to_dict)
    #Here we inspect the values of the dataset, and observe that the Tensors contain numpy values
    print("Printing dataset elements after transforming to dict-like structure")
    for a in dataset:
        print(a)
    #Here we try to apply a function which accepts a dict-like structured dataset element as input
    dataset = dataset.map(map_func=lambda x: _use_dict(dicto=x, key="var1"))
    return dataset

if __name__ == '__main__':
    dataset = load_and_preprocess_dataset()

Csv file content:
var1,var2,var3
2,"foo",1.3
3,"bar",1.5
4,"barfoo",1.8

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
Stack trace:

2019-09-26 16:49:14.948332: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Printing dataset elements after transforming to dict-like structure
{'var1': <tf.Tensor: id=27, shape=(), dtype=int64, numpy=2>, 'var2': <tf.Tensor: id=28, shape=(), dtype=string, numpy=b'foo'>, 'var3': <tf.Tensor: id=29, shape=(), dtype=float64, numpy=1.3>}
{'var1': <tf.Tensor: id=30, shape=(), dtype=int64, numpy=3>, 'var2': <tf.Tensor: id=31, shape=(), dtype=string, numpy=b'bar'>, 'var3': <tf.Tensor: id=32, shape=(), dtype=float64, numpy=1.5>}
{'var1': <tf.Tensor: id=33, shape=(), dtype=int64, numpy=4>, 'var2': <tf.Tensor: id=34, shape=(), dtype=string, numpy=b'barfoo'>, 'var3': <tf.Tensor: id=35, shape=(), dtype=float64, numpy=1.8>}
Printing inspections from _use_dict function
{'var1': <tf.Tensor 'args_0:0' shape=() dtype=int64>, 'var2': <tf.Tensor 'args_1:0' shape=() dtype=string>, 'var3': <tf.Tensor 'args_2:0' shape=() dtype=float64>}
Tensor("args_0:0", shape=(), dtype=int64)
Traceback (most recent call last):
  File "...temp.py", line 50, in <module>
    dataset = load_and_preprocess_dataset()
  File "...temp.py", line 46, in load_and_preprocess_dataset
    dataset = dataset.map(map_func=lambda x: _use_dict(dicto=x, key="var1"))
  File "...site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 1211, in map
    return MapDataset(self, map_func, preserve_cardinality=True)
  File "...site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 3416, in __init__
    use_legacy_function=use_legacy_function)
  File "...site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 2695, in __init__
    self._function = wrapper_fn._get_concrete_function_internal()
  File "...site-packages\tensorflow_core\python\eager\function.py", line 1854, in _get_concrete_function_internal
    *args, **kwargs)
  File "...site-packages\tensorflow_core\python\eager\function.py", line 1848, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "...site-packages\tensorflow_core\python\eager\function.py", line 2150, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "...site-packages\tensorflow_core\python\eager\function.py", line 2041, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "...site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "...site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 2689, in wrapper_fn
    ret = _wrapper_helper(*args)
  File "...site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 2634, in _wrapper_helper
    ret = autograph.tf_convert(func, ag_ctx)(*nested_args)
  File "...site-packages\tensorflow_core\python\autograph\impl\api.py", line 237, in wrapper
    raise e.ag_error_metadata.to_exception(e)
AttributeError: in converted code:
    relative to ...:

    temp.py:46 None  *
        dataset = dataset.map(map_func=lambda x: _use_dict(dicto=x, key="var1"))
    temp.py:25 _use_dict  *
        print(dicto[key].numpy()) #No numpy value, so we get an exception here

    AttributeError: 'Tensor' object has no attribute 'numpy'

Process finished with exit code 1

I wonder if it is related to the closed (but maybe not solved?) issue https://github.com/tensorflow/tensorflow/issues/27519

Try this on top-
tf.enable_eager_execution()

@Efaq ,
Can you try @AkashNagaraj solution? Thanks!

@AkashNagaraj @oanush Tried it now, and I'm getting:
Traceback (most recent call last):
  File "...", line 3, in <module>
    tf.enable_eager_execution()
AttributeError: module 'tensorflow' has no attribute 'enable_eager_execution'

Process finished with exit code 1

I'm in Tensorflow 2.0 rc2, so I supposed the eager execution would be enabled by default. Shouldn't it be the case?
Also, tf.executing_eagerly() returns True, what would make me believe that I'm already in eager execution.
Thank you for the help!

@oanush just noticed that you labeled this issue as rc0, while I have the version v2.0.0-rc1-51-g2646d23074 2.0.0-rc2

I was able to replicate the issue for TF-2.0rc2 and TF-nightly-2.0-preview, kindly find the gist of colab.Thanks!

@oanush just noticed that you labeled this issue as rc0, while I have the version v2.0.0-rc1-51-g2646d23074 2.0.0-rc2

@Efaq ,
We just have 2.0rc0 label available now as the latest version in Github.

@oanush alright!

This is expected behavior. tf.data traces all of its user-defined functions and data objects observable by user-defined functions will be Tensors and not EagerTensors (and Tensors do not have .numpy()). If you would like to execute (a subset of) tf.data user-defined function in eager mode, use py_function. Note that the use of py_function has limitations, including no parallel execution (because of GIL) and precluding TensorFlow from applying static optimizations to your program.

@jsimsa Alright, thanks! Could you please exemplify how to change the code provided in order to correctly run eagerly?
I was really expecting that TF 2.0, when saying it has eager execution by default, would really execute the whole code eagerly, and not only parts of it...

@Efaq How did you resolve this issue?

@AlbinDavid I haven't. Asked some extra help from @jsimsa above but haven't gotten an example.
More specifically, after 2 years dealing with Tensorflow and more recently with Tensorflow Extended, I changed my attention to scikit-learn and pure Keras and I have never been happier. So if you are not tied to very high performance, I would recommend considering the same.
Cheers

@Efaq thanks for your response

Following code worked:
def parse_str(str_tensor):
    raw_string = str_tensor.numpy().decode("utf-8")

    # play with raw string
    raw_string = 'AAA'+raw_string
    return raw_string

Call parse function:
def tf_pre_processing(row):
  return tf.py_function(parse_str, [row['context']], [tf.string])

train = t.map(tf_pre_processing).batch(1).take(1)

list(train)


Year selector calling "shouldDisableDate"

Environment

Tech
Version

@material-ui/pickers
v4-0-0-alpha-8

material-ui
^4.10.1

TypeScript
^3.8.3

React
"^16.13.1"

Browser
any

Peer library
date-fns

Steps to reproduce

Create a custom function to disable weekends, for example

  function shouldDisableDate(day, pickerProps) {
    return day.getDay() === 0 || day.getDay() === 6;
  }

And call it in any datepicker.
This function is being called at the year selector, disabling years that should not be disabled

Expected behavior
Year selector screen should only call shouldDisableYear right?
Actual behavior

https://codesandbox.io/s/cranky-black-iuqu4?file=/src/App.jsx:263-368
Couldn't get it running, but you can see the code

Related to #1774 and #1002.

In short use shouldDisableYear

I don't know if I was clear, in short, the year picker is calling shouldDisableDate. It shouldn't right?
It should only take shouldDisableYear wright?
Sorry about my english :T

@mauriciocarnieletto Yes, I would only expect shouldDisableYear to be called in your context.

Here is an updated reproduction that run: https://codesandbox.io/s/nervous-drake-85kb1?file=/src/App.jsx.

This bug appeared by the fix of #1774. Going to revert changes from #1884


Updated readME file

Appreciate the help with this @pranavishere2. I think this is now fixed by #58, so closing.


Port menus and menu bar to React components

This is a fairly large PR, but the basic gist of this is that the menu bar and its menus are ported to React components. This is a continuation of work in the prototype started in PR https://github.com/codeforamerica/streetmix/pull/569, and influenced by the conversation in PR https://github.com/codeforamerica/streetmix/pull/563, in which we wanted to reduce the amount of imports that create side effects, plus all of the other benefits we gain from having UI organized as React components. (Read https://github.com/codeforamerica/streetmix/pull/569 for more.)
Some of the diffs here are because this branch is based off of existing (and currently open) PRs https://github.com/codeforamerica/streetmix/pull/576 and https://github.com/codeforamerica/streetmix/pull/577. If those are accepted and merged, I will rebase this PR and that will remove some of the diffs that are not needed to be here. In the meantime, this PR is opened for discussion.
All existing functionality in the menu and menu bar are re-implemented as React components. This means that all HTML (written as JSX) is created directly in the component, removing the need to split the view between Jade templates and JavaScript DOM manipulation. Event handlers are added directly to components, removing the need to add event handlers during the app initialization phase. References to menu elements, and state (which one is active, etc) are controlled in the React component ecosystem, meaning less of the other logic elsewhere in the app need to know anything about menus or menu content.
Furthermore, this allowed me to continue to fix a problem with the previous way of doing translation which is that HTML tags in text strings could not be appended as actual HTML. This illustrates the difference before and after:

Furthermore, text for "Sign in" and the keyboard instructions in the Help menu are now translated, when available. The Pirate English language was removed.
You can test out the new menu bar and make sure everything is okay. This branch is deployed to staging and can be accessed at https://streetmix-staging.herokuapp.com/-/40?debug-experimental (don't forget the ?debug-experimental flag), and compare with something on the production server, e.g. http://streetmix.net/-/421124?debug-experimental (same flag applies).
Would appreciate thoughts and review. cc @magul @treyhahn @ryross

I made some inline comments but in general this looks great!

Thank you! Follow up responses also inline.

I've rebased this PR after merging #576, #577, and #581.

Since there aren't any further comments on this branch, I will be merging today! After adding in Chinese to the translations menu.

And there it is. Looks amazing. Thanks for facilitating, @treyhahn

Thanks to Wen Jing! She did a great job.


Quickview arrow navigation

Describe the bug
Quickview not following selection navigation.
To Reproduce
Steps to reproduce the behavior:

while navigating folders with quickview and keyboard arrows, the mini window doesn't follow selection as in default explorer.
you must close the preview with spacebar and press again to preview the new file.

Expected behavior
The preview follows arrow navigation selection
Desktop (please complete the following information):

OS Version: [e.g. Windows 10 2004 19041.508]
App version: [v<IP_ADDRESS>]

Can you confirm if #792 is the same issue?

@yaichenbaum I think it's more likely a duplicate of #1373

@generalguy41 you're right it is.


Feature Request: Some form of path auto-completion

What is your issue?
Copying over from https://github.com/xarray-contrib/datatree/issues/189
Feature Request: Some form of path auto-completion
The more I play with datatree, the more I realize the huge potential of this package. Amazing work @TomNicholas.
I have been using dt for CMIP6 analysis and one of the things that is still fairly slow for me is to navigate into different specific leaves of the tree. Having some sort of incremental auto-complete of the path like this:
dt.['model/<click tab> --> dt.['model/experiment_a/]

similar to what happens in the bash shell with tab completion would be an absolute killer feature. Most users already know how to navigate this (from their terminal). For massive trees this would speed up the workflow tremendously.
On the other hand I have no idea how complex this is to implement. Just wanted to suggest this feature.

To clarify what's said in https://github.com/xarray-contrib/datatree/issues/189 - autocompletion is already implemented, what is not implemented is autocompletion of any path containing a . or ../.
In other words this issue now tracks this ToDo in the code:
https://github.com/pydata/xarray/blob/a74a60508edd844ef425637f27598f1d8c5385a2/xarray/core/datatree.py#L678


add path normalization

As mentioned in https://github.com/juliangruber/hyperdrive-import-files/pull/23#issuecomment-253499239, it would make sense to me to fully treat entry.name as a path and thus also apply normalisation if necessary. Currently it's only treated as a String, and eg there could be two entries /entry and ./entry. This should help the modules depending on this to simplify their path logic.

great idea. do you know a module that helps with this?

this seems to work well:
const normalize = p => path.resolve('/', p)

Normalizes like this:

/ -> /
'./foo' -> /foo
foo/bar -> /foo/bar
foo/bar/ -> /foo/bar/

Cool. In addition to that it should normalize windows paths to unix ones. \ --> /


Revert installing Python module systemwide.

The lines
install(
  TARGETS _morphio
  LIBRARY DESTINATION ${Python3_SITEARCH}/morphio/
)

cause CMake to find the directory where Python modules are installed. This would be the systemwide directory if one is simply building with the systemwide Python. If one has a venv active it'll install into that virtual environment.
Note that CMAKE_INSTALL_PREFIX doesn't affect where the module will be installed. Therefore, this will cause build/installation failure.
Installing into a venv is questionable, because that's the responsibility of the Python package manager. Which will do so correctly without the additional CMake  install statement.
Finally, even if the install succeeds, it  only installs the lib*.so without any __init__.py files. Therefore, it leads to a broken installation, e.g.
python -c "import morphio; morphio.Soma"

will successfully import morphio but there won't be any morphio.Soma, because that's handled via __init__.py

Here's the steps to reproduce on BB5:
module load unstable gcc cmake python hdf5
cmake -DCMAKE_INSTALL_PREFIX=build/install -DMorphIO_WERROR=Off -B build
cmake --build build --parallel --target install

which fails with:
CMake Error at binds/python/cmake_install.cmake:60 (file):
  file cannot create directory:
  /gpfs/bbp.cscs.ch/ssd/apps/bsd/2023-02-23/stage_externals/install_gcc-12.2.0-skylake/python-3.10.8-rvn6l5/lib/python3.10/site-packages/morphio.
  Maybe need administrative privileges.
Call Stack (most recent call first):
  cmake_install.cmake:80 (include)

thanks for finding this, and the fix.

(For posterity.)
Here's a variation that seems to show that even if there isn't a permission issue the lines:
install(
  TARGETS _morphio
  LIBRARY DESTINATION ${Python3_SITEARCH}/morphio/
)

are likely not what we expect. Here's the required step on BB5:
module load unstable gcc cmake python hdf5
# Ensure that we have write permissions during installation of `_morphio*.so`.
python -m venv venv
source venv/bin/activate
cmake -DCMAKE_INSTALL_PREFIX=build/install -DMorphIO_WERROR=Off -B build

We can now run and observe:
$ cmake --build build --parallel --target install
...
-- Installing: /gpfs/bbp.cscs.ch/home/groshein/MorphIO/venv/lib/python3.10/site-packages/morphio/_morphio.cpython-310-x86_64-linux-gnu.so

$ ls /gpfs/bbp.cscs.ch/home/groshein/MorphIO/venv/lib/python3.10/site-packages/morphio/
_morphio.cpython-310-x86_64-linux-gnu.so

$ which python
~/MorphIO/venv/bin/python

$ cd # to avoid the folder `morphio`.
$ python -c "import morphio"
$ python -c "import morphio; morphio.Soma"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'morphio' has no attribute 'Soma'
$ pip uninstall morphio
WARNING: Skipping morphio as it is not installed.

To summarize, even if we have permissions, those lines would install a version of morphio which can be imported, but not used. Moreover, it can't be uninstalled via pip essentially breaking our virtual environment.


DS-2952 SOLR full text indexing multiple bitstreams

This is a fix for https://jira.duraspace.org/browse/DS-2952
This fix is different from https://github.com/DSpace/DSpace/pull/1237 because now the code doesn't copy or create any extra files. The approach I took here is to use a SequenceInputStream that will concatenate all input streams of the various bitstreams of the TEXT bundle (containing the extracted full text): http://docs.oracle.com/javase/7/docs/api/java/io/SequenceInputStream.html
I also grouped all logic concerning the retrieval and data extraction of the relevant bitstreams so that this is no longer present in the SolrServiceImpl class (making it a bit smaller). This also made it easier to unit test my changes.
I ran all unit tests, integration tests and license checks.

+1, tested locally and this works as described.  I will add review comments.
@tomdesair , sorry for the delay in getting back to you on this.
While testing this process, I wondered how DSpace should handle full text extraction for bitstreams with different access policies within an item.  Should the full text index process require anonymous read access in order to be added to the index?

Sorry for the late reply, but I'm still on paternity leave.
This PR kept the original behaviour: Index all bitstreams in the TEXT bundle.
I understand the need to take into account any resource policies on the ORIGINAL bitstream, but I disagree that this should happen at the SOLR index step. If we need to support policy end dates in the SOLR index step, you would need to reindex all items each night.
An alternative solution would be to only extract text of publicly available bitstreams in the filter-media command. This command already runs at least once a day so it will automatically support embargoed bitstreams.
But I think this is a separate (but related) issue that should be tracked in a different Jira ticket and pull-request.

I've cherry-picked these commits to dspace-6_x.


Feature Request: Sort order changes when drop down vs drop up

It'd be nice if there was an option to have the sorting change based on whether the completion menu is going up vs down relative to the cursor. My desired effect would be the matches with the highest score be closest to the cursor. Currently (from what I can tell). It looks like the sorting is fixed to one direction.
For example, if the drop down is going up relative to the cursor, I get this completion:

If the drop down is going down relative to the cursor position, I get this completion:

Note it'd also be nice to have this change work for cmdline completions as well (that was the original motivation for making this request before I realized that my undesired behavior happened in the normal editor window as well).

Please see this.
https://github.com/hrsh7th/nvim-cmp/issues/910

The solution in that post worked. Thanks!


How to build the application

What is your question?
How can I build an executable from the source code on Windows.
What is the context?
What I tried:
npm run bootstrap
npm run app-start

App starts, everything works. Then I changed some code, ran it again, still worked. Now I need to build an executable to distribute it among my team (until you implemented a feature request of mine, actually ;-) )
So after studying the package.js script, I tried:
set BUILD_TARGETS=portable && npm run app-package

But it always stops with [build] Skipping build for ref ""
Please add instructions on howto build the actual executable to the docs.

until you implemented a feature request of mine

Hi @mklemm2, it's awesome that you've got a fix for (I presume) #3117! Would you be interested in opening a PR with that change into this repository? I'm sure we can look to distribute it through the main channels.
Also, you'll be happy to note that we have just published an official portable version for Windows as well 😄

Sorry, so far I have only changed the "Authorization" header to my custom "X-Something" header, I have not made it configurable or anything. JS is not my language and frontend is not my domain, so it might take a while until I have leveled up enough to present a proper pull request.
So, in the meantime, it would be really great to be able to just build Insomnia into an .exe.

Ah, no worries! The build and packaging scripts fetch information from a git ref (tag) but when packaging for smoke tests we bypass that. On Windows, if you run npm run app-package:smoke from the root directory you should have an executable in the dist folder. If that doesn't work for you I can explore another option, let me know how you get on!

Thanks, works perfectly! Now I'll dig into the rest of the code to make it actually configurable. When I'm done, I'll create a pull request.

Awesome 😄 I will close this issue as resolved. Feel free to open a new one should you have any more questions!


Webpack override remain when recreating project

Version
4.0.5
Environment info
Environment Info:

  System:
    OS: macOS Mojave 10.14.5
    CPU: (8) x64 Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz
  Binaries:
    Node: 12.12.0 - /usr/local/bin/node
    Yarn: 1.19.1 - /usr/local/bin/yarn
    npm: 6.11.3 - /usr/local/bin/npm
  Browsers:
    Chrome: 78.0.3904.87
    Firefox: 69.0.1
    Safari: 12.1.1
  npmPackages:
    @vue/babel-helper-vue-jsx-merge-props:  1.0.0
    @vue/babel-plugin-transform-vue-jsx:  1.1.2
    @vue/babel-preset-app:  4.0.5
    @vue/babel-preset-jsx:  1.1.2
    @vue/babel-sugar-functional-vue:  1.1.2
    @vue/babel-sugar-inject-h:  1.1.2
    @vue/babel-sugar-v-model:  1.1.2
    @vue/babel-sugar-v-on:  1.1.2
    @vue/cli-overlay:  4.0.5
    @vue/cli-plugin-babel: ^4.0.0 => 4.0.5
    @vue/cli-plugin-eslint: ^4.0.0 => 4.0.5
    @vue/cli-plugin-router: ^4.0.0 => 4.0.5
    @vue/cli-plugin-vuex: ^4.0.0 => 4.0.5
    @vue/cli-service: ^4.0.0 => 4.0.5
    @vue/cli-shared-utils:  4.0.5
    @vue/component-compiler-utils:  3.0.2
    @vue/eslint-config-standard: ^4.0.0 => 4.0.0
    @vue/preload-webpack-plugin:  1.1.1
    @vue/web-component-wrapper:  1.2.0
    eslint-plugin-vue: ^5.0.0 => 5.2.3
    vue: ^2.6.10 => 2.6.10
    vue-apollo: ^3.0.0-beta.11 => 3.0.0
    vue-cli-plugin-apollo: ^0.21.3 => 0.21.3
    vue-eslint-parser:  5.0.0
    vue-hot-reload-api:  2.3.4
    vue-loader:  15.7.2
    vue-router: ^3.1.3 => 3.1.3
    vue-style-loader:  4.1.2
    vue-template-compiler: ^2.6.10 => 2.6.10
    vue-template-es2015-compiler:  1.9.1
    vuex: ^3.0.1 => 3.1.2
  npmGlobalPackages:
    @vue/cli: Not Found

Steps to reproduce

Existing Vue cli 3 project test has webpack overrides in vue.config.js
Copy existing project test to test_old
vue create test with Vue cli 4
Create a new vue.config.js with no overrides
vue inspect shows overrides from previous project with same name.

What is expected?
A fresh start with no remnant overrides
What is actually happening?
Overrides from previous project with same name remain.

This is my vue.config.js
→ cat vue.config.js
module.exports = {
  outputDir: '../../priv/static',
  assetsDir: '/',
  runtimeCompiler: true
}

Results in the error:
 ERROR  WebpackOptionsValidationError: Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.
 - configuration.output.chunkFilename: A relative path is expected. However, the provided value "/js/[name].js" is an absolute path!
   -> The filename of non-entry chunks as relative path inside the `output.path` directory.

This configuration is from the older version!
Vue inspect shows:
<snip>

output: {
    path: '/Volumes/univers/dev/elixir/sites/exhaustion/priv/static',
    filename: '/js/[name].js',
    publicPath: '/',
    chunkFilename: '/js/[name].js'
  },

<snip>

Is there any way to blow out this old cached thing?
Thanks!

o you copied th project and then ran create again in the old, existing project, explicitly asking not to override anything, keeping /node_modules and everything in place?
That's not a valid use case of create. an existing project has caches in place, i.e. in /node_modules/.cache.
I don't think it's worth tracking this specific behaviour that you describe, as you essentially broke your existing project by doing this.

Sorry, I MOVED the existing project from test to test_old.
My apologies for the wrong terminology.

did you restart your terminal in between? Are you sure you were in the correct folder?


Compatiblity with React 15.1.0

Is this compatible with 15.1.0 ?
If yes, can you update the dependencies?
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| 15.0.x but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.

I would suggest using peerDependency "^15.0.1" instead of "15.0.x" as we already have 15.1.0. Thanks ;)

Sorry for the delay. Updating and releasing now.


Update environment variables docs

Describe your changes

Please include a summary of the changes and the related issue/task. Please also include relevant motivation and context. List any dependencies that are required for this change, including links to other pull requests/branches in other repositories if applicable.

Updated environment variables used in https://github.com/mcode/rems-smart-on-fhir/pull/105 and https://github.com/mcode/pims/pull/86
Issue ticket number and Jira link

Please include the Jira Ticket Number and Link for this issue/task.

https://jira.mitre.org/browse/REMS-710
https://jira.mitre.org/browse/REMS-712
Checklist before requesting a review

[x] I have performed a self-review of my code
[x] Ensure the target / base branch for any feature PR is set to dev not main (the only exception to this is releases from dev and hotfix branches)

Checklist for conducting a review

[ ] Review the code changes and make sure they all make sense and are necessary.
[ ] Pull the PR branch locally and test by running through workflow and making sure everything works as it is supposed to.

Workflow
Owner of the Pull Request will be responsible for merge after all requirements are met, including approval from at least one reviewer. Additional changes made after a review will dismiss any approvals and require re-review of the additional updates. Auto merging can be enabled below if additional changes are likely not to be needed. The bot will auto assign reviewers to your Pull Request for you.

This is rebased off rems-intermediary-mongo currently since I'm using Docker Compose to run MongoDB


[rampup] Batch merkle path database accesses

!ci-build-me

!ci-build-me

!ci-build-me

Unrelated conflicts causing check-merges fail.
CONFLICT (content): Merge conflict in buildkite/src/Command/DockerImage.dhall
CONFLICT (content): Merge conflict in buildkite/scripts/run-snark-transaction-profiler.sh

Ignore these for this PR chain for now and we'll merge rampup <- berkeley <- develop after to resolve the conflict

!approved-for-mainnet


Disqus Comment

The latest commit is making issues with Disqus comment system. Please check.

Could you elaborate a bit? What error are you seeing? What is the expected behavior? What steps reproduce the problem?

Fixed in be0abd9df3b2c9b22b776091f14c16c33f427fe2


Please expose remove_gamma() and apply_gamma()

Your functions srgb_to_lrgb() and lrgb_to_srgb() contain some useful formulae that implement sRGB-style gamma correction. I wish that these formulae were exposed as functions, e.g.:
/// Remove gamma correction from a sample.
pub fn remove_gamma(x: f32) -> f32 {
    if x <= 0.04045 {
        x / 12.92
    } else {
        ((x + 0.055) / 1.055_f32).powf(2.4)
    }
}

/// Apply gamma correction to a sample.
pub fn apply_gamma(x: f32) -> f32 {
    if x <= 0.0031308 {
        x * 12.92
    } else {
        1.055 * (x.powf(1.0 / 2.4)) - 0.055
    }
}

Where would that formula even be used outside of SRGB?

Grayscale images, for example.
For RGB images, it is useful when the samples are interleaved in one big array. The simple thing is to apply the formula to every element of the array.
Also for RGB images, the three colour components are sometimes in three separate arrays. Again, it's inconvenient to copy them into an array of RGB triples.

For interleaved rgb you can use array_chunks() to iterate over it as an [f32; 3] and it's really fast. Never had channels as separate arrays but you can probably zip+flatten to achieve something similar.
Good idea for single-channel images though. Also maybe SRGB <-> LRGB would possibly auto-vectorize better if it was your only op.
I'll bookmark this as a TODO when I get home

Added in colcon 0.2.1


blinking

Issue:
i am change some different in blinking button. i hope you like it.

Thank you for your work @adrsh79086 but I'm not a big fan of how that looks. Either change the design or close this PR.


Fireworks colors

1.8 clients can see colored fireworks such as blue, green, yellow...
while 1.13 clients see them only as red.
I think is a minor problem but still.

Unable to reproduce this issue, colors work fine for me in 1.13. Does this command work for you? /summon fireworks_rocket ~ ~ ~ {LifeTime:40,FireworksItem:{id:fireworks,Count:1,tag:{Fireworks:{Explosions:[{Type:0,Trail:1,Colors:[I;2437522],FadeColors:[I;2437522]}],Flight:2}}}}


Call SAP API Business Hub API from a SAPUI5 Application

Tutorials: https://developers.sap.com/tutorials/hcp-abh-api-ui5-app.html
In Step 1 of the tutorial, it is not clear why we need to create two destinations and when are we using each destination.
One of the destination connects to sandbox.api.sap.com. Not sure what is the role of sandbox here.

Hi @krisho007,
The tutorial is updated to provide clarity in the Step 1.
Updated Tutorial : https://developers.sap.com/tutorials/hcp-abh-api-ui5-app.html

Thank you for your feedback. The issue seems to be resolved, so I am closing the issue.
If you still have questions, feel free to reopen the issue.


Add filter reset feature in Table View

When working with column filters in the Table View, it can be very difficult to track which filters are applied especially since we currently have "all" rows shown as null/blank in the filter fields. It would be generally useful to have menu option to turn filters off at a global level (somewhat similar to turning off filters in Excel or similar spreadsheet programs).  The current workaround to exit AbC Manager and reload the SQLite file is cumbersome.

closing as duplicate of #241


drivers: apds9960: rework and add trigger

Rework apds9960 driver, use interrupt callback for sample_fetch and add trigger mode for proximity.
The goal is to reduce the energy consumption of the sensor.
Low power state in trigger mode does not work properly, current stays at 140uA instead of 1uA. DNM until it is solved.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@1c16cfc). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master   #9989   +/-   ##
========================================
  Coverage          ?   52.5%
========================================
  Files             ?     213
  Lines             ?   26140
  Branches          ?    5634
========================================
  Hits              ?   13726
  Misses            ?   10159
  Partials          ?    2255

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c16cfc...d8c5c13. Read the comment docs.


Rework of date plugin with custom date NodeSpec

Works good, I think we are missing 2 things, first of all the date gets setted to the actual time it's now, we should set the hour, minutes, seconds and ms to 0 if we are not specifing them in the modal. Second we have to modify the variable plugin to insert this new node :)

Ok I will be looking into reworking the insert-variable plugin configuration in order to support this.


fix(nested-tab-available): make the another tabs into one tabs available

SUMMARY
Unable to add Tabs inside a Tab unless the Tab already has other elements (Dashboard Layout)
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before:

After:

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[x] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up

Pinging @graceguo-supercat for any additional historical context on why we have this limitation, in case @williaster is unavailable.


Excel file Data Read and Integration not working with version 1.5.0

Excel Data Read is not working giving .Please go through below information
ExcelReader.txt
used above java method to read data and using it in our feature file as given below
Background :
def ExcelReader = Java.type('com.bold.automation.utility.ExcelReader')
def json_payload_data = ExcelReader.readExcel('src/test/java/TEST_DATA.xlsx', 'sheet1')
@sanity
Scenario Outline : Verify data through Excel
Given url BASE_URL
And def getskill_endpoint = endpoints_getskill.getSkill_V4.replace("{userid}",userid)
And path getskill_endpoint
And param userid = userid
And param criteria.jobTitle = jobtitle
And param criteria.productCD = productcd
And param criteria.cultureName = culturename
And param criteria.contentTypeCD = contenttypecd
And param criteria.sectionTypeCD = sectiontypecd
And param criteria.contentGroupCD = contentgroupcd
And param criteria.portalCD = portalcd
And headers request_headers
When method GET
Then status 200

print 'Response----------',response

Examples:
|json_payload_data|
Error :
*
feature/Experiments/SUMM/Exp_Snapshot_SUMM.feature[1:210] dynamic expression evaluation failed: json_payload_data
0
js failed:
01: json_payload_data
<<<<
org.graalvm.polyglot.PolyglotException: ReferenceError: "json_payload_data" is not defined
.:program(Unnamed:1)

@ptrthomas @maxandersen @julianladisch Please help.

@rehanahmad71292 please follow this process to re-open: https://github.com/karatelabs/karate/wiki/How-to-Submit-an-Issue
there are reasons why we need a full working project, see the last part of the above link. giving a *.txt file is not enough.


[RESOLVED] High ICMP ping response time

Problem started at 08:31:29 on 2023.04.21
Problem name: High ICMP ping response time
Host: HP_iLO
Severity: Warning
Operational data: Value: 695ms
Original problem ID: 19805
Event details in Zabbix: http://<IP_ADDRESS>/tr_events.php?triggerid=23268&eventid=19805

Problem has been resolved in 5m 0s at 08:36:29 on 2023.04.21
Problem name: High ICMP ping response time
Host: HP_iLO
Severity: Warning
Original problem ID: 19805


Proposal: Search keywords directly to the content page

Such as: Search "灰度"，enter the “灰度”, should enter the third screenshot, should not jump through the second screenshot.

@kubesphere/sig-docs

/kind proposal


GitHub Integration - I can't find the repo of a repo I am an "Outside Collaborator" on

Please describe which feature you have a question about?
Connecting the GitHub repo to Amplify Console
Provide additional details
I am building a client project. The GitHub repo is under their GitHub organization and I am added to that repo as an "Outside Collaborator" with Admin access (to the repo). I can't find the repo when I auth GitHub in Amplify Console and check the list of repos. What do I need to do in order for this repository to appear?
I would consider this a standard workflow for building a client site, where I may not be able to be added as a member of their GitHub organization, but instead am added just to the repository.

Hello  @drake-smith,
I was able to setup a test organization, and invite a test user I created as an outside collaborator. Then, I was able to use the test user's github to access the required repo under the org. You have to make sure to click request access next to the organization, when you first authorize Amplify to access the required organization, as follows:

You would have to wait for the owner of the organization to approve the request in order to see the repo in your list.
Since you would have already added Amplify as an authorized OAuth application in github, you would first need to revoke access for it, and try again. This time make sure to request access to the required organization.
Let me know if you have any questions.


Change redirectUrl when payment was completed and 'back to the game' button is clicked

Hello,
I am adding Xsolla as payment service to our game. Currently on our browser-game we use the paystation widget v1.0.7 and the sandbox-mode is set to true in the options. The created app uses our dev-server which is the backend for the browser-game. The browser-game has socket-connections to the backend and the backend informs the frontend via socket that a payment was completed.
When the payment was completed and the 'back to the game' button is clicked, the whole site is reloaded. I don't want to reload the whole page, I just want to close the paystation widget. How can I do this? I don't find a setting.
I could add a 'status-done' event and close the paystation widget via javascript but I think that's not the best way... (this would hide the back to the game button and hence it's not possible to click it, so there won't be a redirection)
Any advice?
Thank you!

There is a working solution, thanks to the fast and great xsolla support.
A project can have a Return URL. The Return URL can have additional options (even when it's empty). The Redirect Conditions and Status for manuel redirection can be set to 'Never' and the redirect-button is gone :) Thanks again!

hey @hamburml is there any doc for these conditions? can I see your redirec_url if possible?


System-tests: brokered: webconsole tests

create/delete brokered address space
create/delete queue/topic in brokered address space
sorting/filtering

testing scenarios: https://github.com/EnMasseProject/enmasse/pull/608/commits/c3d53f82f79ad8f64794a505a5d0da7074a7d94a

this task is too general, so I'm closing it for now with defined scenarios. Implementation of each test will be tracked separately.


Testing new process info discovery

Notes

Ensure the race detector is enabled
Check on Parca
Check also on Polar Signals
Check for stack traces correctness (i.e. before vs after in differential flame graphs)

Runtimes

[x] Go
[x] Native code (not Go) with debug info not extracted
[x] Native code (not Go) with extracted debug info
[ ] Native code with debuginfo only in debuginfod servers
[x] JIT
[x] jitdump

Important software

[x] MySQL
[ ] Postgres
[ ] Redpanda
[x] Short-lived Go procs
[ ] Short-lived native (not Go) procs

Check:

[x] Normalization errors
[x] Other added metrics
[x] Error logs
[x] Logs that are too verbose by default
[x] Symbolization
[x] Open file descriptors (ensure there are no leaks)
[ ] CPU and memory shouldn't regress

Issues found in Demo / Prod:
cc/ @kakkoyun

These are one-time manual tests, correct? Is this change also tested with unit/integration tests?

These are one-time manual tests, correct? Is this change also tested with unit/integration tests?

There are one-time tests. It would be nice to have integration/regression tests for these eventually.

@javierhonduco If I remember correctly, you had an app to reproduce short-lived processes?

@kakkoyun For frame pointers: https://github.com/parca-dev/testdata/blob/main/src/main.go

FYI Updating the issue as test (ticking the boxes)

MySQL: https://pprof.me/53340c1

Postgres: https://pprof.me/585c5f1

Redpanda: https://pprof.me/faf4ab0 (On a 4-node local cluster)


Gemfile error when running app.groovy with Spring Boot CLI

In the "JAR support and Groovy support" section after installing the Spring Boot CLI with Homebrew on Mac OS X Mavericks I get a "no Gemfile present" error when running app.groovy with the "spring run app.groovy" command.
The error I get is:
groovy(master) > spring run app.groovy
/Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:46:in find_project_root': Spring was unable to locate the root of your project. There was no Gemfile present in the current directory (/Users/kent/git/public/spring-projects/koj-gs-spring-boot/src/main/groovy) or any of the parent directories. (Spring::UnknownProject)     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:infind_project_root'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:in find_project_root'     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:infind_project_root'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:in find_project_root'     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:infind_project_root'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:in find_project_root'     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:infind_project_root'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:in find_project_root'     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:48:infind_project_root'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/configuration.rb:37:in project_root_path'     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/env.rb:27:inproject_root'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/env.rb:41:in application_id'     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/env.rb:53:inpidfile_path'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/env.rb:68:in server_running?'     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/client/run.rb:24:incall'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/client/command.rb:7:in call'     from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/lib/spring/client.rb:26:inrun'
from /Users/kent/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/spring-1.1.3/bin/spring:48:in <top (required)>'     from /Users/kent/.rbenv/versions/2.1.2/bin/spring:23:inload'
from /Users/kent/.rbenv/versions/2.1.2/bin/spring:23:in `'

spring is a gem used by Ruby on Rails framework as of version 4.1 (http://guides.rubyonrails.org/4_1_release_notes.html) and it just happens to also use an executable called 'spring'. It's pretty funny how the first commit to this spring gem is about 2 months after the first commit to spring-boot project
https://github.com/rails/spring/commit/a035f47a4f1c930cbc452eb87aeabaab409a0e64
Unfortunate coincidence? :8ball:

Coincidence indeed. I guess Rails users have to alias "spring" to "springboot" or something.

As mentioned by @dsyer , here's steps to execute on your shell
gem uninstall spring
cd /usr/local/bin
sudo ln -sf ../Cellar/springboot/*.RELEASE/bin/spring .

Don't need to uninstall anything...
cd /usr/local/bin
sudo ln -sf ../Cellar/springboot/*.RELEASE/bin/spring ./springboot

Then just call springboot init or whatever

Thanks @dsmithco !

@dsmithco Worked for me. Thank you.