Make specifying static props opt-in, and offer simpler usage

This is in preview in the v6-trey branch.

We currently need to have a static props definition so we can define properties that cause the component to re-render. This got me thinking: what if there was a way to not have to specify props and have detect props somehow?
The obvious solution here is proxies but I think we should still support IE11, so we'd have to come up with a "good enough" algorithm for detecting props the consumer means to have re-render the component when updated.
The best I could come up with is in the v6-trey branch and goes something like:

During the connectedCallback of first element to be connected

If the consumer has defined static props

Go through the standard prop defining algorithm.

Otherwise

Get own props on the element instance.
For each own prop

Use the value as the default value.
Define a prop on the prototype (and element because proto will not affect the first element) that will re-render the component based off of the constructor of the default value (if it has one - undefined / null won't).

It's important to note this is a non-breaking enhancement as static props is currently required and, if specified, will opt-out of any sort of default behaviour, thus not incurring any performance penalties, or edge-cases that it may present.

Consider an example where a Component is created without props and is working fine (reacting to some instance prop).
Later in the project, someone adds a props definition: suddenly the Component breaks for not clear reason.
The same could happen by refactoring code to remove a property setting in constructor

Consider an example where a Component is created without props and is working fine (reacting to some instance prop).

Later in the project, someone adds a props definition: suddenly the Component breaks for not clear reason.

I can see where you're coming from with this, but to me, this would be expected.

The same could happen by refactoring code to remove a property setting in constructor

If you define a property in the constructor, you're initializing it. If you remove that, then there is the issue that it's no longer observed, but it's likely that you don't want it observed if you're removing it. It feels like that gap can be closed by verbose documentation.
This is definitely still in a proposal stage but I really like the idea of having a super-streamlined, minimal API. The props API isn't really changing, it's just that we offer a baseline, default behaviour now when it's not specified. I would hope documentation would then be clear as to what happens when you do or don't specify props.

We ended up not doing this as it requires a lot of magic that just isn't worth it in the end.


[MENFORCER-431] Introduce property to skip specific rules

I am creating this draft PR to discuss couple of points. This is not yet ready to merge. See the inline comments.

[x] Make sure there is a JIRA issue filed
for the change (usually before you start working on it).  Trivial changes like typos do not
require a JIRA issue.  Your pull request should address just this issue, without
pulling in other changes.
[x] Each commit in the pull request should have a meaningful subject line and body.
[x] Format the pull request title like [MENFORCER-XXX] - Fixes bug in ApproximateQuantiles,
where you replace MENFORCER-XXX with the appropriate JIRA issue. Best practice
is to use the JIRA issue title in the pull request title and in the first line of the
commit message.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Run mvn clean verify to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
[x] You have run the integration tests successfully (mvn -Prun-its clean verify).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.
To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

[x] I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

[x] In any other case, please file an Apache Individual Contributor License Agreement.

@psiroky please look at #207 - it can impact on your code

@slawekjaranowski thanks for the heads-up. That seems to change things quite a bit, so I will probably wait until it gets merged and then re-do my changes on top of that.

@psiroky  #207 - merged
Please look.
I will have next changes, so wait for you.

@slawekjaranowski I believe this should be now ready for review. Please take another look when you get a chance.


Yodayo link switching.

Apologies to drive you crazy but they did a switch again back to Yodayo and it broke, I tried using the older version too so there's something different as well.

Hi, they are under maintenance at yodayo... Moescape.ai domain should still
work.
On Mon, Oct 21, 2024, 12:06 AM edge2177 @.***> wrote:

Apologies to drive you crazy but they did a switch again back to Yodayo
and it broke, I tried using the older version too so there's something
different as well.
—
Reply to this email directly, view it on GitHub
https://github.com/pervertir/Yodayo-Custom-Bg-Char-Image/issues/2, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/BJICPOOI64HAC4G3U6AVW6LZ4RSD5AVCNFSM6AAAAABQJER6ECVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYYDCMJSGYZTCNI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

But if you want, you can still goto my script and change the @match part
from moescape.ai to yodayo.com
On Mon, Oct 21, 2024, 12:28 AM Honey Illuzion @.***> wrote:

Hi, they are under maintenance at yodayo... Moescape.ai domain should
still work.
On Mon, Oct 21, 2024, 12:06 AM edge2177 @.***> wrote:

Apologies to drive you crazy but they did a switch again back to Yodayo
and it broke, I tried using the older version too so there's something
different as well.
—
Reply to this email directly, view it on GitHub
https://github.com/pervertir/Yodayo-Custom-Bg-Char-Image/issues/2, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/BJICPOOI64HAC4G3U6AVW6LZ4RSD5AVCNFSM6AAAAABQJER6ECVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYYDCMJSGYZTCNI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thanks will try. The moescape one stopped working for some reason too.

Afraid even moescape still isn't working if you have a chance to peek at it again.

If you have some free time this week take a peek at the script again please? I don't seem to be able to get it to work on Chrome, Edge, or Firefox. Thank you!


Update lodash

https://nodesecurity.io/advisories/577

I've run the tests locally on my machine and they are passing. So maybe it's only a timeout of some sort?


Update Update-M365TeamsApp.md

Update documentation for Update-M365TeamsApp

Hi @srimantabagchi
Following is a blocking issue that need to be resolved before we can merge:
The identifiers used on lines 52 to 59 don't seem to follow the approved pattern for GUIDs.

Reference
[Criteria used for pull request reviews](https://review.learn.microsoft.com/en-us/office-authoring-guide/pr-criteria?branch=main#content-and-formatting)
Blocking

GUIDs must use an approved pattern from Approved sensitive identifiers.

Use approved identifiers on lines 52 to 59.

After you resolve the blocking issues, remove the Back to submitter label and add the Sign off label again.
If you need help resolving issues in this PR, post your question in the Ask an Admin Teams Channel.
Thanks,
Padma


Deployment of the opc-ua-publisher fails and returns an exception

Environment:

Advantech 2271G-AE, 4GB RAM, 32GB SSD, Intel Atom E3815 processor
Xubuntu 18.04, patch level on par
IotEdged 1.04, edgeAgent 1.0.4, edgeHub 1.0.4
one other custom module (heartbeat every 5 seconds sent to IoTHub)
deploying from a Windows 10 Professional 1809, Visual Studio Code setup

I used this instructions to create a shared directory on the host: How to Install and Configure the OPC-UA Publisher on IoT Edge. Inside the shared folder I created a pn.json file with the following content:
[   {     "EndpointUrl": "opc.tcp://1xx.1xx.1xx.1xx:Port",     "UseSecurity": false,     "OpcNodes": [       {         "Id": "ns=2;s=Channel1.Device1.Tag1",         "OpcSamplingInterval": 2000,         "OpcPublishingInterval": 5000       }     ]   } ]
I have used  the following section in my deployment.template.json:
"microsoftOpcUaPublisher": {             "version": "1.0",             "type": "docker",             "status": "running",             "restartPolicy": "always",             "settings": {               "image": "mcr.microsoft.com/iotedge/opc-publisher",               "createOptions": "{\"Hostname\":\"opcuapublisher-jw\",\"Cmd\":[\"publisher\",\"--pf=./config/pn.json\",\"--di=60\",\"--to\",\"--aa\",\"--si=0\",\"--ms=0\"],\"HostConfig\":{\"PortBindings\":{\"62222/tcp\":[{\"HostPort\":\"62222\"}]},\"Binds\":[\"test_cfx509certstores:/root/.dotnet/corefx/cryptography/x509stores\",\"/shared:/config\"],\"ExtraHosts\":[\"localhost:<IP_ADDRESS>\"]}}"             }           }
And the createOptions unescaped:
{     "Hostname": "opcuapublisher-jw",     "Cmd": [         "publisher",         "--pf=./config/pn.json",         "--di=60",         "--to",         "--aa",         "--si=0",         "--ms=0"     ],     "HostConfig": {         "PortBindings": {             "62222/tcp": [                 {                     "HostPort": "62222"                 }             ]         },         "Binds": [             "test_cfx509certstores:/root/.dotnet/corefx/cryptography/x509stores",             "/shared:/config"         ],         "ExtraHosts": [             "localhost:<IP_ADDRESS>"         ]     } }
Deployment of the module fails. While checking the module logs with $iotedge logs edgeAgent --tail 75 the following exception is logged:
2018-11-06 07:37:12.047 +00:00 [WRN] - Reconcile failed because of the an exception System.AggregateException: One or more errors occurred. (Error calling Create module microsoftOpcUaPublisher: unknown: The operation is disallowed.) ---> Microsoft.Azure.Devices.Edge.Agent.Edgelet.EdgeletCommunicationException: Error calling Create module microsoftOpcUaPublisher: unknown: The operation is disallowed.    at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.Execute[T](Func1 func, String operation) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 194
at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.CreateModuleAsync(ModuleSpec moduleSpec) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 89
at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 36
at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunners.OrderedRetryPlanRunner.ExecuteAsync(Int64 deploymentId, Plan plan, CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/planrunners/OrdererdRetryPlanRunner.cs:line 86
--- End of inner exception stack trace ---
at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunners.OrderedRetryPlanRunner.<>c.b__7_0(List1 f) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/planrunners/OrdererdRetryPlanRunner.cs:line 114    at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunners.OrderedRetryPlanRunner.ExecuteAsync(Int64 deploymentId, Plan plan, CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/planrunners/OrdererdRetryPlanRunner.cs:line 115    at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 176    at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 187 ---> (Inner Exception #0) Microsoft.Azure.Devices.Edge.Agent.Edgelet.EdgeletCommunicationException- Message:Error calling Create module microsoftOpcUaPublisher: unknown: The operation is disallowed., StatusCode:500, at:   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.Execute[T](Func1 func, String operation) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 194
at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.CreateModuleAsync(ModuleSpec moduleSpec) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 89
at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 36
at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunners.OrderedRetryPlanRunner.ExecuteAsync(Int64 deploymentId, Plan plan, CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/planrunners/OrdererdRetryPlanRunner.cs:line 86<---
`
Any ideas what is going wrong?

Changed the issue a bit since I mixed up testing of the OPC UA Publisher module and the Blob Storage module. Sorry about that. This logging and scenario now stated is the real issue I have.

First thing i would change is in the create options --pf option: "./config/pn.json" to "/config/pn.json"
Then: remove the bind: "test_cfx509certstores:/root/.dotnet/corefx/cryptography/x509stores", this will conflict with the IoT Edge cert store and is already updated in the README.md of the OPC Publisher github.
Finally change the routes to:
{
"routes": {
"route": "FROM /messages/* INTO $upstream"
}
}
I will update this in the README.md soon.
Please let me know if it worked.

I have now updated the README. Please verify that you have specified a tag for the container, otherwise IoT Edge does use tag 1.0 which does not exist.
Please let me know if you still see the issue.

I used the 2.1.4-linux-amd64 image to create the module and changed the settings in createOptions as you said. This is the result:
microsoftOpcUaPublisher   stopped          Stopped 21 seconds ago  mcr.microsoft.com/iotedge/opc-publisher:2.1.4-linux-amd64
*I started it in a stopped state since the pn.json file is listing non exsisting servers still. Will work on that now.
Thanks for pointing out and clarifying the createOptions in better details in the updated README. Now let's connect some OPC UA Servers.

Ok, that makes sense. Can you use either latest or 2.2.0-preview001 as tag? I will release 2.2.0 tomorrow.
From: Jan Willem Groenenberg<EMAIL_ADDRESS>Sent: Dienstag, 6. November 2018 22:50
To: Azure/iot-edge-opc-publisher<EMAIL_ADDRESS>Cc: Hans Gschoßmann<EMAIL_ADDRESS>Assign<EMAIL_ADDRESS>Subject: Re: [Azure/iot-edge-opc-publisher] Deployment of the opc-ua-publisher fails and returns an exception (#162)
I used the 2.1.4-linux-amd64 image to create the module and changed the settings in createOptions as you said. This is the result:
microsoftOpcUaPublisher stopped Stopped 21 seconds ago mcr.microsoft.com/iotedge/opc-publisher:2.1.4-linux-amd64
*I started it in a stopped state since the pn.json file is listing non exsisting servers still. Will work on that now.
Thanks for pointing out and clarifying the createOptions in better details in the updated README.https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fiot-edge-opc-publisher%2Fblob%2Fmaster%2FREADME.md&data=02|01|johanng%40microsoft.com|3d448300c75d498ec12208d6447d4b1e|72f988bf86f141af91ab2d7cd011db47|1|0|636771702257077461&sdata=33P5sY8rzGUmqP%2BhoaF1fSD5LnbwxyaNkKd2eCIX6fk%3D&reserved=0 Now let's connect some OPC UA Servers.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fiot-edge-opc-publisher%2Fissues%2F162%23issuecomment-436522312&data=02|01|johanng%40microsoft.com|3d448300c75d498ec12208d6447d4b1e|72f988bf86f141af91ab2d7cd011db47|1|0|636771702257087470&sdata=C0zts39KL%2BeqYHcbUAi7r0%2Fkv2GjyrrT25lmDdwM6zE%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHxeltrHJDt9jYi3TO_p0Wkkvz-SVwrlks5usoKvgaJpZM4YOqkg&data=02|01|johanng%40microsoft.com|3d448300c75d498ec12208d6447d4b1e|72f988bf86f141af91ab2d7cd011db47|1|0|636771702257087470&sdata=YsQM9Ldzn0rKA3%2FguCzd3P16KWytNNXjJ8Mu2CyBHS8%3D&reserved=0.

Tested the 2.2.0-preview001 as well. Besides a huge size difference of the image it gives the same result.
Will test with a simulator as described in this blogpost from my colleague @sandervandevelde: Getting  started with OPC-UA on Azure IoT Edge.

And i have released 2.2.0 just a few hours ago. Can you give this a try?
From: Jan Willem Groenenberg<EMAIL_ADDRESS>Sent: Mittwoch, 7. November 2018 11:58
To: Azure/iot-edge-opc-publisher<EMAIL_ADDRESS>Cc: Hans Gschoßmann<EMAIL_ADDRESS>Assign<EMAIL_ADDRESS>Subject: Re: [Azure/iot-edge-opc-publisher] Deployment of the opc-ua-publisher fails and returns an exception (#162)
Tested the 2.2.0-preview001 as well. Besides a huge size difference of the image it gives the same result.
Will test with a simulator as described in this blogpost from my colleague @sandervandeveldehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsandervandevelde&data=02|01|johanng%40microsoft.com|ede7e76fc3da44fb01ef08d644eb60f5|72f988bf86f141af91ab2d7cd011db47|1|0|636772175062869962&sdata=7MkIZuGICzZy0zINuuuw9DFmpZLhJfsVLYO28OZj0sM%3D&reserved=0: Getting started with OPC-UA on Azure IoT Edgehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsandervandevelde.wordpress.com%2F2018%2F11%2F06%2Fgetting-started-with-opc-ua-on-azure-iot-edge%2F&data=02|01|johanng%40microsoft.com|ede7e76fc3da44fb01ef08d644eb60f5|72f988bf86f141af91ab2d7cd011db47|1|0|636772175062869962&sdata=xzu9vbWEHfB5tKYYHag8A2dhvxDuMPL%2BsNSaB295GOE%3D&reserved=0.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fiot-edge-opc-publisher%2Fissues%2F162%23issuecomment-436757208&data=02|01|johanng%40microsoft.com|ede7e76fc3da44fb01ef08d644eb60f5|72f988bf86f141af91ab2d7cd011db47|1|0|636772175062879966&sdata=%2BA2Tbk3lHaCTclbvJf3%2Bw%2BAEm6dpM79wYTmGlw45QRk%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHxelqGDYuH0kJ_zNlzbKrt7GotU72q6ks5usztggaJpZM4YOqkg&data=02|01|johanng%40microsoft.com|ede7e76fc3da44fb01ef08d644eb60f5|72f988bf86f141af91ab2d7cd011db47|1|0|636772175062889979&sdata=fFc0tkxsRYqAYH%2BkhXfyoDHw4En1jRJ1gzYUalJsOyI%3D&reserved=0.

Deployment of 2.2.0 went flawless. Unfortunately no time to test deeper into the configuration of the servers, alarms and events due to demo preparation for the SPS Drives in Nuremberg at the end of the month. (BTW, are you by any chance at the SPS Drives this year for the OPC foundation or Microsoft. You deserve a bag of stroopwafels for your help. ;-) )


Obsidian Crashing when Pasting Image

Shortly after attaching/pasting an image, Obsidian crashes, reopening itself on the Vault selection page. And then, once the vault is selected, reopening on the note with the attached image now correctly renamed.
Sometimes, this is the error that appears:
Uncaught Exception: TypeError: Cannot read properties of undefined (reading 'origin') at Function.<anonymous> (/Applications/Obsidian.app/Contents/Resources/obsidian.asar/main.js:27:9882)
Disabling this plugin stabilize the issue

Tks for report this issues, can you provide your obsidian version and the list of your plugins ? (you can find them in <your vault folder>/.obsidian/community-plugins.json).


The future is now

Merge the experimental 5.3 feature branch into master, behind a Swift version check.
This makes Swift Numerics extensions for Float16 available if the following criteria are satisfied:

You are using a Swift 5.3 or greater toolchain
You are not compiling for macOS (Float16 is not available in the macOS standard library because Intel hasn't defined the calling conventions for Float16 yet, so it cannot be made binary-stable).

This also adds a Swift version check for differentiable features; they are now only available when using Swift 5.3 or later, while previously they only checked if _Differentiation could be imported.

@compnerd Can you sanity-check the windows build with this?

@stephentyrone sure, I'll check it tomorrow.

This seems to have caused some issues on Windows, specifically in the tests:
https://dev.azure.com/compnerd/3133d6ab-80a8-4996-ac4f-03df25cd3224/_apis/build/builds/35505/logs/13
2020-08-05T01:38:17.1036766Z D:\a\1\s\Tests\RealTests\IntegerExponentTests.swift:89:64: error: type 'Float16' has no member 'pow'
2020-08-05T01:38:17.1038873Z     assertClose(-7.3890572722436554354625993393835304, Float16.pow(-u, 0x801))

Fixed the build issue with #137.  That uncovers a few failures:
testDouble fails:
Test Case 'IntegerExponentTests.testDouble' started
ULP error was unacceptably large: expected 1.7976931348623157e+308 but saw 1.797693134862245e+308 (354.0-ulp error).
S:\SourceCache\swift-numerics\Tests\RealTests\IntegerExponentTests.swift:152: error: IntegerExponentTests.testDouble : failed -
ULP error was unacceptably large: expected -8.98846567431158e+307 but saw -8.988465674311225e+307 (177.5-ulp error).
S:\SourceCache\swift-numerics\Tests\RealTests\IntegerExponentTests.swift:153: error: IntegerExponentTests.testDouble : failed -
ULP error was unacceptably large: expected 8.98846567431158e+307 but saw 8.988465674311227e+307 (176.5-ulp error).
S:\SourceCache\swift-numerics\Tests\RealTests\IntegerExponentTests.swift:154: error: IntegerExponentTests.testDouble : failed -
Test Case 'IntegerExponentTests.testDouble' failed (0.0 seconds)
Test Suite 'IntegerExponentTests' failed

Ok, I'll relax the tolerances for these tests on Windows for now.


FIX: ignore warnings tests

Before submitting (must do checklist)

[ ] Did you read the contribution guide?
[ ] Did you update the docs? We use Numpy format for all the methods and classes.
[ ] Did you write any new necessary tests?
[ ] Did you update the CHANGELOG?

Proposed Changes

Closing issues

Codecov Report

Merging #880 (151d1e9) into master (c066733) will decrease coverage by 35.31%.
The diff coverage is n/a.

:exclamation: Current head 151d1e9 differs from pull request most recent head ab19fd2. Consider uploading reports for the commit ab19fd2 to get more accurate results

@@             Coverage Diff             @@
##           master     #880       +/-   ##
===========================================
- Coverage   84.75%   49.44%   -35.32%
===========================================
  Files         132      132
  Lines        7473     7473
===========================================
- Hits         6334     3695     -2639
- Misses       1139     3778     +2639

Impacted Files
Coverage Δ

etna/commands/__init__.py
0.00% <0.00%> (-100.00%)
:arrow_down:

etna/commands/backtest_command.py
0.00% <0.00%> (-97.06%)
:arrow_down:

etna/commands/forecast_command.py
0.00% <0.00%> (-94.88%)
:arrow_down:

etna/models/utils.py
11.11% <0.00%> (-88.89%)
:arrow_down:

etna/commands/__main__.py
0.00% <0.00%> (-87.50%)
:arrow_down:

etna/libs/pmdarima_utils/arima.py
16.00% <0.00%> (-84.00%)
:arrow_down:

etna/commands/resolvers.py
0.00% <0.00%> (-80.00%)
:arrow_down:

etna/analysis/outliers/density_outliers.py
22.44% <0.00%> (-75.52%)
:arrow_down:

etna/datasets/datasets_generation.py
27.02% <0.00%> (-72.98%)
:arrow_down:

etna/transforms/timestamp/time_flags.py
27.02% <0.00%> (-72.98%)
:arrow_down:

... and 81 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


wr.quicksight.create_athena_dataset(): Changing sql_name to optional variable with uuid

Enhancement

wr.quicksight.create_athena_dataset(): Changing sql_name to optional variable with uuid

Detail

Change sql_name to optional
If sql_name is not specified use default from before CustomSql as a prefix and a uniquely generated id as a suffix to avoid the issues as discussed in #1511

Relates

#1511

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: GitHubStandardCodeBuild8C06-llutOAimTATs
Commit ID: f0de73c4e597cec2a4dacb4b8922fcc2219acd5f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubLoadTests5656BB24-s6u9F3qN9oFy
Commit ID: f0de73c4e597cec2a4dacb4b8922fcc2219acd5f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Data Quality Evaluation Suite

We need to compose an evaluation suite to support various decisions during the pre-training data preparation stage, such as:

Language filtering, quality filtering, content filtering
Deduplication
Mixing proportions of different data sources
Deciding whether or not to use synthetic data, code data etc.

It would be interesting to explore whether practices followed in LLM pre-training (e.g. Dolma/OLMo) transfer to encoder models.  For this evaluation suite, we will require a set of high signal benchmarks.  A high signal eval dataset should satisfy the following properties (Reference: https://youtu.be/2-SPH9hIKT8?t=1893):

Monotonicity: model performance improves monotonically as training progresses (avoid early saturation)
Low variance:

when comparing two known reference datasets (e.g. Pile vs C4)
when comparing with various sub-parts of data and seeds
above random baseline

We can test various common NLP benchmarks (e.g. tasks in SuperGLUE) against these properties. A viable strategy could be to train randomly initialized deberta-v3-base models on subsets of Pile/C4 (with RTD/MLM) and track eval metrics at intermediate checkpoints.  Alternative suggestions/feedback are welcome!

@orionw probably of interest to you too!

Quick question - does the data quality validation benchmark need to be different from the final downstream benchmarks?

I don't think data quality validation and downstream benchmarks should be very different (expected to have high overlap). However, I think it's better to have a different evaluation setup for pre-training / data ablation studies:
For pre-training data quality evaluation, I think:
- We should avoid fine-tuning the base model and reformulate the benchmarks to measure performance in a zero shot manner. As a start, I am thinking about cloze style reformulation (using prompt + verbalizer pairs) (as in the AdaPET paper Appendix A)
- I think data quality validation should have a different focus (training efficiency, diversity, coverage, training stability, avoiding loss spikes etc) and evaluation method.
- We can also take into account MLM/RTD loss
I quite like the data quality evaluation strategy in Fineweb (https://huggingface.co/datasets/HuggingFaceFW/fineweb#ablation-evaluation-benchmarks) -- but need to adapt it for our use case.

Quick question - does the data quality validation benchmark need to be different from the final downstream benchmarks?

I think data quality validation and downstream benchmarks are expected to have a high overlap. However, I think it's better to have a different evaluation setup for the data ablation studies:
- Instead of finetuning the base model, we should measure performance in a zero shot manner (isolating the impact of pre-training). As a start, I'm thinking cloze style reformulation using prompt + verbalizer pairs -- as in the AdaPET paper Appendix A: https://arxiv.org/pdf/2103.11955
- Additionally, we can monitor training efficiency, stability/loss spikes, MLM / RTD loss
I quite like the data quality evaluation strategy in Fineweb -- but we need to adapt it for encoder training.


ProducesResponseType with ModelStateDictionary

I'm trying to use multiple ProducesResponseTypeAttribute on my action, but if I use built in types like ModelStateDictionary the swagger page doesn't contain this class' model. If I use custom type it works like charm.
Not working:
[HttpGet("{id}")]
[ProducesResponseType(typeof(Product), (int)HttpStatusCode.OK)]
[ProducesResponseType(typeof(ModelStateDictionary), (int)HttpStatusCode.BadRequest)]
public IActionResult Get(int id)
{
    return Ok(_mapper.Map<Product>(_productService.GetProduct(id)));
}

Working but I don't want this:
[HttpGet("{id}")]
[ProducesResponseType(typeof(Product), (int)HttpStatusCode.OK)]
[ProducesResponseType(typeof(Order), (int)HttpStatusCode.BadRequest)]
public IActionResult Get(int id)
{
    return Ok(_mapper.Map<Product>(_productService.GetProduct(id)));
}

ping

Okay I dig into this a little bit and I found that the asp.net core pipeline does not use ModelStateDictionary as a return value for BadRequest rather than a SerializableError class. This is formally a Dictionary<string, object> but technically a Dictionary<string, string[]> see.
So we need to use [ProduceResponseType(typeof(Dictionary<string, string[]>)] attribute to achive the same functionality as the MVC pipeline works. However after this the swagger ui still cannot visualize the response's model, because the Dictionary<string, whatever> is a simple json object with unspecified property names. Besides that in the swagger schema is correct so the client side code generation should work.

Could we find out some way to visualize this model in the swagger UI?

Essentially a dup of #337


Korean keyboard won't type korean syllables

Hi Francisco,
For example: it should be "워", but it always types as '우 ㅓ'. How can I fix the problem? Thank you in advance.

Hello @hanuz06, in most language layouts we rely on unicode character composition, which I don't think is working here. However, I've recently moved towards candidate box suggestions / IME starting with the Chinese (Pinyin) keyboard.

I think the best way to address your concerns is to move the Korean layout to this kind of input method. To accomplish this, I would need a detailed list of key candidates, such as this one: https://github.com/simple-keyboard/simple-keyboard-layouts/blob/master/src/lib/layouts/chinese.ts#L24
I will keep this task open as I look further into this.
Regards,
Francisco Hodge

Hi Francisco,
Thank you for your reply. It's cool that the project is alive. The Korean language uses the alphabet and words are syllables. Luckily, there are much fewer syllable constructions than in Chinese. It looks like this site has all possible Korean syllables combinations: . Hopefully, this helps. Thank you for your hard efforts.

Hi Francisco,
Thank you for your reply. It's cool that the project is alive. The Korean language uses the alphabet and words are syllables. Luckily, there are much fewer syllable constructions than in Chinese. It looks like this site has all possible Korean syllables combinations: . Hopefully, this helps. Thank you for your hard efforts.

Thanks for the information, I will look into this soon


refactor sprite render.

now we have a better render system, it's easier to expand. :)
But we still need to consider the name of the modules.


Add script to update repository stars

I noticed the star counts on several of the repositories were outdated, sometimes by quite a lot. Since it's helpful to have these and a pain to update them all manually, I wrote a small script that parses the README and uses the GitHub API to fetch the new count and update. To use the script you just need to define the GITHUB_USERNAME and GITHUB_TOKEN environment variables.
I made the text bold if the repository has more than 100 stars since that seemed to roughly the threshold which was already used.

Ah, I didn't realize there was a bot for this. Normally I would ping repo maintainers before building something like this, but I was having fun so I decided to go for it anyway :)

Until the awesome-lists-bot is going you can view README_STARS.md.

Thanks!

@kazupon, is there an ETA on the awesome-lists-bot solution?


Honor afterware promises when querying

When query'ing using the default network interface, applyMiddleware and applyAfterware both return promises, and the query logic waits for middleware promises but ignores afterware promises.
I've created tests for both middleware and afterware promises to validate the behavior and the afterware test fails.
Given that this works for middleware and applyAfterwares method returns a Promise, this seems to be a bug?
If it is a bug, I've implemented a fix in my branch and can submit a PR once I get confirmation from the team.
Thanks!

Is there something you want to do in an afterWare that requires a promise? Just curious about the use case.

I’ve actually figured out a workaround where I don’t need a Promise, but my actual use case is such that I want to specify my own reviver function to parse the JSON vs. receiving the output of response.json() (via query()) and then processing it from there.
I could write a custom network interface for this, but given the existing behavior for middleware, I wanted to see if honoring promises in the afterware should be supported.
My workaround is to just override the json function in the response with my own flavor ( via async/await vs. Promises just b/c this project is using es7 via Babel ):
{
  applyAfterware: ({response}, next) => {
    if (response.status === 200) {
      Object.defineProperty(response, 'json', {
        value: async() => JSON.parse(await response.text(), this.reviver)
      });
    }
    next();
  }
}

Given this info, let me know if you’d like me to leave this open / submit a PR — otherwise, I’m happy to close.
Thanks for the quick response and also for all of the work on this client!

On Sep 20, 2016, at 5:33 PM, Sashko Stubailo<EMAIL_ADDRESS>wrote:
Is there something you want to do in an afterWare that requires a promise? Just curious about the use case.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/apollostack/apollo-client/issues/682#issuecomment-248441398, or mute the thread https://github.com/notifications/unsubscribe-auth/ACOAMkBQsJJN0aIxySe1qLIJQeKQR_3qks5qsFEpgaJpZM4KCEdy.

I’ve actually figured out a workaround where I don’t need a Promise, but my actual use case is such that I want to specify my own reviver function to parse the JSON vs. receiving the output of response.json() (via query()) and then processing it from there.
I could write a custom network interface for this, but given the existing behavior for middleware, I wanted to see if honoring promises in the afterware should be supported.
My workaround is to just override the json function in the response with my own flavor ( via async/await vs. Promises just b/c this project is using es7 via Babel ):
{
  applyAfterware: ({response}, next) => {
    if (response.status === 200) {
      Object.defineProperty(response, 'json', {
        value: async() => JSON.parse(await response.text(), this.reviver)
      });
    }
    next();
  }
}

Given this info, let me know if you’d like me to leave this open / submit a PR — otherwise, I’m happy to close.
Thanks for the quick response and also for all of the work on this client!

On Sep 20, 2016, at 5:33 PM, Sashko Stubailo<EMAIL_ADDRESS>wrote:
Is there something you want to do in an afterWare that requires a promise? Just curious about the use case.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/apollostack/apollo-client/issues/682#issuecomment-248441398, or mute the thread https://github.com/notifications/unsubscribe-auth/ACOAMkBQsJJN0aIxySe1qLIJQeKQR_3qks5qsFEpgaJpZM4KCEdy.

@shanonvl A PR would be appreciated! I think it was a simple omission and not intentional, so we should fix it, even if there's a workaround.

recently fixed, I believe.


[CL]: Architecture.md term alignment

Background
We utilized various verbiage throughout the design of CL's tick spacing.
Roman suggested the following change, which was partially implemented in the tick spacing PR:

k_{p1} -> exponentAtPriceOne
k_incrementDistance -> geometricExponentIncrementDistanceInTicks
k_delta -> geometricExponentDelta
k_current -> exponentAtCurrentTick
currentIncrement -> currentAdditiveIncrementInTicks

Given that we agree on these new terms, we need to scrub through architecture.md, tests, and in line comments to ensure we are not using old terms (specifically k values)
https://github.com/osmosis-labs/osmosis/pull/3874#discussion_r1066563744
Suggested Design
Go with the new terms suggested by Roman
Acceptance Criteria

No more reference to old terms can be found in the code base

I went through the architecture.md and it appears all these terms are no longer used! Closing this issue


NP only shows on full activity and not when only a portion is selected

In a ride analysis, the normalised power for the whole ride is shown as such:

However, if you select only a portion of the ride the np value disappears.
This could potentially be quite a quick fix, so I'm submitting an issue. If it's rather complex I am happy to take a look at your code and see if I can implement.
It's a ripper extension, thanks so much,
Tom

Hey Tom,
NP should work for any durations over 20 minutes.  Andy Coggan doesn't recommend NP for anything less than about 20 minutes so the calcs only work with 1000 data points.  Are you not seeing it display for long selections too?
Here's the snippet of code...

Good call, I am seeing it for longer selections after giving it a go. Thanks for the info and sorry I didn't reply earlier, just forgot to reply. Cheers :)


gpflow GPR slow compared to sklearn's GaussianProcessRegressor

Bug / performance issue / build issue
I think this potential issue could be split into two parts. Therefore if you would like me to submit two issues, please let me know.
First of all, a comparison of sklearn's GaussianProcessRegressor model with gpflow's GPR model:
I was interested in comparing the accuracy and computational performance of the two model implementations aforementioned, on the toy dataset used in:
https://gpflow.readthedocs.io/en/develop/notebooks/advanced/gps_for_big_data.html
(i.e. a simple 1d example).
For both models I used the scipy l-bfgs optimiser- the sklearn model claims only to support this optimiser by default
Furthermore, using sklearn's GaussianProcessRegressor model meant that I had to fix both the kernel and likelihood variances of the gpflow GPR model (see code below), to make the examples comparable.
Upon training and making inferences with the two models, to my surprise I found that the gpflow model was slower than the sklearn equivalent.
I played with the number of training datapoints (N) and generally found that the results diverged more as I increased N. n.b. for the results below I used N=10000.
Secondly, using the tf probability implementation of l-bfgs with the gpflow GPR model:
As a possible workaround to the above case (though even if it were successful, it wouldn't answer it IMO) I tried using the tf probability implementation of the l-bfgs optimiser. To use this optimiser on the GPR model
I wrote a TfProb based on optimiser class based on gpflow.optimizers.scipy.minimize() (see code below).
While this provided some speedup compared to using the gpflow GPR with the scipy optimiser, it was still in general slower than the sklearn version.
Furthermore I found using the tf probability optimiser very buggy. See code below for details, but basically using it meant I suffered with two issues:
a) out of memory issues (when e.g. N=10000 without using GPU, which weren't experienced using scipy optimiser with either the sklearn or the gpflow models)
b) invertible matrix issues (particularly when reducing the value of sigma in the toy data to e.g. 0.1)- upon inspection of the GPR model the kernel lengthscale had been set to zero, explaining the invertibility issues.
The tl;dr here is that, in general it for the example considered here it seems the gpflow implementation of GPR is slower than sklearn, and I'm not sure whether this is down to implementation, or different methods being deployed
(in the latter case I would be curious to hear what the benefits of gpflow's implementation are given its seemingly slower runtime).
I also didn't know if you guys were aware of this, so I'm hoping I've bought it to light (and that the light isn't too "distorted" by my experiments :))
I have provided what I hope, is an understandable minimum working example (mwe) explaining both issues. If anything doesn't make sense in it, or anything needs adding please let me know.
In the experiments I conducted, I varied N ({1000, 2000, 5000, 10000}) and sigma i.e. the data noise {0.1, 0.2}.

To reproduce
Minimal, reproducible example

from sklearn.gaussian_process import GaussianProcessRegressor
import sklearn.gaussian_process.kernels as kernels
import numpy as np
import gpflow
from time import time
from gpflow.utilities import print_summary
from gpflow.models import GPR
from gpflow.models.util import data_input_to_tensor # not defined in gpflow versions earlier than 2.0.2
rng = np.random.RandomState(123)
import tensorflow as tf
import tensorflow_probability as tfp
tf.random.set_seed(42)

def func(x):
    return np.sin(x * 3 * 3.14) + 0.3 * np.cos(x * 9 * 3.14) + 0.5 * np.sin(x * 7 * 3.14)

N = 10000  # Number of training observations
sigma = 0.2 # change this to e.g. 0.1 to obtain invertible matrix error with tfp optimiser
X = rng.rand(N, 1) * 2 - 1  # X values
Y = func(X) + sigma * rng.randn(N, 1)  # Noisy Y values
data = (X, Y)
Xt = np.linspace(-1.0, 1.0, 1000)[:, None]
Yt = func(Xt)

# wrapper round gpflow gpr to fix LL variance
class GPReg(GPR):
    """
    same as gpflow.models.GPR but can pass likelihood object
    to e.g. fix likelihood variance
    """
    def __init__(
        self,
        data,
        kernel,
        mean_function,
        likelihood
        ):
        _, Y_data = data
        super(GPR, self).__init__(kernel, likelihood, mean_function, num_latent_gps=Y_data.shape[-1])
        self.data = data_input_to_tensor(data)

# set LL variance to true value, set LL/kernel variances to non-trainable as is in sklearn
gaussian_likelihood = gpflow.likelihoods.Gaussian(variance=sigma)
#sklearn matern kernel has unit variance afaik
matern_kernel = gpflow.kernels.Matern52(variance=1.0)
gpflow.set_trainable(gaussian_likelihood.variance, False)
gpflow.set_trainable(matern_kernel.variance, False)

# gpflow version with scipy optimiser

# train
gp_gpr = GPReg(data=(X,Y), kernel=matern_kernel, mean_function=None, likelihood=gaussian_likelihood)
scipy_opt = gpflow.optimizers.Scipy()
a = time()
scipy_opt_logs = scipy_opt.minimize(gp_gpr.training_loss, gp_gpr.trainable_variables)
b = time()
gp_scipy_gpr_time = b - a
print('gpflow with scipy optimiser training time')
print(gp_scipy_gpr_time)

# gpflow prediction
# predictions consistently slower than sklearn implementation
# with or without GPU(...!)
a = time()
gp_scipy_gpr_predictions = gp_gpr.predict_f(Xt)
b = time()
gp_scipy_gpr_time = b - a
print('gpflow with scipy optimiser prediction time')
print(gp_scipy_gpr_time)
# sklearn implementation
# train
sk_gpr = GaussianProcessRegressor(alpha=sigma, kernel=kernels.Matern(nu=2.5))
a = time()
sk_gpr.fit(X, Y)
b = time()
sk_gpr_time = b - a
print('sklearn training time')
print(sk_gpr_time)

# sklearn prediction
a = time()
sk_predictions = sk_gpr.predict(Xt, return_std = True, return_cov = False)
b = time()
sk_gpr_prediction_time = b - a
print('sklearn prediction time')
print(sk_gpr_prediction_time)

# learned parameters are ~same
print('sklearn learned length scale')
print(sk_gpr.kernel_.get_params()['length_scale'])
print('gpflow learned length scale')
print_summary(gp_gpr)

# gpflow version with tf optimiser
# wrapper for tfp l-bfgs based on gpflow.optimizers.scipy.minimize()

class TfProb:
    def minimize(
        self,
        closure,
        variables,
        **tf_prob_kwargs,
    ):
        variables = tuple(variables)

        initial_params = self.initial_parameters(variables)
        func = self.eval_func(closure, variables)

        return tfp.python.optimizer.lbfgs_minimize(
            func, initial_params, **tf_prob_kwargs)

    @classmethod
    def initial_parameters(cls, variables):
        return cls.pack_tensors(variables)

    @classmethod
    def eval_func(
        cls, closure, variables
    ):
        def _tf_eval(x):
            values = cls.unpack_tensors(variables, x)
            cls.assign_tensors(variables, values)
            loss, grads = _compute_loss_and_gradients(closure, variables)
            return loss, cls.pack_tensors(grads)
        return _tf_eval

    @staticmethod
    def pack_tensors(tensors):
        flats = [tf.reshape(tensor, (-1,)) for tensor in tensors]
        tensors_vector = tf.concat(flats, axis=0)
        return tensors_vector

    @staticmethod
    def unpack_tensors(to_tensors, from_vector):
        s = 0
        values = []
        for tensor in to_tensors:
            shape = tf.shape(tensor)
            tensor_size = tf.reduce_prod(shape)
            tensor_vector = tf.cast(from_vector[s : s + tensor_size], tensor.dtype)
            tensor_vector = tf.reshape(tensor_vector, shape)
            values.append(tensor_vector)
            s += tensor_size
        return values

    @staticmethod
    def assign_tensors(to_tensors, values):
        for tensor, tensor_vector in zip(to_tensors, values):
            tensor.assign(tensor_vector)

def _compute_loss_and_gradients(loss_closure, variables):
    with tf.GradientTape(watch_accessed_variables=False) as tape:
        tape.watch(variables)
        loss = loss_closure()
    grads = tape.gradient(loss, variables)
    return loss, grads

# create new instances of gpflow ll/kernel
gaussian_likelihood = gpflow.likelihoods.Gaussian(variance=sigma)
matern_kernel = gpflow.kernels.Matern52(variance=1.0)
gpflow.set_trainable(gaussian_likelihood.variance, False)
gpflow.set_trainable(matern_kernel.variance, False)

# train gpflow with tf optimiser
# OOM error with N = 10000; sigma = 0.2 without GPU
# OOM error with N = 5000; sigma = 0.2 with GPU
# Input matrix is not invertible error with sigma = 0.1
# (lengthscale has been trained to 0)

gp_gpr = GPReg(data=(X,Y), kernel=matern_kernel, mean_function=None, likelihood=gaussian_likelihood)
tf_opt = TfProb()
a = time()
tf_opt_logs = tf_opt.minimize(gp_gpr.training_loss, gp_gpr.trainable_variables)
b = time()
gp_gpr_tf_time = b - a
print('gpflow with tf optimiser training time')
print(gp_gpr_tf_time)

print_summary(gp_gpr)

Stack trace, or error message
Slower performance of gpflow implementation of GPR with scipy optimiser, compared with sklearn GaussianProcessRegressor,
for N=10000, sigma=0.2:
gpflow with scipy optimiser training time
1607.2928245067596

sklearn training time
214.8000783920288

gpflow prediction time
43.224738121032715

sklearn prediction time
12.143613338470459

Resources exhausted error which occurs when using the tf_probability l-bfgs optimiser for above setup:
---------------------------------------------------------------------------
ResourceExhaustedError                    Traceback (most recent call last)
<ipython-input-6-94dcf5feb61c> in <module>
      5 tf_opt = TfProb()
      6 a = time()
----> 7 tf_opt_logs = tf_opt.minimize(gp_gpr.training_loss, gp_gpr.trainable_variables)
      8 b = time()
      9 gp_gpr_tf_time = b - a

<ipython-input-5-c2e963d4ed8f> in minimize(self, closure, variables, **tf_prob_kwargs)
     16
     17         return tfp.python.optimizer.lbfgs_minimize(
---> 18             func, initial_params, **tf_prob_kwargs)
     19
     20     @classmethod

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/lbfgs.py in minimize(value_and_gradients_function, initial_position, num_correction_pairs, tolerance, x_tolerance, f_relative_tolerance, initial_inverse_hessian_estimate, max_iterations, parallel_iterations, stopping_condition, name)
    253                                        initial_position,
    254                                        num_correction_pairs,
--> 255                                        tolerance)
    256     return tf.while_loop(
    257         cond=_cond,

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/lbfgs.py in _get_initial_state(value_and_gradients_function, initial_position, num_correction_pairs, tolerance)
    269       value_and_gradients_function,
    270       initial_position,
--> 271       tolerance)
    272   empty_queue = _make_empty_queue_for(num_correction_pairs, initial_position)
    273   init_args.update(position_deltas=empty_queue, gradient_deltas=empty_queue)

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/bfgs_utils.py in get_initial_state_args(value_and_gradients_function, initial_position, grad_tolerance, control_inputs)
     80       f0, df0 = value_and_gradients_function(initial_position)
     81   else:
---> 82     f0, df0 = value_and_gradients_function(initial_position)
     83   converged = norm(df0, dims=1) < grad_tolerance
     84   return dict(

<ipython-input-5-c2e963d4ed8f> in _tf_eval(x)
     29             values = cls.unpack_tensors(variables, x)
     30             cls.assign_tensors(variables, values)
---> 31             loss, grads = _compute_loss_and_gradients(closure, variables)
     32             return loss, cls.pack_tensors(grads)
     33         return _tf_eval

<ipython-input-5-c2e963d4ed8f> in _compute_loss_and_gradients(loss_closure, variables)
     60     with tf.GradientTape(watch_accessed_variables=False) as tape:
     61         tape.watch(variables)
---> 62         loss = loss_closure()
     63     grads = tape.gradient(loss, variables)
     64     return loss, grads

~/anaconda3/lib/python3.7/site-packages/gpflow/models/training_mixins.py in training_loss(self)
     48         Returns the training loss for this model.
     49         """
---> 50         return self._training_loss()
     51
     52     def training_loss_closure(self, *, compile=True) -> Callable[[], tf.Tensor]:

~/anaconda3/lib/python3.7/site-packages/gpflow/models/model.py in _training_loss(self, *args, **kwargs)
     66         adds the log density of all priors to maximum_log_likelihood_objective().
     67         """
---> 68         return -(self.maximum_log_likelihood_objective(*args, **kwargs) + self.log_prior_density())
     69
     70     @abc.abstractmethod

~/anaconda3/lib/python3.7/site-packages/gpflow/models/gpr.py in maximum_log_likelihood_objective(self)
     53
     54     def maximum_log_likelihood_objective(self) -> tf.Tensor:
---> 55         return self.log_marginal_likelihood()
     56
     57     def log_marginal_likelihood(self) -> tf.Tensor:

~/anaconda3/lib/python3.7/site-packages/gpflow/models/gpr.py in log_marginal_likelihood(self)
     64         """
     65         X, Y = self.data
---> 66         K = self.kernel(X)
     67         num_data = X.shape[0]
     68         k_diag = tf.linalg.diag_part(K)

~/anaconda3/lib/python3.7/site-packages/gpflow/kernels/base.py in __call__(self, X, X2, full_cov, presliced)
    170
    171         else:
--> 172             return self.K(X, X2)
    173
    174     def __add__(self, other):

~/anaconda3/lib/python3.7/site-packages/gpflow/kernels/stationaries.py in K(self, X, X2)
     73     def K(self, X, X2=None):
     74         r2 = self.scaled_squared_euclid_dist(X, X2)
---> 75         return self.K_r2(r2)
     76
     77     def K_r2(self, r2):

~/anaconda3/lib/python3.7/site-packages/gpflow/kernels/stationaries.py in K_r2(self, r2)
     79             # Clipping around the (single) float precision which is ~1e-45.
     80             r = tf.sqrt(tf.maximum(r2, 1e-36))
---> 81             return self.K_r(r)  # pylint: disable=no-member
     82         raise NotImplementedError
     83

~/anaconda3/lib/python3.7/site-packages/gpflow/kernels/stationaries.py in K_r(self, r)
    210     def K_r(self, r):
    211         sqrt5 = np.sqrt(5.0)
--> 212         return self.variance * (1.0 + sqrt5 * r + 5.0 / 3.0 * tf.square(r)) * tf.exp(-sqrt5 * r)
    213
    214

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/math_ops.py in r_binary_op_wrapper(y, x)
    926     with ops.name_scope(None, op_name, [x, y]) as name:
    927       x = ops.convert_to_tensor(x, dtype=y.dtype.base_dtype, name="x")
--> 928       return func(x, y, name=name)
    929
    930   # Propagate func.__doc__ to the wrappers

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/math_ops.py in _mul_dispatch(x, y, name)
   1199   is_tensor_y = isinstance(y, ops.Tensor)
   1200   if is_tensor_y:
-> 1201     return gen_math_ops.mul(x, y, name=name)
   1202   else:
   1203     assert isinstance(y, sparse_tensor.SparseTensor)  # Case: Dense * Sparse.

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_math_ops.py in mul(x, y, name)
   6120         pass  # Add nodes to the TensorFlow graph.
   6121     except _core._NotOkStatusException as e:
-> 6122       _ops.raise_from_not_ok_status(e, name)
   6123   # Add nodes to the TensorFlow graph.
   6124   _, _, _op, _outputs = _op_def_library._apply_op_helper(

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6604   message = e.message + (" name: " + name if name is not None else "")
   6605   # pylint: disable=protected-access
-> 6606   six.raise_from(core._status_to_exception(e.code, message), None)
   6607   # pylint: enable=protected-access
   6608

~/anaconda3/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

ResourceExhaustedError: OOM when allocating tensor with shape[10000,10000] and type double on /job:localhost/replica:0/task:0/device:CPU:0 by allocator cpu [Op:Mul] name: minimize/mul/

Slower performance of training of gpflow implementation of GPR with scipy/tfp optimiser, compared with sklearn GaussianProcessRegressor,
for N=5000, sigma=0.2:
gpflow with scipy optimiser training time
221.29815769195557

gpflow with tf optimiser training time
67.4831428527832

sklearn training time
24.701717615127563

Invertible matrix error which occurs when using tf_probability l-bfgs optimiser with N=2000, sigma=0.1:
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-45-458991b45fe3> in <module>
      4 tf_opt = TfProb()
      5 a = time()
----> 6 tf_opt_logs = tf_opt.minimize(gp_gpr.training_loss, gp_gpr.trainable_variables)
      7 b = time()
      8 gp_gpr_tf_time = b - a

<ipython-input-44-c2e963d4ed8f> in minimize(self, closure, variables, **tf_prob_kwargs)
     16
     17         return tfp.python.optimizer.lbfgs_minimize(
---> 18             func, initial_params, **tf_prob_kwargs)
     19
     20     @classmethod

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/lbfgs.py in minimize(value_and_gradients_function, initial_position, num_correction_pairs, tolerance, x_tolerance, f_relative_tolerance, initial_inverse_hessian_estimate, max_iterations, parallel_iterations, stopping_condition, name)
    258         body=_body,
    259         loop_vars=[initial_state],
--> 260         parallel_iterations=parallel_iterations)[0]
    261
    262

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/control_flow_ops.py in while_loop_v2(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, maximum_iterations, name)
   2476       name=name,
   2477       maximum_iterations=maximum_iterations,
-> 2478       return_same_structure=True)
   2479
   2480

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/control_flow_ops.py in while_loop(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, name, maximum_iterations, return_same_structure)
   2712                                               list(loop_vars))
   2713       while cond(*loop_vars):
-> 2714         loop_vars = body(*loop_vars)
   2715         if try_to_pack and not isinstance(loop_vars, (list, _basetuple)):
   2716           packed = True

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/lbfgs.py in _body(current_state)
    236           current_state,
    237           value_and_gradients_function, search_direction,
--> 238           tolerance, f_relative_tolerance, x_tolerance, stopping_condition)
    239
    240       # If not failed or converged, update the Hessian estimate.

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/bfgs_utils.py in line_search_step(state, value_and_gradients_function, search_direction, grad_tolerance, f_relative_tolerance, x_tolerance, stopping_condition)
    151       initial_step_size=_broadcast(1, state.position),
    152       value_at_zero=val_0,
--> 153       converged=inactive)  # No search needed for these.
    154
    155   state_after_ls = update_fields(

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/linesearch/hager_zhang.py in hager_zhang(value_and_gradients_function, initial_step_size, value_at_initial_step, value_at_zero, converged, threshold_use_approximate_wolfe_condition, shrinkage_param, expansion_param, sufficient_decrease_param, curvature_param, step_size_shrink_param, max_iterations, name)
    253         value_at_initial_step,
    254         value_at_zero,
--> 255         threshold_use_approximate_wolfe_condition)
    256
    257     valid_inputs = (hzl.is_finite(val_0) & (val_0.df < 0) &

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/linesearch/hager_zhang.py in _prepare_args(value_and_gradients_function, initial_step_size, val_initial, val_0, approximate_wolfe_threshold)
    634     else:
    635       initial_step_size = np.float32(1.)
--> 636     val_initial = value_and_gradients_function(initial_step_size)
    637     eval_count += 1
    638

~/anaconda3/lib/python3.7/site-packages/tensorflow_probability/python/optimizer/bfgs_utils.py in _restricted_func(t)
    246     pt = position + t[..., tf.newaxis] * direction
    247     t = _broadcast(t, position)
--> 248     objective_value, gradient = value_and_gradients_function(pt)
    249     return ValueAndGradient(
    250         x=t,

<ipython-input-44-c2e963d4ed8f> in _tf_eval(x)
     29             values = cls.unpack_tensors(variables, x)
     30             cls.assign_tensors(variables, values)
---> 31             loss, grads = _compute_loss_and_gradients(closure, variables)
     32             return loss, cls.pack_tensors(grads)
     33         return _tf_eval

<ipython-input-44-c2e963d4ed8f> in _compute_loss_and_gradients(loss_closure, variables)
     60     with tf.GradientTape(watch_accessed_variables=False) as tape:
     61         tape.watch(variables)
---> 62         loss = loss_closure()
     63     grads = tape.gradient(loss, variables)
     64     return loss, grads

~/anaconda3/lib/python3.7/site-packages/gpflow/models/training_mixins.py in training_loss(self)
     48         Returns the training loss for this model.
     49         """
---> 50         return self._training_loss()
     51
     52     def training_loss_closure(self, *, compile=True) -> Callable[[], tf.Tensor]:

~/anaconda3/lib/python3.7/site-packages/gpflow/models/model.py in _training_loss(self, *args, **kwargs)
     66         adds the log density of all priors to maximum_log_likelihood_objective().
     67         """
---> 68         return -(self.maximum_log_likelihood_objective(*args, **kwargs) + self.log_prior_density())
     69
     70     @abc.abstractmethod

~/anaconda3/lib/python3.7/site-packages/gpflow/models/gpr.py in maximum_log_likelihood_objective(self)
     53
     54     def maximum_log_likelihood_objective(self) -> tf.Tensor:
---> 55         return self.log_marginal_likelihood()
     56
     57     def log_marginal_likelihood(self) -> tf.Tensor:

~/anaconda3/lib/python3.7/site-packages/gpflow/models/gpr.py in log_marginal_likelihood(self)
     73
     74         # [R,] log-likelihoods for each independent dimension of Y
---> 75         log_prob = multivariate_normal(Y, m, L)
     76         return tf.reduce_sum(log_prob)
     77

~/anaconda3/lib/python3.7/site-packages/gpflow/logdensities.py in multivariate_normal(x, mu, L)
     94
     95     d = x - mu
---> 96     alpha = tf.linalg.triangular_solve(L, d, lower=True)
     97     num_dims = tf.cast(tf.shape(d)[0], L.dtype)
     98     p = -0.5 * tf.reduce_sum(tf.square(alpha), 0)

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_linalg_ops.py in matrix_triangular_solve(matrix, rhs, lower, adjoint, name)
   1894         raise
   1895     except _core._NotOkStatusException as e:
-> 1896       _ops.raise_from_not_ok_status(e, name)
   1897   # Add nodes to the TensorFlow graph.
   1898   if lower is None:

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6604   message = e.message + (" name: " + name if name is not None else "")
   6605   # pylint: disable=protected-access
-> 6606   six.raise_from(core._status_to_exception(e.code, message), None)
   6607   # pylint: enable=protected-access
   6608

~/anaconda3/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: Input matrix is not invertible. [Op:MatrixTriangularSolve]

Upon inspection the kernel lengthscale is zero
Expected behaviour
Using the gpflow implementation of GPR with the scipy l-bfgs optimiser (which is what sklearn uses), without a GPU, I would expect the train/prediction times
to be roughly comparable with the sklearn implementation, for the example I posed here, which I tried to make as like-for-like as possible.
In my case using the tf_probability optimiser closed the gap a little on the sklearn implementation (when it worked) in my experience.
However when using the tf_probability optimiser, I didn't expect the gpflow GPR/ the optimiser to be so prone to OOM/non-invertible errors, though I stress again this may
be more of a tf_probability bug than gpflow.
Using a GPU (see below) also provided some speedup, but still gives surprising results IMO.
System information

GPflow version: 2.0.2
GPflow installed from: pip install gpflow
TensorFlow version: 2.1.0
Python version: 3.7.6
Operating system: Ubuntu 18.04.4 LTS

Additional context
Note I also ran the gpflow GPR model using the scipy optimser with compile=False (just in case evaluating the function and its derivative in tensorflow was somehow slowing things down), but found no improvement.
I also passed the bounds used by the sklearn implementation for the kernel lengthscale (array([[-11.51292546,  11.51292546]])), to the scipy optimiser in the gpflow GPR case, but again found this didn't help.
To my understanding, it doesn't seem that you can pass bounds to the tf probability optimiser.
I appreciate one of tensorflow's biggest strengths is its ability to execute on GPUs, but I think this makes the comparison with sklearn sort of meaningless.
Nevertheless I tried running the gpflow with scipy/tf optimisers on the GPU, and in general found the performance to be below what I expected:

gpflow with scipy optimiser still slower than sklearn for training
gpflow with tf_probability optimiser comparable with sklearn for training (but prone to bugs mentioned above)
gpflow slower than sklearn for making predictions

Additional system information:

CPU i7-8750H @ 2.20 GHz x 12
GPU Nvidia GTX 1050 Ti 4GB
RAM 16GB

Additional package versions:

tensorflow probability: 0.9.0
sklearn: 0.22.1
scipy: 1.4.1

Apologies if it isn't quite clear what I'm trying to convey, there are quite a few variables in this (data size, likelihood noise, which optimiser, with/without GPU etc.) but I've
tried to make my approach methodical. Please do shout out if anything isn't clear.
Thanks so much for any input on this!

Hi @SuperKam91, thanks for the thorough write-up. There is a crucial difference between the sklearn implementation and GPflow's that probably explains most of the performance difference: sklearn's GaussianProcessRegressor precomputes and stores the Cholesky of K and alpha = K^{-1} y (see sklearn source). In contrast, GPflow's GPR re-computes this on each call to predict_f. This is something that could (and should) be improved, but we haven't had the resources to carry out this task yet. (Help appreciated!). Try timing both the .fit() and .predict() together!
That said, in the cases where sklearn does what you want to do and is faster, of course it makes sense to use that. I believe GPflow comes through when you want to build more complex models, e.g. using sparse approaches (SGPR, SVGP), non-Gaussian likelihoods (VGP, SVGP), multi-output models, or implementing your own kernels and likelihoods (TensorFlow's autodiff).
I thought we already had open issues regarding the lack of storing pre-computable quantities for fast prediction but can't seem to find it right now...

(Also if you have a GPU available, then GPflow's ability to leverage that via TensorFlow can give a significant speed-up compared to CPU!)

A PR has just been merged into develop to extend the fast prediction functionality from the SVGP model to the GPR model: #1696. As such I am going to close this issue.

Hey @johnamcleod  -- are there examples on how the GPR model can be sped up?
Training a model with Input[5000, 4], Output[5000, 1], these are the speeds I am obtaining during one prediction call:

sklearn:  7.790e-03 s
GPflow: 2.110s

I can make an example script if needed. Currently i am not using a GPU, but I assume this might not be drastically improved even with a GPU.


Error when importing Foundation

NOOB question.Just starting with swift on windows. Can run helloworld program ok but when I import Foundation, get the following error.

c:\Swift\setPath.bat   & c:\Swift\usr\bin\swift.exe test2.swift
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "CoreFoundation.h"
         ^
c:\Swift\usr\lib\swift\CoreFoundation/CoreFoundation.h:101:10: note: in file included from c:\Swift\usr\lib\swift\CoreFoundation/CoreFoundation.h:101:
#include <CoreFoundation/ForSwiftFoundationOnly.h>
         ^
c:\Swift\usr\lib\swift\CoreFoundation/ForSwiftFoundationOnly.h:22:10: note: in file included from c:\Swift\usr\lib\swift\CoreFoundation/ForSwiftFoundationOnly.h:22:
#include <CoreFoundation/CFPriv.h>
         ^
c:\Swift\usr\lib\swift\CoreFoundation/CFPriv.h:571:74: error: variable has incomplete type 'struct timespec'
CF_INLINE CFAbsoluteTime _CFAbsoluteTimeFromFileTimeSpec(struct timespec ts) {
                                                                         ^
c:\Swift\usr\lib\swift\CoreFoundation/CFPriv.h:571:65: note: forward declaration of 'struct timespec'
CF_INLINE CFAbsoluteTime _CFAbsoluteTimeFromFileTimeSpec(struct timespec ts) {
                                                                ^
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "CoreFoundation.h"
         ^
c:\Swift\usr\lib\swift\CoreFoundation/CoreFoundation.h:101:10: note: in file included from c:\Swift\usr\lib\swift\CoreFoundation/CoreFoundation.h:101:
#include <CoreFoundation/ForSwiftFoundationOnly.h>
         ^
c:\Swift\usr\lib\swift\CoreFoundation/ForSwiftFoundationOnly.h:22:10: note: in file included from c:\Swift\usr\lib\swift\CoreFoundation/ForSwiftFoundationOnly.h:22:
#include <CoreFoundation/CFPriv.h>
         ^
c:\Swift\usr\lib\swift\CoreFoundation/CFPriv.h:575:27: error: incomplete result type 'struct timespec' in function definition
CF_INLINE struct timespec _CFFileTimeSpecFromAbsoluteTime(CFAbsoluteTime at) {
                          ^
c:\Swift\usr\lib\swift\CoreFoundation/CFPriv.h:575:18: note: forward declaration of 'struct timespec'
CF_INLINE struct timespec _CFFileTimeSpecFromAbsoluteTime(CFAbsoluteTime at) {
                 ^
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "CoreFoundation.h"
         ^
c:\Swift\usr\lib\swift\CoreFoundation/CoreFoundation.h:101:10: note: in file included from c:\Swift\usr\lib\swift\CoreFoundation/CoreFoundation.h:101:
#include <CoreFoundation/ForSwiftFoundationOnly.h>
         ^
c:\Swift\usr\lib\swift\CoreFoundation/ForSwiftFoundationOnly.h:22:10: note: in file included from c:\Swift\usr\lib\swift\CoreFoundation/ForSwiftFoundationOnly.h:22:
#include <CoreFoundation/CFPriv.h>
         ^
c:\Swift\usr\lib\swift\CoreFoundation/CFPriv.h:576:20: error: variable has incomplete type 'struct timespec'
   struct timespec ts;
                   ^
c:\Swift\usr\lib\swift\CoreFoundation/CFPriv.h:575:18: note: forward declaration of 'struct timespec'
CF_INLINE struct timespec _CFFileTimeSpecFromAbsoluteTime(CFAbsoluteTime at) {
                 ^
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "CoreFoundation.h"
         ^
c:\Swift\usr\lib\swift\CoreFoundation/CoreFoundation.h:101:10: note: in file included from c:\Swift\usr\lib\swift\CoreFoundation/CoreFoundation.h:101:
#include <CoreFoundation/ForSwiftFoundationOnly.h>
         ^
c:\Swift\usr\lib\swift\CoreFoundation/ForSwiftFoundationOnly.h:31:10: error: 'pthread.h' file not found
#include <pthread.h>
         ^
<unknown>:0: error: could not build C module 'CoreFoundation'
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "../../../../../mingw64/x86_64-w64-mingw32/include/assert.h"
         ^

Thanks for using this program.
Could you show me your test2.swift?

Tried sample one
import Foundation
let number = 16.0
print("\(number) squared is \(number * number), and its square root is \(sqrt(number))")

For some reason it says it can't find pthread.h

Facing an error when adding 'import Foundation' :
Note: in file included from :1:
#include "CoreFoundation.h"
^
c:\Swift\mingw64\lib\swift\CoreFoundation/CoreFoundation.h:25:10: error: 'assert.h' file not found
#include <assert.h>
^
:0: error: could not build C module 'CoreFoundation'
:1:10: note: in file included from :1:
#include "../../../../../mingw64/x86_64-w64-mingw32/include/assert.h"

@starmanthanjain8999 , i am facing that same issue. if you found out whats wrong can you please revert with the answer


MT6889Z

Waiting for bootrom...[2021-04-05 10:35:12.303609] Waiting for bootrom
Traceback (most recent call last):
[2021-04-05 10:35:33.746503] Found port = COM3
File "C:\ProgramData\obexs\Data\main.py", line 169, in
main()
File "C:\ProgramData\obexs\Data\main.py", line 63, in main
raise e
File "C:\ProgramData\obexs\Data\main.py", line 56, in main
config = Config().default(hw_code)
File "C:\ProgramData\obexs\Data\src\config.py", line 14, in default
self.from_file(config, hw_code)
File "C:\ProgramData\obexs\Data\src\config.py", line 27, in from_file
raise NotImplementedError("Can't find {} hw_code in config".format(hw_code))
NotImplementedError: Can't find 0x816 hw_code in config
Hello dear and god programmer, is there a any chance / possibility that there will be possible a bypass for the MT6889Z { Mediatek Dimensity 1000+ } chipset in the future?
so that we can lift realme phones and other branded devices with this chipset from the world of the dead, I mean, with our devices in a state of hard bricked?

here are the dump: https://drive.google.com/file/d/1qse1C3gSpd29NFmp6QLZuMD6SfaUfEY5/view?usp=sharing

https://drive.google.com/file/d/1-5C_XaUC2gtd4h0F2VjJr4eCrAZUPbCa/view?usp=drivesdk
Replace default_config.json5 with my own and place mt6889_payload.bin to payloads/
var_1 may be incorrect

Added support in https://github.com/MTK-bypass/exploits_collection/commit/61bd3f363219a45b3d58446565c3997cae12e2fd


Secrets Store documentation

Documented the new service EGI Secrets Store.
Added a page with the service's design/architecture, but set it as draft. Should we publish it?

@thebe14 can you remove the draft state so that we can see the live preview?
I guess it's fine to include all the relevant information that can help users.

All this look great, I think it would be useful to publish the architecture too. I've seen a few sentences that may benefit from some small modifications, will try to do a proper review and suggestions later.
On a side note, I'm a bit uncomfortable with the marketing-like/overselling stuff on the main security page, maybe we should try to be a bit more sober?
Thanks!

removed

The failed link check is because the service seems to be down today.

@enolfc , @andrea-manzi can you please have a look?

one thing, i see that https://vault.services.fedcloud.eu:8200/ is the endpoint of the service. Maybe we could try to have an URL similar to the name of the service, like secrets.egi.eu  ?

one thing, i see that https://vault.services.fedcloud.eu:8200/ is the endpoint of the service. Maybe we could try to have an URL similar to the name of the service, like secrets.egi.eu ?

I asked Viet if we can have an endpoint without the port. He said yes, waiting for the implementation. And once that is ready, we can have whatever domain name we want. But let's not hold up this PR for that, will update when it becomes available.

one thing, i see that https://vault.services.fedcloud.eu:8200/ is the endpoint of the service. Maybe we could try to have an URL similar to the name of the service, like secrets.egi.eu ?

I asked Viet if we can have an endpoint without the port. He said yes, waiting for the implementation. And once that is ready, we can have whatever domain name we want. But let's not hold up this PR for that, will update when it becomes available.

yes sure, that change can be done later

@gwarf Do you know who owns egi.eu? An alias record secrets.egi.eu to point to vault.service.fedcloud.eu would be great.

We own and manage this domain ourselves, it could be doable in theory if/as this is an EGI owned service. Still a bit confused about the usage/management of that particular fedcloud.eu domain.
We can discuss a possible domain under egi.eu via other channels.


Fix log10 for float32

@vincentqb @cpuhrsch We got a very interesting result. Using float64 make the test result closer but still not within the range of the original test tolerance.
actual = tensor([[-24.5305, -27.5407, -57.3955,  ..., -57.3955,  -9.3496,  -5.2270],
        [-22.7423, -24.7428, -28.5632,  .....3955,  ..., -57.3955, -51.0177, -45.8937],
        [-57.3955, -57.3955, -57.3955,  ..., -57.3955, -52.8348, -46.1031]])
expected = tensor([[-24.5304, -27.5407, -57.3955,  ..., -57.3955,  -9.3496,  -5.2270],
        [-22.7423, -24.7429, -28.5633,  .....3955,  ..., -57.3955, -51.0175, -45.8940],
        [-57.3955, -57.3955, -57.3955,  ..., -57.3955, -52.8340, -46.1029]])
rtol = 1e-05, atol = 0.005, equal_nan = True
msg = 'Not within tolerance rtol=1e-05 atol=0.005 at input[95, 1061] (-57.3322639465332 vs. -57.342262268066406) and 1 other locations (0.00%)

I changed the function now to do the recasting at the end of the calculations.


Upload BPER coin

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

[x] Filename in the lowercase format
[x] Image format: png
[x] Image size: 256 by 256 px

Upload BPER COIN

Added correct in correct format here https://github.com/TrustWallet/tokens/commit/c83e3fb6d9afd12e4ad9beeb9c41b9b774223149


Updates to support streaming writes to flat files

Enable JDBC extract parallelization
Enable streaming
Support streaming writes to JSONL file
Add schema and set row schema
Add example config file

Enable JDBC extract parallelization

I am reverting this particular change as it is actually counterproductive. What this specific method does is it tries to reshuffle batches evenly between workers. While this sounds good on paper, what it does in function is it drastically increases memory load, as we already (roughly) evenly spread this out with our batching strategy beforehand w/ pagination and offsets.
Additionally, on some runners and depending on individual configuration, this can cause the entire ResultSet to be loaded into memory, as that is required for shuffling (can't shuffle what you don't already have in memory), which is not feasible for reasons that may be obvious

Support streaming writes to JSONL file
This is a good idea and I will most likely merge this in, with some caveats:

I will add a commit move this into its own separate Extract (e.g., JSONExtract and CSVExtract), instead of full-on replacing CSV functionality. CSV will still be desired in many use cases
Potential concern as CSV is what is being sent to N3C, you would have to do additional processing post-hoc if you wanted to do this to convert. If the concern is newlines in snippet, I would note that snippets are supposed to be truncated from the N3C submission anyways.
Removal of the dependency on streaming datasource

Thanks Andrew! Your points are valid and I agree them. Feel free to make the changes.

I did notice the JDBC extract parallelization seemed constrained into batches, which like you said helps optimize for memory, so that change I made is unnecessary. It can help ease some backpressure, which I noticed was high for the extract step.
I was trying to load everything back into SQL and use JOINs and ROW NUMBER OVER() for the note_nlp_id and other fields from the source table. But this is wasteful when it can be done in one go. I believe the id assigning can be done in the final step before the write, during collection, since the order doesn't matter, so it is possible to write to the expected CSV output directly without any slowdown.


set default type to undefined to fix and close #1793

Q
A

Fixed Issues?
Fixes #1793

Patch: Bug Fix?
Yes

Major: Breaking Change?
No

Minor: New Feature?
No

Tests Added + Pass?
Yes

Any Dependency Changes?
No

added default types as 'undefined':
before:
after: <T = undefined>

We have type tests here - https://github.com/redux-saga/redux-saga/tree/928c6fbdcd73405690a8f3ef1483f172c3a49bdf/packages/core/tests/typescript . Please add tests so we can know the context of this change.

Hello @Andarist @rami-res
I'm not sure redux-saga should change. It's a problem with redux-saga-test-plan as I've explained here : https://github.com/redux-saga/redux-saga/issues/1793#issuecomment-467975398

Closing because we haven't received enough information about why the change should be included.


Allow setting secret key by referencing a present/custom secret

Hello @ralgozino & @Al-Pragliola
Right now you have to set the config.secretKey in Helm values in plain text to set the GPM_SECRET_KEY env var. I thought it would be beneficial to allow users to set the env var by referencing a secret that can be deployed separately.
To make this possible, I refactored the Helm Chart slightly. Now it's possible to set config.secretRef in Helm values. If this field is set (and config.secretKey is set to null), the GPM_SECRET_KEY env var in deployment is set from the custom secret (not the default one).
I have tested the changes in a production-grade k8s cluster.

@sudoleg FYI: the chart including your proposed changes has been released:
https://github.com/sighupio/gatekeeper-policy-manager/releases/tag/gatekeeper-policy-manager-0.11.0


Adding support for RuntimeMethodHandle nodes.

This is again part of the larger GVM work, which I'm splitting into multiple PRs.
This change just enables the RMH support, but won't emit any RMH structures yet.

@davidwrighton PTAL

:clock1:


chore: add docker to dependabot config

I have notice that some of the dockerfiles are slightly out of date. This PR updates the dependabot configuration, to keep them updated.

@abdfnx do you have some idea why dependabot does not update this:
https://github.com/abdfnx/botway/blob/eb6a6a095fc2247084a4ef1dc5106c6181301f7b/docker/gp-image.dockerfile#L1
It seems to be out of date - the newest available is gitpod/workspace-base/2023-11-02-17-35-17.

@abdfnx do you have some idea why dependabot does not update this:
https://github.com/abdfnx/botway/blob/eb6a6a095fc2247084a4ef1dc5106c6181301f7b/docker/gp-image.dockerfile#L1
It seems to be out of date - the newest available is gitpod/workspace-base/2023-11-02-17-35-17.

Yep, it's very weird, I'll find a solution, thanks for your comment

@abdfnx do you have some idea why dependabot does not update this:
https://github.com/abdfnx/botway/blob/eb6a6a095fc2247084a4ef1dc5106c6181301f7b/docker/gp-image.dockerfile#L1
It seems to be out of date - the newest available is gitpod/workspace-base/2023-11-02-17-35-17.

Yep, it's very weird, I'll find a solution, thanks for your comment

As far as I read, the line directory: "/" does not make the dependabot to scan all of the directories recursively (with an exception of /.github/workflows in case of package-ecosystem: "github-actions"). I have found this:
https://github.com/dependabot/dependabot-core/issues/2824#issuecomment-741693711
Should I update the dependabot config, to have "docker" entry for every directory containing dockerfiles?

No problem man


Kotlin 2.0.20

Maintains compatibility with 2.0.0 via reflection.

Hmm, failure of the 2.0.10-RC sample build suggests that backwards compatibility may be a problem even with the reflection implemented.

Gonna have to drop the compatibility testing for this one. I'll leave the reflection compat code in place in case it's possible to work around this, but I won't be able to guarantee backwards compatibility.

> Failed to apply plugin 'org.jetbrains.kotlin.multiplatform'.
> Could not create an instance of type org.jetbrains.kotlin.gradle.dsl.KotlinMultiplatformExtension.
Could not generate a decorated class for type KotlinMultiplatformExtension.
Cannot have abstract method KotlinMultiplatformSourceSetConventions.getAndroidInstrumentedTest().


Continuous text/ initial start position?

Love this plugin, it's exactly what I'm looking for. I've got 4 tickers stacked,  I have the duration set to 20000 as I"m looking for a very very slow ticker.
Here's the problem, with it sliding in from the right, it takes about a minute to reach:

Of course, it's an extremely slow ticker. But I would like it to start from the position left aligned and slide from there, e.g.

also, when the text ends, you get a long blank period before the text repeats. Any way of reducing this gap to zero so that the text is continuous?
I'm happy to roll up my sleeves and have a go at updating the code if you can give me some pointers and I could submit a PR.
One other thing FYI I am occasionally getting some flickering and data cut off with long text - this one counts from item 1 to item 100:

Hey @tigger04,
I basically added support for this in #4
The usage would look like this:
Marquee {
    Text(...)
}
.marqueeDelay(20_000)
.marqueeBoundary(.inner)


Tensorflow/Keras framework does not seem to be supported

In doing some experimentation, chassis throws a pickling error when trying to build an image from a tf/keras model.
Since mlflow has built direct support for both tf and keras (built in "flavors"), and chassis currently uses the generic mlflow.pyfunc flavor, this issue may be resolvable if a new flavor was created, say mlflow-tf or something like this, that leverages the mlflow.tensorflow or mlflow.keras ml model flavors directly.
This new flavor would essentially be a duplicate of the existing chassis/service/flavours/mlflow flavor, but would use the tf/keras flavor directly.

More detailed information:
The main difference between mlflow.pyfunc.save_model and mlflow.tensorflow.save_model is the TF flavor requires an already serialized TF model, saved using the tf.saved_model.save method. Moreover, the mlflow.tensorflow.save_model method takes three TF-specific as arguments: (1) <some-path> that contains the properly saved tf model, (2) a list of tf metagraph tags, and (3) a signature definition key, which identifies the input/output signature associated with the model. These three items are directly associated with the serialized TF model.
As a result, this part of the Chassis SDK will not work for TF models. Currently, this code takes in a ChassisModel object as the python_model parameter to the mlflow.pyfunc.save_model method. Two things need to be changed to support TF-specific models:

The construction of the ChassisModel object needs to be changed (or can take optional TF specific values).
When the mlflow save model execution occurs, TF models need to be saved using the mlflow.tensorflow.save_model method instead of the generic pyfunc method. This would assume the proper TF objects are accessible, most realistically through the ChassisModel object.

Notes following discussion on this issue 3/24:

It appears that unlike the generic mlflow.pyfunc model class, where you can customized the predict method (passed through in Chassis as process_fn or batch_process_fn, saving a model in mlflow.tensorflow format does not allow you to overwrite the predict method.
To successfully save, load, and then run predict on a tensorflow model in mlflow format, it must be saved as mlflow.tensorflow then loaded back and run inference in three very specific lines of code.
We likely need to write another custom class where we inherit from the mlflow.tensorflow class and then overwrite the predict method.

Hi there, I was wondering if this ticket was closed as completed? I am trying to build a TensorFlow model with Chassis now that I have successfully built a sci-kit model into a container. The documentation page says "Coming Soon" under the examples for TF + Keras. Could someone help point me in the right direction to get started with building TF through Chassis?

Hi @brooks-ulap, thanks so much for your question! Do you happen to be working with a TF v2 model (e.g., something like this https://www.tensorflow.org/tutorials/quickstart/beginner#build_a_machine_learning_model)?
If so, that should be supported. We have not yet put an example on the docs site, but here's a quick block of sample code that should put you in the right direction.
Let's use this tutorial as an example model: https://www.tensorflow.org/tutorials/images/cnn.
Assuming you follow this in a notebook, you could append this chassis code to build a container image from it:
import chassisml

# define process function
def process(input_bytes):

    img = tf.io.decode_image(input_bytes, channels=3)
    img = tf.expand_dims(img, 0)
    img = tf.image.resize(img, [32,32])

    preds = model.predict(img)

    return preds

# instantiate chassis client and create model object
chassis_client = chassisml.ChassisClient("https://chassis.app.modzy.com/")
chassis_model = chassis_client.create_model(process_fn=process)

# test model with sample iamge
sample_filepath = "image.jpg" # assumes you have a ciphar sample image saved as "./image.jpg"
results = chassis_model.test(sample_filepath)
print(results)

# publish model
dockerhub_user = "<insert-docker-user>"
dockerhub_pass = "<insert-docker-pass>"

response = chassis_model.publish(
    model_name="TF Keras Test Image Classification",
    model_version="0.0.1",
    registry_user=dockerhub_user,
    registry_pass=dockerhub_pass,
)


how to use envoy to proxy thrift request

how can i use envoy to proxy thrift request,
is there a example config for thrift?

@zuercher

can you offer a detailed config file for envoy thrift?


Additions to have dispute tokens

Hi @grunch , @Catrya
added some data in structs SmallOrder and in Peer enum to have the dispute tokens.
Simply a quick implementation, maybe we can do it better. Looking here seems that tokens are not saved on a db, so I did a quick generation of random number in the u16 range and sent the message to buyer, seller and admin who takes dispute. Messages are sent when admin takes the dispute, don't know if this is the desired behaviour and maybe we need to send messages when dispute is opened.
Let me know.
Did not test now because I will be AFK til the evening now.

maybe we need to send messages when dispute is opened.

Yes, the buyer and seller must know their dispute token when the dispute is opened, so that in case an admin impersonator writes to them, they ask for the token and they know that it is a scammer because they will not be able to give it to them.


Three.js shadow support

Shadows don't currently work.

Removed the bug label, referenced issue closed. Does this mean ... we have shadows on your branch @kkananen ? :)

Not yet - just that it's not a bug as such, more like a feature :)


Changing xy_5 to plot data using fractional dates

Closes #153
In the NCL version of the xy_5 example, the datafile represents dates in a YYYYMM format. To make the plot, the NCL example first converts the dates into floats using a format where the decimal component represents a fraction of the year in place of months.
The GeoCAT version of xy_5 uses another method to accurately plot the date while also converting the dates; however the fractional dates are never used and are also calculated incorrectly. This PR makes the GeoCAT example more accurate and consistent with the NCL one.

Visually the plot is unchanged.


Unarmed Damage Die Implementation

Type
What type of pull request is this? (e.g., Bug fix, Feature, Refactor, etc.)

[x] Bug fix
[x] Feature
[ ] Refactor
[ ] Other (please describe):

Description
Add logic to calculate damage die for unarmed strikes.
Related Issue
Closes #91
How Has This Been Tested?
Built and ran on local foundry. Screenshots below.
Screenshots (if applicable)
I had one rank in Athletics to confirm damage scales with Athletics but die scales with Str attribute.
Str 6

Str 1

Str 4

Str 8

Str 9

Checklist:

[ ] I have commented on my code, particularly in hard-to-understand areas.
[ ] My changes do not introduce any new warnings or errors.
[ ] My PR does not contain any copyrighted works that I do not have permission to use.
[ ] I have tested my changes on Foundry VTT version: [insert version here].

Additional context
Add any other context or information here that would be useful for reviewers.

I have some concern about this implementation in that this means nobody can change the unarmed strike damage at all (outside of effects that apply to all damage). I see 2 probably better solutions:

Make the unarmed strike damage formula a roll formula visible in the item's damage field, using math functions to calculate the correct amount of dice. This will be a really long formula probably, so it would look really ugly, but it would at least be easily changeable.

Store the possible values of the unarmed strike as a range in CONFIG.COSMERE. That way it can be edited by a world script for users who wish to do so, and it also makes it so its not buried somewhere in the code that may be annoying to find.

I have some concern about this implementation in that this means nobody can change the unarmed strike damage at all (outside of effects that apply to all damage). I see 2 probably better solutions:
1. Make the unarmed strike damage formula a roll formula visible in the item's damage field, using math functions to calculate the correct amount of dice. This will be a really long formula probably, so it would look really ugly, but it would at least be easily changeable.

2. Store the possible values of the unarmed strike as a range in CONFIG.COSMERE. That way it can be edited by a world script for users who wish to do so, and it also makes it so its not buried somewhere in the code that may be annoying to find.

That is a valid concern. I like option 2, I can work on implementing that. If option 1 is preferable over 2 I'm cool with that as well.

@MangoFVTT I added the possible die values of unarmed strike to CONFIG.COSMERE and am using that to determine unarmed damage. I've tested for all possible STR values and it's working for me locally.
Feel free to give me any suggestions or nits you want, this is my first time developing in Foundry so I'm out of my element!

I have some further concerns that apply to the current implementation as well. I think the approach of having the unarmed strike damage depend on the id of the item is flawed. It's opaque to the user why the unarmed strike deals the damage it does and ignores the formula.
Having some item just ignore its config and deal some arbitrary amount of damage (for most users at least), just feels weird.
I think we should solve this using the formula. Though I am aware that fitting a function to the table may not be easy.
On another note; We should definitely move the other "tables" that we use for the derived statistics to the config as well. I don't believe they are right now.

@stanavdb Perhaps the best way to do it would be to have the unarmed strike scale be an @'able property? In the dnd5e system for example you can refer to things like @barbarian.rage.scale that just automatically calculates the correct number but still allows you to call it in any damage field without hardcoding it to an item name or specific item. We could do the same here.

I'm going to see if I can figure out how to do this! I should be able too. If it's too steep a learning curve for me, I'll let you know so it can be taken care of by someone more fitting.

I'm going to see if I can figure out how to do this! I should be able too. If it's too steep a learning curve for me, I'll let you know so it can be taken care of by someone more fitting.

It should be fairly easy. Just move the calculations over to the getRollData function on the actor document.

I'm converting this a draft, as its still in progress


Add a function to refresh broker list in client

Add a function to refresh broker list in client.
Fix: https://github.com/danielqsj/kafka_exporter/issues/134#issuecomment-624670818

@readygodom kafka_exporter is using github.com/Shopify/sarama v1.22.1 which is missing the fix #1645 from @Stephan14 - Remove broker(s) which no longer exist in metadata
I think if you just bump kafka_exporter to latest Sarama release then your original
problem will be fixed already

We can use new release after fix:https://github.com/Shopify/sarama/pull/1645 to resolve this problem @Shini31 https://github.com/danielqsj/kafka_exporter/issues/134#issuecomment-624670818


BUG: Login Page is not Responsive

Is there an existing issue for this?

[X] I have searched the existing issues

What went down?
The Login page is not able to work in mobiles or Tabs. I want to make the login page responsive for all devices which will add new look and feel for the website.

Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@SUGAM-ARORA Please assign me this issue, I want to work on it.

@saketh-05 Issue assigned to you


Error loading plugin homebridge-LEDStrip

Hi!
I've problem with pigpio at initialise.
I really need homebridge plugin that works with pigpio. Do you know what is wrong? I tried also reinstalling pigpio in homebridge-LEDStrip folder.
Thank you!

I sorry i cant help you with that i threw this plugin together a year ago. And didnt really use it afterwards.


Setting kube-api-qps and kube-api-burst are not working

Report
Sets kube-api-qps and kube-api-burst for KEDA adapter will not take effective.
It was set here
	cfg, err := config.GetConfig()
	if cfg != nil {
		cfg.QPS = adapterClientRequestQPS
		cfg.Burst = adapterClientRequestBurst
		cfg.DisableCompression = disableCompression
	}

However never got used, when new Manager, it will use another config
         mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
		MetricsBindAddress: metricsBindAddress,
		Scheme:             scheme,
		Namespace:          namespace,
		LeaseDuration:      leaseDuration,
		RenewDeadline:      renewDeadline,
		RetryPeriod:        retryPeriod,
	})

Expected Behavior
Setting kube-api-qps and kube-api-burst should be working
Actual Behavior
Setting kube-api-qps and kube-api-burst are not working
Steps to Reproduce the Problem

Pass kube-api-qps and kube-api-burst as args, as following

- /usr/local/bin/keda-adapter
        - --secure-port=6443
        - --logtostderr=false
        - --log_file=/tmp/apiserver.log
        - --log_dir=/tmp/log
        - --kube-api-qps=70
        - --kube-api-burst=100

Logs from KEDA operator
N/A
KEDA Version
2.8.1
Kubernetes Version
1.23
Platform
Any
Scaler Details
No response
Anything else?
No response

The main branch code is using the correct config.
This feature was added recently, and will be released as part of KEDA v2.9.

The main branch code is using the correct config.
https://github.com/kedacore/keda/blob/af948a5a8f72bb060b1757a3b33c2a3ad4616a52/main.go#L117-L122
This feature was added recently, and will be released as part of KEDA v2.9.

Ah, I am dumb, you're talking about the metrics server, where I can see it's not being used.
https://github.com/kedacore/keda/blob/af948a5a8f72bb060b1757a3b33c2a3ad4616a52/adapter/main.go#L127

Good catch, thanks for reporting.

Relates to https://github.com/kedacore/keda/issues/3730

Relates to #3730

Thanks @zroubalik for the fixing! I was also planning to submit a PR for this:)