Add nonlinear battery example to NonlinearProblem

This is an example nonlinear problem resulting from JuliaSimCompiler with 33 equations whose unknowns span ~13 orders of magnitude. The solution is quite sensitive to the algorithm choice and error tolerances. 15/20 of the tested algorithms failed to solve this problem (the solution L2-norm should be ~8e-7):
[Warn] Solver Newton Raphson (No line search) returned retcode Unstable with an residual norm =<PHONE_NUMBER>036127.
[Warn] Solver Newton Raphson (Hager & Zhang line search) returned retcode MaxIters with an residual norm =<PHONE_NUMBER>1352945.
[Warn] Solver Newton Raphson (More & Thuente line search) returned retcode MaxIters with an residual norm =<PHONE_NUMBER>669025.
[Warn] Solver Newton Raphson (Nocedal & Wright line search) returned retcode MaxIters with an residual norm =<PHONE_NUMBER>256991.
[Warn] Solver Newton Trust Region returned retcode MaxIters with an residual norm = 87.55359385515213.
[Info] Solver Newton Trust Region (NLsolve radius update) successfully solved the problem.
[Warn] Solver Newton Trust Region (Nocedal Wright radius update) returned retcode MaxIters with an residual norm = 87.55359358466573.
[Warn] Solver Newton Trust Region (Hei radius update) returned retcode MaxIters with an residual norm = 87.72887154657568.
[Warn] Solver Newton Trust Region (Yuan radius update) returned retcode ConvergenceFailure with an residual norm =<PHONE_NUMBER>036127.
[Info] Solver Newton Trust Region (Bastin radius update) successfully solved the problem.
[Warn] Solver Newton Trust Region (Fan radius update) returned retcode MaxIters with an residual norm =<PHONE_NUMBER>6402642.
[Info] Solver Levenberg-Marquardt (α_geodesic=0.75) successfully solved the problem.
[Warn] Solver Levenberg-Marquardt (α_geodesic, with CCholesky) returned retcode MaxIters with an residual norm = 0.0004065143786534784.
[Info] Solver Levenberg-Marquardt (α_geodesic=0.5) successfully solved the problem.
[Warn] Solver Levenberg-Marquardt (α_geodesic=0.5, with CCholesky) returned retcode MaxIters with an residual norm = 0.0004065143786534784.
[Warn] Solver Modified Powell (CMINPACK) threw an error: ErrorException("cfunction: closures are not supported on this platform").
[Warn] Solver Levenberg-Marquardt (CMINPACK) threw an error: ErrorException("cfunction: closures are not supported on this platform").
[Info] Solver Newton Raphson (NLSolveJL) successfully solved the problem.
[Warn] Solver Newton Trust Region (NLSolveJL) returned retcode Failure with an residual norm =<PHONE_NUMBER>403987.
[Warn] Solver Newton-Krylov (Sundials) had a residual of NaN.

cc @ChrisRackauckas

One suggestion on the above changes is that L2 errors > 1e-3 should not be considered converged. Can we do something to exclude those results?

I will create a PR with these changes. It seems MINPACK was returning converged even when it did not :shrug:

Continuing with https://github.com/SciML/SciMLBenchmarks.jl/pull/792


erro "no CUDA-capable device is detected"

error while running CUDA_VISIBLE_DEVICES=${GPU_ID} ./tools/demo.py
I am developing everything in this image with nvidia grid k520
Following are the details
if I run nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.81                 Driver Version: 384.81                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GRID K520           Off  | 00000000:00:03.0 Off |                  N/A |
| N/A   25C    P8    18W / 125W |     11MiB /  4036MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
What can be the issue?

did you install the gpu version of tf?

Did you set the right GPU_ID? It should be 0 if you only have one GPU. I met the same error when I forget execute command "GPU_ID=0" before testing.

Please export your cuda path manually when you run by other account
or pip install TensorFlow-gpu


Feat add graph simplifier

Proposed changes
Issue Number: close #13903
Problem summary
Add a graph simplifier that can simplfy the hypergraph according cost.
Now we only support inner join, the very simple cost function.
And the circle detector has not been added
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[x] No
[ ] I don't know

Has unit tests been added:

[x] Yes
[ ] No
[ ] No Need

Has document been added or modified:

[ ] Yes
[x] No
[ ] No Need

Does it need to update dependencies:

[ ] Yes
[x] No

Are there any changes that cannot be rolled back:

[ ] Yes (If Yes, please explain WHY)
[x] No

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 38.2 seconds
load time: 571 seconds
storage size:<PHONE_NUMBER>3 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221102115816_clickbench_pr_38296.html


[GithubActions] Test old ASDF minor versions

Changes
Run asdf tests against all (supported) minor versions of ASDF.
Related Issues
Related #834, #829

the pip version takes priority over the conda install version I assume?
We could also add an exclude option to pydeps2env to do these kind of things


Vhost not working properly

For some reason, every endpoint is being counted as a valid vhost that returns 403 status. Gobuster version is 3.6
To recreate-

Create a file vhost-list.txt with any random text (i.e. ajfhjakshduh39y98ih3oi) in it
Run ./gobuster vhost -u https://wazuh.com -t 1024 -w vhost-list.txt --append-domain
See results-

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:             https://wazuh.com
[+] Method:          GET
[+] Threads:         1024
[+] Wordlist:        common-vhosts.txt
[+] User Agent:      gobuster/3.6
[+] Timeout:         10s
[+] Append Domain:   true
===============================================================
Starting gobuster in VHOST enumeration mode
===============================================================
Found: ajfhjakshduh39y98ih3oi.wazuh.com Status: 403 [Size: 915]
Progress: 1 / 2 (50.00%)
===============================================================
Finished
===============================================================

However, running curl ajfhjakshduh39y98ih3oi.wazuh.com would return curl: (6) Could not resolve host: ajfhjakshduh39y98ih3oi.wazuh.com

the dns name does not need to exist in vhost mode. The correct command would be curl -H “Host: xxxxx” ip. This is not a bug


crash after upgrade from 0.92.3

Hi,
After upgrade from 0.92.3 (RealmSwift) to 0.92.4 or 0.94.0, app crashed at the same place when call
let realm = Realm()
This crash occurs only when running on the devices. (No crashes occur on the simulators iOS 8.4)
Environment:
iOS 8.1.3
iPhone 5
Xcode 6.4
RealmSwift (0.92.4 to 0.94.0)
There is no output, only (lldb)

@pengj are you sure it's not an exception being thrown, with a message? If not, having a project we can run would make investigating much easier, thanks!

I also have crash on launch with version 0.94.0
Link to the crash http://crashes.to/s/ab443d71878

@Armenm that crash looks like https://github.com/realm/realm-cocoa/issues/2332, and does not look like @pengj's crash.

Thanks @segiddins for quick reply. Will comment there. Maybe it will help to fix the bug.

@segiddins There is no exception being thrown when app crashed, the screenshot shows the moment when app crashed. I try to make a simple project to reproduce the error, but I can't reproduce it.

@segiddins When I configure the app build as release, there is no crash.

@pengj in that case, since it's actually the swift standard library crashing and not Realm, and since you can't reproduce it, I'm going to close this issue for now. Feel free to re-open it if you see this happening again!

@segiddins see my updated comments

Can you

@segiddins I sent you a email with my project. Thanks for you help.

@pengj I can't reproduce this on the iPad Mini 2 I have at hand.

@segiddins the app crash when test in iPhone5 with iOS 8.1.3, it works for iPad air with iOS 8.4.

@pengj I don't have an iOS 8.1.3 device around, but it worked fine on a 32 bit iPod touch running 8.3.

@segiddins I will close the issue for now. The crash only happens on iPhone 5 with iOS 8.1.3 as far as I can see, works well for simulator with iOS 8.1. If the build is configure as release rather than debug, it also works well. Let's wait if other developers have the same problem when update. Thanks for your help.


Update kaniko fork

Coder's kaniko fork is outdated and prevents updating some of the dependencies that have vulnerabilities.
Details can be seen here: https://github.com/coder/envbuilder/security/dependabot
This is a must-do before we launch 1.0
cc: @bpmct

https://github.com/coder/kaniko/pull/7

Should we add a scheduled workflow to keep it in sync weekly or monthly?
https://github.com/wei/pull looks promising

Kaniko has since updated to go1.22 which means we also need to do so. Unfortunately our codersdk deps prevent this right now. For now I'm going to manually vendor these in.

@johnstcn Is there anything left here or can we resolve this issue?

@mtojek https://github.com/coder/kaniko/pull/7 is still un-merged. Some of the tests are failing partially due to some changes we have that upstream does not.
Then https://github.com/coder/envbuilder/pull/195 can be updated and approved.

This is done now.


Implement Upheno absent pattern

Implement by adding absent.yaml to external.txt in the src/patterns/dosdp-patterns directory.
Implement first instance: 'alae absent' in respective absent.tsv. When you are done, remember to delete from donotedit. Feel free to add as many absent ones as you find.

Make sure you remove the two new classes in absent.tsv and missingProcess.tsv from here: https://github.com/obophenotype/c-elegans-phenotype-ontology/blob/master/src/ontology/wbphenotype-equivalent-axioms-subq.owl
If they are mentioned at all.


Unable to launch terminal

Issue Type: Bug
Click terminal menu, click new terminal
VS Code version: Code 1.60.1 (83bd43bc519d15e50c4272c6cf5c1479df196a4d, 2021-09-10T17:07:10.714Z)
OS version: Windows_NT x64 10.0.19043
Restricted Mode: No

System Info

Item
Value

CPUs
AMD Ryzen 5 5500U with Radeon Graphics          (12 x 2096)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: enabled_onvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
7.40GB (1.68GB free)

Process Argv
--crash-reporter-id 3c90e116-f972-4a2b-aa5b-c1adc7da0f4e

Screen Reader
no

VM
0%

Extensions: none

/needsMoreInfo
What is your terminal configuration? Do you see any errors?

Also getting the same issue after I updated it to 1.60. The issue is not resolved by the 1.60.1 update either. Once I open the integrated terminal, it returns a blank. Before the PowerShell opened every time, instead of bash which is my default selected terminal.
There may be an association with #132150

@IllusionMH This is a clean install, I have deleted ".vscode" folder in my home dir and "Code" in appdata roaming.
I don't get any error.

Also getting the same issue after the 1.60.1 update! The 1.60.0 update forced PowerShell but now the Terminal doesn't even get launched!

Its working after windows update, not sure what fixed it!


Rearrange section above footer of landing page

As per design suggestion from @nrizvi from issue https://github.com/coderplex/coderplex/issues/38 with a slight change in design is required
Proposed

Required

Discord section on left
Email section on right
On mobile both sections should stack above each other

If you like our project and believe in our goal, please star the project for it to reach others out there

sir, can i do this issue ?

sure please go ahead

fixed in #57


Add patroni_use_kubernetes parameter.

Add a dedicated parameter to enable support for Kubernetes in Patroni
instead of relying on a legacy hack of setting etcd_host to an empty
string. The latter still works, however, it will show a warning, nudging
users to change to the new parameter.
For the sake of not breaking existing non-Kubernetes setups,
patroni_use_kubernetes has not effect if etcd_host is set to a non-empty
value.

Coverage decreased (-0.02%) to 11.257% when pulling 76be1605b23d9d59b02b03137b866442f15234e3 on use_kubernetes_toggle into 97d1bde5b68c598b1c040b3ebc98b34e3a7c7a2d on master.


混合swift编译失败

:1:9: note: in file included from :1:
#import "Headers/EaseUI-umbrella.h"
^
/Users/mllw/Library/Developer/Xcode/DerivedData/VHSIPro-cbzywzzpqnzuigdfrtadqljjhgtr/Build/Products/Debug-iphoneos/EaseUI/EaseUI.framework/Headers/EaseUI-umbrella.h:25:9: note: in file included from /Users/mllw/Library/Developer/Xcode/DerivedData/VHSIPro-cbzywzzpqnzuigdfrtadqljjhgtr/Build/Products/Debug-iphoneos/EaseUI/EaseUI.framework/Headers/EaseUI-umbrella.h:25:
#import "wav.h"
^
/Users/mllw/Library/Developer/Xcode/DerivedData/VHSIPro-cbzywzzpqnzuigdfrtadqljjhgtr/Build/Products/Debug-iphoneos/EaseUI/EaseUI.framework/Headers/wav.h:24:1: error: unknown type name 'class'; did you mean 'Class'?
class WavWriter {
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.3.sdk/usr/include/objc/objc.h:38:28: note: 'Class' declared here
typedef struct objc_class *Class;
^
:1:9: note: in file included from :1:
#import "Headers/EaseUI-umbrella.h"
^
/Users/mllw/Library/Developer/Xcode/DerivedData/VHSIPro-cbzywzzpqnzuigdfrtadqljjhgtr/Build/Products/Debug-iphoneos/EaseUI/EaseUI.framework/Headers/EaseUI-umbrella.h:25:9: note: in file included from /Users/mllw/Library/Developer/Xcode/DerivedData/VHSIPro-cbzywzzpqnzuigdfrtadqljjhgtr/Build/Products/Debug-iphoneos/EaseUI/EaseUI.framework/Headers/EaseUI-umbrella.h:25:
#import "wav.h"
^
/Users/mllw/Library/Developer/Xcode/DerivedData/VHSIPro-cbzywzzpqnzuigdfrtadqljjhgtr/Build/Products/Debug-iphoneos/EaseUI/EaseUI.framework/Headers/wav.h:24:16: error: expected ';' after top level declarator
class WavWriter {
^
/Volumes/skd/workspace/iOS/objc/VHSIPro/VHSIPro/swift/VHAppDelegate.swift:12:8: error: could not build Objective-C module 'EaseUI'
import EaseUI
^

@mllw 此问题已经修复，pod update，重新下载EaseUI


PR #27 causes issues with RFV

Describe the bug
The RFV shown on the dashboard is -$0.00.
To Reproduce
Steps to reproduce the behavior:

Go on the Netlify preview URL for #27 or the current production app URL
Enter dashboard
See an impossibly low RFV

Expected behavior
The actual RFV should be displayed.
Workarounds (if any)
Use a preview URL for a PR before #27.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Irrelevant, it affects all users
Smartphone (please complete the following information):
Irrelevant, it affects all users
Additional context
Educated guess as I'm not on my computer, but I believe it may be caused by the new MIM-AVAX turbine being added without updating the RFV calculation logic

Thanks for the issue! fixed by #29


Fixing issues when playing / seeking remote streamed audio

This fixes the issue when calling playerNode.stop() in seekHelperDispatchQueue resulted in deadlocking... The problem was access to variables in incorrect thread of the scheduleBuffer completion
https://github.com/tanhakabir/SwiftAudioPlayer/issues/55

I can't repro the seeking issues but this might have been solved in recent fixes. The lockscreen seeking may still be broken.


check-in counts api

User Story
In order to support the desktop offering view, we need to provide a "daily check in" count.
This means we need an API that returns:

total count of desktops (nodes) that have checked in to Automate
total count of desktops (nodes) that have checked in to Automate over the last 24 hours

Considerations
Since this is useful information that can be used in views that are not desktop-only, we should make sure we design this API in a generic way.
Please refer to the epic description, q&a doc, and designs to validate requirements. Chat with @apriofrost  @chef/superteam about questions.
API endpoint should be discussed with team and agreed upon with UX and Product.
Let's try to keep most discussion/conclusions documented in this issue so we can reference later
https://chef.invisionapp.com/share/E9W3TEFXV87#/screens/406241157_Desktop_Dashboard-V2-Changes_Selected

NOTE: in the current designs, there is a view that provides count of nodes checked in per 24 hrs compared to total nodes; we may want to combine those two APIs? let's chat about it

request/
days ago: 1
response/
[ {start date: 2020-03-09[time], end date: 2020-03-10[time], checked in: 45, total: 50} ]
request/
days ago: 3
response/
[
    {start date: 2020-03-07[time], end date: 2020-03-08[time], checked in: 30, total: 40},
    {start date: 2020-03-08[time], end date: 2020-03-09[time], checked in: 40, total: 50}
    {start date: 2020-03-09[time], end date: 2020-03-10[time], checked in: 45, total: 50}
]

discussed in slack about doing a request/response like the above

Option 1:
Daily Total:
From the node-state index, for each day bucket find the number of nodes created before the end of the day. Subtract the number of nodes deleted before the end of the day.
Daily Check-in:
From the converge-history indexes, get the number of unique nodes that have reports.
If we can not find away to create aggregations to calculate these values with only two or three requests then this method would have a slow response. This would be flexible in the start and end dates of the day bucket.
Option 2:
Create a daily summary index. The day would start and end on UTC time. The index would store the total number of of nodes for each day and the total number of check-ins. This would be a faster response, but inflexalbe for the start and end of the day bucket.
I am going to look into Option 1 first to find out if we can create aggregations to collect these values.

We currently do not have a way to add project filtering on to the check-in time series because we do not have project tags on the infra reports only on the current node summary index. We could do project filtering for the last 24 hours because we can pull that information from the node summary index.
Another problem is that when a node is deleted we do not delete or mark the reports in any way. So, the time series will not remove deleted nodes from the checked-in count.

On second thought about the deleted nodes problem mentioned above, it should not be a problem that they are not removed from this historical runs indexes because after they are deleted those nodes will not be adding new runs to the indexes.

curl --silent --insecure -H "api-token: $ACCEPTANCE_TOKEN" https://a2-local-inplace-upgrade-acceptance.cd.chef.co/api/v0/cfgmgmt/stats/checkin_counts_timeseries?days_ago=5
{"counts": [
{"start":"2020-03-21T23:00:00Z","end":"2020-03-22T22:59:59Z","count":0,"total":2},
{"start":"2020-03-22T23:00:00Z","end":"2020-03-23T22:59:59Z","count":0,"total":2},
{"start":"2020-03-23T23:00:00Z","end":"2020-03-24T22:59:59Z","count":0,"total":2},
{"start":"2020-03-24T23:00:00Z","end":"2020-03-25T22:59:59Z","count":0,"total":2},
{"start":"2020-03-25T23:00:00Z","end":"2020-03-26T22:59:59Z","count":1,"total":3}
]}


chore(pyright): run the pyright lib analysis tool

Aim for 100% complete types.

Output of initial run:
starlite_saqlalchemy.cache.cache_key_builder
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/cache.py:15:5 - error: Type of parameter "request" is partially unknown
    Parameter type is "Request[Unknown, Unknown]"
      Type argument 1 for class "Request" has unknown type
      Type argument 2 for class "Request" has unknown type
starlite_saqlalchemy.cache.config
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/cache.py:26:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "CacheConfig"
starlite_saqlalchemy.compression.config
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/compression.py:6:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "CompressionConfig"
starlite_saqlalchemy.dependencies.provide_filter_dependencies
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/dependencies.py:93:5 - error: Type of parameter "id_filter" is partially unknown
    Parameter type is "CollectionFilter[Unknown]"
      Type argument 1 for class "CollectionFilter" has unknown type
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/dependencies.py:93:5 - error: Return type is partially unknown
    Return type is "list[FilterTypes]"
      Type argument 1 for class "list" has partially unknown type
starlite_saqlalchemy.exceptions.logger
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/exceptions.py:34:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "Logger"
starlite_saqlalchemy.exceptions.repository_exception_to_http_response
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/exceptions.py:72:5 - error: Type of parameter "request" is partially unknown
    Parameter type is "Request[Unknown, Unknown]"
      Type argument 1 for class "Request" has unknown type
      Type argument 2 for class "Request" has unknown type
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/exceptions.py:72:5 - error: Return type is partially unknown
    Return type is "Response[Unknown]"
      Type argument 1 for class "Response" has unknown type
starlite_saqlalchemy.exceptions.service_exception_to_http_response
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/exceptions.py:94:5 - error: Type of parameter "request" is partially unknown
    Parameter type is "Request[Unknown, Unknown]"
      Type argument 1 for class "Request" has unknown type
      Type argument 2 for class "Request" has unknown type
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/exceptions.py:94:5 - error: Return type is partially unknown
    Return type is "Response[Unknown]"
      Type argument 1 for class "Response" has unknown type
starlite_saqlalchemy.logging.AccessLogFilter.path_filter
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/logging.py:35:14 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "Pattern[str]"
starlite_saqlalchemy.logging.config
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/logging.py:47:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "LoggingConfig"
starlite_saqlalchemy.openapi.config
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/openapi.py:9:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "OpenAPIConfig"
starlite_saqlalchemy.orm.convention
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/orm.py:25:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "dict[str, str]"
starlite_saqlalchemy.orm.Base.registry
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/orm.py:55:5 - error: Ambiguous base class override
    Type declared in base class is "_RegistryType"
    Type inferred in child class is "registry"
    Inferred child class type is missing type annotation and could be inferred differently by type checkers
starlite_saqlalchemy.redis.client
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/redis.py:10:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "Redis[Any]"
starlite_saqlalchemy.repository.abc.AbstractRepository.list
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/repository/abc.py:72:15 - error: Type of parameter "filters" is partially unknown
    Parameter type is "FilterTypes"
      Type argument 1 for class "CollectionFilter" has unknown type
starlite_saqlalchemy.repository.sqlalchemy.SQLAlchemyRepository
   error: Type of base class "starlite_saqlalchemy.repository.abc.AbstractRepository" is partially unknown
starlite_saqlalchemy.repository.sqlalchemy.SQLAlchemyRepository.list
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/repository/sqlalchemy.py:100:15 - error: Type of parameter "filters" is partially unknown
    Parameter type is "FilterTypes"
      Type argument 1 for class "CollectionFilter" has unknown type
starlite_saqlalchemy.repository.types.FilterTypes
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/repository/types.py:10:1 - error: Type argument 1 for class "CollectionFilter" has unknown type
starlite_saqlalchemy.response.Response
   error: Type of base class "starlite.response.base.Response" is partially unknown
    Type argument 1 for class "Response" has unknown type
starlite_saqlalchemy.service.logger
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/service.py:27:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "Logger"
starlite_saqlalchemy.service.Service.list
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/service.py:104:15 - error: Type of parameter "filters" is partially unknown
    Parameter type is "FilterTypes"
      Type argument 1 for class "CollectionFilter" has unknown type
starlite_saqlalchemy.service.Service.repository
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/service.py:70:14 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "AbstractRepository[ModelT@Service]"
starlite_saqlalchemy.service.make_service_callback
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/service.py:255:11 - error: Type of parameter "_ctx" is partially unknown
    Parameter type is "dict[Unknown, Unknown]"
      Type argument 1 for class "dict" has unknown type
      Type argument 2 for class "dict" has unknown type
starlite_saqlalchemy.settings.api
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/settings.py:157:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "APISettings"
starlite_saqlalchemy.settings.app
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/settings.py:159:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "AppSettings"
starlite_saqlalchemy.settings.db
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/settings.py:161:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "DatabaseSettings"
starlite_saqlalchemy.settings.openapi
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/settings.py:163:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "OpenAPISettings"
starlite_saqlalchemy.settings.redis
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/settings.py:165:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "RedisSettings"
starlite_saqlalchemy.settings.sentry
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/settings.py:167:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "SentrySettings"
starlite_saqlalchemy.sqlalchemy_plugin.engine
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/sqlalchemy_plugin.py:39:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "AsyncEngine"
starlite_saqlalchemy.sqlalchemy_plugin.config
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/sqlalchemy_plugin.py:115:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "SQLAlchemyConfig"
starlite_saqlalchemy.sqlalchemy_plugin.plugin
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/sqlalchemy_plugin.py:122:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "SQLAlchemyPlugin"
starlite_saqlalchemy.static_files.here
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/static_files.py:10:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "Path"
starlite_saqlalchemy.static_files.config
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/static_files.py:13:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "StaticFilesConfig"
starlite_saqlalchemy.worker.queue
  /home/peter/PycharmProjects/starlite-saqlalchemy-test/.venv/lib/python3.11/site-packages/starlite_saqlalchemy/worker.py:56:1 - error: Type is missing type annotation and could be inferred differently by type checkers
    Inferred type is "Queue"

Symbols exported by "starlite_saqlalchemy": 320
  With known type: 281
  With ambiguous type: 24
  With unknown type: 15
    (Ignoring unknown types imported from other packages)
  Functions without docstring: 7
  Functions without default param: 0
  Classes without docstring: 6

Other symbols referenced but not exported by "starlite_saqlalchemy": 0
  With known type: 0
  With ambiguous type: 0
  With unknown type: 0

Type completeness score: 87.8%

Completed in 2.256sec


Feature/refactor get results

Minor refactor allows for direct passing of the event id url from the get_events function. Also negates the requirement to specify whether the weight categories are old or new.

Looks to be working, thank you.
My bad on the recursion.

@jwc20 - The recursion function was my bad! I thought I was being smart but gave up and went from a slightly longer version.


When train a model with Dataset, model.predict() does not work properly

Describe the current behavior
In Tensorflow's guide about feeding a model with dataset: Input tf.data datasets. The way given by the example to use dataset to train and predict cannot work properly.
Version used: 1.12.0
Describe the expected behavior
model.predict() will return the correct predicted value for the given data.
Code to reproduce the issue
model = tf.keras.Sequential([
layers.Dense(64, activation='relu'),
layers.Dense(64, activation='relu'),
layers.Dense(10, activation='softmax')])
model.compile(optimizer=tf.train.AdamOptimizer(0.001),
              loss='categorical_crossentropy',
              metrics=['accuracy'])
data = np.random.random((1000, 32)).astype(np.float32)
labels = np.random.random((1000, 10)).astype(np.float32)
dataset = tf.data.Dataset.from_tensor_slices((data, labels))
dataset = dataset.batch(32)
dataset = dataset.repeat()
model.fit(dataset, epochs=10, steps_per_epoch=30)
result = model.predict(data, batch_size=32)
print(result.shape)

Other info / logs
Stack trace:

Traceback (most recent call last):
File "xxx.py", line 255, in
result = model.predict(data, batch_size=32)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training.py", line 1864, in predict
x, check_steps=True, steps_name='steps', steps=steps)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training.py", line 992, in _standardize_user_data
class_weight, batch_size)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training.py", line 1117, in _standardize_weights
exception_prefix='input')
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training_utils.py", line 257, in standardize_input_data
'expected no data, but got:', data)
ValueError: ('Error when checking model input: expected no data, but got:', array([[5.7695776e-01, 2.4554299e-01, 6.4072663e-01, ..., 5.3916293e-01,
5.7996696e-01, 1.7735572e-05],
[9.4239563e-01, 6.3614279e-01, 5.1306045e-01, ..., 3.2898217e-01,
1.9013086e-01, 1.5709637e-01],
[9.4871348e-01, 3.7692937e-01, 1.2676715e-02, ..., 2.7022544e-01,
7.6909864e-01, 8.5889444e-02],
...,
[8.3822417e-01, 9.3324584e-01, 8.0759123e-02, ..., 1.3924435e-01,
2.8058958e-01, 3.2993883e-01],
[8.9985609e-01, 1.6618696e-01, 2.4748170e-01, ..., 5.6301647e-01,
6.0354501e-01, 1.3682739e-01],
[2.7904582e-01, 8.5806495e-01, 1.5504396e-01, ..., 3.0398217e-01,
4.7553891e-01, 4.4561669e-01]], dtype=float32))

@ChesterWQC I could reproduce the issue with tf 1.12.0, but when I tried with tf-nightly it worked fine.
Could you give tf-nightly a try and see if the issue has been fixed?

@yongtang Thanks for the quick response!
I just tried with tf-nightly build, it did behave differently and seems to be working fine.
I was wondering how soon can we expect this bug to be fixed in official release build?
Thanks.

@ChesterWQC I don't know enough about the release schedule, though I assume the changes will show up in 1.13 (not knowing the date yet).

@yongtang Thanks! Good to know that. I'm closing this issue.

Are you satisfied with the resolution of your issue? Yes No


What would be the optimal image size for testing the LoFTR model for pose estimation?

I see in the paper the authors say they trained LoFTR on Scannet with images resized to 480 x 640 pixels, and on Megadepth with images resized to have the longer dimension equal to 840 (and 1200 for validation). However, I wonder how the image sizes will affect the training/testing results? If the original image is bigger, will resizing it to a smaller dimension increase/decrease the accuracy? I personally find in my case, for pose estimation, resizing the image to have the longer side = 840 pixels decreases the testing result compared to resizing to 1200 since many of my testing images have dimensions bigger than 1000 x 1000, I don't know if this problem is investigated.

Generally, larger image resolution leads to better accuracy. Therefore, I recommend keeping your original resolution for matching regardless of the training resolutions. One exception is when the images to be matched differ a lot in scale. I would recommend aligning their scales through image resizing before matching.

I see, thank you for the prompt reply. I have another question, why did you resize the images to 840 pixels for training but 1200 for validation?

How does this data set download too big for me？


quiche: add ActiveQuicListener

Implement UdpListenerCallbacks interface for QUIC according to: https://docs.google.com/document/d/1My-Tssc9d6rW-Xu96y4DdUxvzIKmptP6mm6i_zOXr58/edit#heading=h.c0uts5ftkk58
Risk Level: low, not in use
Testing: Added unit tests for ActiveQuicListener
Part of #2557

/reopen

/assign @alyssawilk @goaway

Ping? As it's blocking #8091


Bug fix 0008

Description

added partial select for check box icon for tables
added rename modal inside group details view
changed toolbar ids to toolbar-header / toolbar-footer to be unique on page

Fixes # (issue)
https://issues.redhat.com/browse/THEEDGE-2201
Type of change
What is it?

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Documentation update
[ ] Tests update

Checklist:

[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I run npm run lint:js:fix to check that my code is properly formatted

Codecov Report

Merging #468 (b7d84c7) into master (15f3648) will decrease coverage by 0.01%.
The diff coverage is 30.00%.

:exclamation: Current head b7d84c7 differs from pull request most recent head d28e7c6. Consider uploading reports for the commit d28e7c6 to get more accurate results

@@            Coverage Diff             @@
##           master     #468      +/-   ##
==========================================
- Coverage   32.84%   32.83%   -0.02%
==========================================
  Files         124      124
  Lines        2542     2552      +10
  Branches      768      772       +4
==========================================
+ Hits          835      838       +3
- Misses       1565     1571       +6
- Partials      142      143       +1

Impacted Files
Coverage Δ

src/components/general-table/BulkSelect.js
18.18% <0.00%> (-4.05%)
:arrow_down:

src/Routes/GroupsDetail/GroupsDetail.js
40.69% <37.50%> (-0.33%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15f3648...d28e7c6. Read the comment docs.


Use GET http method for reverse replication agents

The current version will cause 'com.day.cq.replication.ReplicationException: Unsupported http method POST' messages in the reverse replication agent log.
This will set the reverse replication agent to 'GET' by default.

Looks good. Thanks for the fix!


Cached CompletedTask

excluded from #908

This is blocked on dotnet/roslyn#3218.

Merging this based on the comments on the issue linked above along with a review of the implementation. The paths taken for normal async/await operations do not throw if the Task is both completed and disposed.


@recording messages not showing

Describe the bug
After fb0e3b0 showing @recording messages is not working with both recipes from wiki, the macro itself is getting recorded.
Which version of Neovim are you using?

OS: pop!_os
Terminal: alacritty with tmux
Neovim: latest nightly

NVIM v0.9.0-dev-284+g0faf007a2
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/gcc-10 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -DNVIM_TS_HAS_SET_MATCH_LIMIT -DNVIM_TS_HAS_SET_ALLOCATOR -O2 -g -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wdouble-promotion -Wmissing-noreturn -Wmissing-format-attribute -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=always -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=3 -I/home/runner/work/neovim/neovim/build/cmake.config -I/home/runner/work/neovim/neovim/src -I/home/runner/work/neovim/neovim/.deps/usr/include -I/usr/include -I/home/runner/work/neovim/neovim/build/src/nvim/auto -I/home/runner/work/neovim/neovim/build/include
Compiled by runner@fv-az105-324

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/share/nvim"

Run :checkhealth for more info

To Reproduce
Steps to reproduce the behavior:

Press qa
Observe that lualine component is not showing and a notification is cleared too early

Expected Behavior
Statusline component should be shown and the notification should not be cleared too early.
Screenshots
Not applicable.
Noice Log
No errors are printed.
Minimal configuration

Config

vim.cmd([[set runtimepath=$VIMRUNTIME]])
vim.cmd([[set packpath=/tmp/nvim/site]])
local package_root = '/tmp/nvim/site/pack'
local install_path = package_root .. '/packer/start/packer.nvim'
local function load_plugins()
  require('packer').startup({
    {
      { 'wbthomason/packer.nvim' },
      { 'rebelot/kanagawa.nvim' },
      { 'nvim-treesitter/nvim-treesitter', run = ':TSUpdate' },
      {
        'folke/noice.nvim',
        requires = { 'MunifTanjim/nui.nvim' },
        --  commit = '0fe5a1ea053ee086327eab386594c5f2e7f15e77', -- <-- previous commit works
      },
      { 'nvim-lualine/lualine.nvim' },
    },
    config = {
      package_root = package_root,
      compile_path = install_path .. '/plugin/packer_compiled.lua',
      display = { non_interactive = true },
    },
  })
end
_G.load_config = function()
  vim.opt.termguicolors = true
  vim.cmd([[colorscheme kanagawa]])
  require('nvim-treesitter.configs').setup({})
  require('noice').setup({
    routes = {
      {
        view = 'mini',
        filter = { event = 'msg_showmode' },
      },
    },
  })
  require('lualine').setup({
    sections = {
      lualine_x = {
        {
          require('noice').api.statusline.mode.get,
          cond = require('noice').api.statusline.mode.has,
          color = { fg = '#ff9e64' },
        },
      },
    },
  })
end
if vim.fn.isdirectory(install_path) == 0 then
  print('Installing dependencies.')
  vim.fn.system({
    'git',
    'clone',
    '--depth=1',
    'https://github.com/wbthomason/packer.nvim',
    install_path,
  })
end
load_plugins()
require('packer').sync()
vim.cmd([[autocmd User PackerComplete ++once lua print('Ready'); load_config()]])

Fixed. Thank you for reporting!

I can confirm that the bug is fixed. Thank you!

I'm having actually this issue
Looking at the commit that fixed it, I have this setting: messages -> view_history -> "messages", but I even commenting all the settings, when I press "qa" doesn't shows any message about the recording started


Feature/create exists in export form

Looks good to me – what about other forms?

@Kadrian the other forms are either obsolete through the jupyter forms, or Jupyter Forms which just issues Export forms.


line, path: support size aesthetic

from discussion

-4- I am having trouble trying to plot a line (or path) with evolving width along the line. When specifying size='size values' in aes(), geom_line() and geom_path() only apply one value for the line width. I did use geom_segment() as a workaround but it is not as good as the result from ggplot2's geom_line() (and scale_linewidth()).

Variadic size/color support added in 4.0.0: https://nbviewer.org/github/JetBrains/lets-plot/blob/master/docs/f-23c/aes_size_color_variadic_lines.ipynb
cc @egayer

Awesome ! thank you !


feature(kamel): Validate --context when given

Test failure is unrelated to this change:
The job exceeded the maximum log length, and has been terminated.

working on it

Updated with extra check.


Allow creation of events - Functions

Based on the previous year's competition, I noticed that the best/recommended solution involved the use of functions for the more complex missions. As such, I think it would be important to add this if time allows.
We'd have to consider how to place these added functions.

Wonder Workshop calls functions "events"

functions are created and working


(GH-2588) Add plan name to missing_plan_parameter warning

This adds the name of the plan to the missing_plan_parameter warning,
which is logged when a plan documents a parameter that is not present in
the plan's signature. This makes it easier for users to locate which
plan documents a missing parameter.
!bug

Include plan name in missing_plan_parameter warnings
(#2588)
The missing_plan_parameter warning now includes the name of the plan
that the message was logged for.

Looks like just a small test change - happy to get this in for 3.0!


netrw.vim seems to reuse s:netrw_usercul, s:netrw_usercuc without reinitializing

Instructions: Replace the template text and remove irrelevant text (including this line)
Describe the bug
When executing the ":Explore" multiple times, the values of  "cursorline, cursorcolumn" are ignored from the second time onwards.
To Reproduce

C:> vim t.txt
:set cursorline
:set cursorcolumn
:Explore
// Select "t.txt"
:echo &cursorline    -->    1
:echo &cursorcolumn    -->    1
:set nocursorline
:set nocursorcolumn
:Explore
// Select "t.txt" again
:echo &cursorline    -->    1 ( Bug : "set nocursorline" is ignored )
:echo &cursorcolumn    -->    1 ( Bug : "set nocursorcolumn" is ignored )

Expected behavior
10. :echo &cursorline    -->    0
11. :echo &cursorcolumn    -->    0
Screenshots

Environment (please complete the following information):

Vim 8.2.2847
OS: [Windows 10 64bit, 20H2, 19042.985]

Additional context
// When executing ":Explore" at first these two lines are initialized
[netrw.vim] -- 332 --

// and when executing ":Explore" from the second time  ...
// it seems that "s:netrw_usercul, s:netrw_usercuc" are resued without reinitializing.
[netrw.vim] -- 11759 --

// When disabling these two lines, there is no problem anymore.
[netrw.vim] -- 11759 --

@cecamp


[Unmaintained] aegisubdc

Problem
aegisubdc is unmaintained
Solution
replace with https://github.com/wangqr/Aegisub
Scoop Configuration

//# Your configuration here

https://github.com/arch1t3cht/Aegisub should also be added as it's currently the best fork.

AegisubDC is based on wangqr's Aegisub with additional features. It is better to add wangqr one as another manifest instead of replacing AegisubDC.


Fixed area percentage chart to use 1 y axis

Coverage remained the same at 97.426% when pulling 7057d044727d06864ad824024fa9f59c08b1d2f6 on mikeengland:master into a65cd5047a702ae9e1459d9dbac4a3d5504646a9 on kayak:master.

Coverage remained the same at 97.426% when pulling 7057d044727d06864ad824024fa9f59c08b1d2f6 on mikeengland:master into a65cd5047a702ae9e1459d9dbac4a3d5504646a9 on kayak:master.

Coverage remained the same at 97.426% when pulling 7057d044727d06864ad824024fa9f59c08b1d2f6 on mikeengland:master into a65cd5047a702ae9e1459d9dbac4a3d5504646a9 on kayak:master.


cluster manager: make add/update and initial host set update atomic

Previously, we would do a separate TLS set to add/update the cluster,
and then another TLS operation to populate the initial host set. For
cluster updates for a server already in operation this can lead to a
gap of no hosts.
Fixes https://github.com/envoyproxy/envoy/issues/13070
Risk Level: High
Testing: I didn't add any new tests, because it's unclear to me what tests to add.
I could have added a unit test to cover the race condition, but it would be broken
by the change and then deleted. Note that getting all of the existing tests to pass
was quite difficult, so I do feel there is good coverage here, but if anyone has good
ideas on how to better test this please let me know.
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

@snowp @lambdai could you take a first pass on this? This was much harder to fix than I thought. I tried to do the minimal change possible and left some TODOs for follow ups.
cc @rgs1 it would be great if you would be willing to smoke test this once we get past the initial round of reviews.

My concern is that cluster update is not always along with member change and Envoy should propagate this cluster update ASAP instead of waiting for the next member change.

Essentially, this is the fix. Now, we could track whether the cluster is new or updated, and only do this in the updating case, but I'm not sure the extra logic is worth it. In the bootstrap case, we do what you describe above, but as I mentioned in the TODOs I would actually like to remove it from the few paths that require it (e.g. route validation).

Is there any way we could try racing requests and cluster updates and check for intermittent failures?

You mean like in an integration test? This seems kind of fragile from a test perspective.

You mean like in an integration test? This seems kind of fragile from a test perspective.

Yeah that was my thinking, just to have something that could hit this in case of a regression. But yeah introducing a test that would likely just end up being flaky if something broke might be too painful to debug etc if something actually breaks, so it might not be a great idea.

@lambdai @snowp updated per comments.
@rgs1 I think this should be ready for a smoke test. Thank you!

I'm going to work on the follow up clean up to this PR. @rgs1 depending on when you have time to smoke test we might want to just get them both in together.

I'm going to work on the follow up clean up to this PR. @rgs1 depending on when you have time to smoke test we might want to just get them both in together.

Won't happen before Thu, I am afraid. I've been looking at the new TCP conn pool with Alyssa. So if you can have both by then, I'll test them together.

I'm going to work on the follow up clean up to this PR. @rgs1 depending on when you have time to smoke test we might want to just get them both in together.

Won't happen before Thu, I am afraid. I've been looking at the new TCP conn pool with Alyssa. So if you can have both by then, I'll test them together.

Ok, giving this a try now.


Error: fetch failed. retrying in 120 seconds

Status       : Error: fetch failed. retrying in 120 seconds
Waiting      : 0h 1m 12s
Game Status  :

plinko     : pending (-)
singlewheel: pending (-)
mines      : pending (-)

update the code and re-run


Policy editing

Can you add a section on Policy editing.
I'm finding that you cannot edit parameter section once the policy has been created in Azure portal.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8f728627-f8e4-0e41-c747-658d9496e9aa
Version Independent ID: 443bc940-a2a2-7c99-6d96-d7c6dd904173
Content: Use Azure Policy to create and manage policies to enforce organizational compliance
Content Source: articles/governance/policy/tutorials/create-and-manage.md
Service: azure-policy
GitHub Login: @DCtheGeek
Microsoft Alias: dacoulte

@JozoSlejko Thanks for bringing this to our attention. Your feedback has been shared with the content owner for further review.

@JozoSlejko That is correct. Once the parameters are defined on a policy or initiative, the parameters definition can't be changed (although you can provider different parameter values during assignment). This prevents existing assignments of the policy or initiative from being indirectly invalidated.
I'll add a note on the policy definition for parameters to make this more clear.  Thanks for bringing it up!
@MicrosoftDocs/azure-cxp-triage #in-progress


Update DG

Update UG to make examples more consistent (e.g on some lines EventNewCommand is stated as the example but in the next VolunteerCommandParser is used as an example to explain the flow of object calls).
Updated use cases to be consistent. Previously only some use cases had the system check if the input given by the user was valid.
Add extensions to use cases. Previously remove date and assign did not check for clashing events/availability.

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.


500 Internal Server Error with netlify deploy

- Do you want to request a feature or report a bug?
Bug
- What is the current behavior?
500 Internal Server Error when trying to run netlify upload
- If the current behavior is a bug, please provide the steps to reproduce.

Create a zip file locally using zip-it-and-ship-it then run netlify deploy on that folder that has the zip file.
When I run DEBUG=* netlify deploy I get the following

DEBUG=* netlify deploy
  @oclif/config reading core plugin netlify-cli +0ms
  @oclif/config loadJSON netlify-cli/package.json +0ms
  @oclif/config loadJSON netlify-cli/oclif.manifest.json +3ms
  @oclif/config:netlify-cli using manifest from netlify-cli/oclif.manifest.json +0ms
  @oclif/config reading user plugins pjson netlify-cli/package.json +0ms
  @oclif/config loadJSON netlify-cli/package.json +2ms
  @oclif/config loading plugins [
  '@oclif/plugin-not-found',
  '@oclif/plugin-plugins',
  '@oclif/plugin-help'
] +7ms
  @oclif/config loadJSON netlify-cli/package.json +9ms
  @oclif/config loadJSON netlify-cli/package.json +0ms
  @oclif/config loadJSON netlify-cli/package.json +0ms
  @oclif/config loadJSON package.json +12ms
  @oclif/config loadJSON package.json +1ms
  @oclif/config loadJSON package.json +0ms
  @oclif/config reading core plugin @oclif/plugin-help +0ms
  @oclif/config loadJSON @oclif/plugin-help/package.json +7ms
  @oclif/config reading core plugin @oclif/plugin-plugins +0ms
  @oclif/config loadJSON @oclif/plugin-plugins/package.json +0ms
  @oclif/config reading core plugin @oclif/plugin-not-found +0ms
  @oclif/config loadJSON @oclif/plugin-not-found/package.json +0ms
  @oclif/config loadJSON @oclif/plugin-plugins/oclif.manifest.json +5ms
  @oclif/config loadJSON @oclif/plugin-help/oclif.manifest.json +1ms
  @oclif/config loadJSON @oclif/plugin-not-found/oclif.manifest.json +5ms
  @oclif/config:@oclif/plugin-plugins using manifest from @oclif/plugin-plugins/oclif.manifest.json +0ms
  @oclif/config:@oclif/plugin-help using manifest from @oclif/plugin-help/oclif.manifest.json +0ms
  @oclif/config:@oclif/plugin-not-found using manifest from @oclif/plugin-not-found/oclif.manifest.json +0ms
  @oclif/config config done +38ms
  @oclif/config start init hook +0ms
  netlify:netlify-cli:hooks:init start netlify-cli/src/hooks/init +0ms
  netlify:netlify-cli:hooks:init done +327ms
  @oclif/config init hook done +898ms
  netlify init version<EMAIL_ADDRESS>argv: [ 'deploy' ] +0ms
  @oclif/config runCommand deploy [] +6ms
  @oclif/config:netlify-cli require netlify-cli/src/commands/deploy.js +951ms
  @oclif/config start prerun hook +4s
  @oclif/config prerun hook done +3ms
  netlify options: {
  netlify   userAgent: 'netlify/js-client',
  netlify   scheme: 'https',
  netlify   host: 'api.netlify.com',
  netlify   pathPrefix: '/api/v1'
  netlify } +0ms
  netlify Setting access token +1ms
  @oclif/config start analytics hook +202ms
  netlify:netlify-cli:hooks:analytics start netlify-cli/src/hooks/analytics +0ms
  netlify:netlify-cli:hooks:analytics done +2ms
  @oclif/config analytics hook done +2ms
  netlify:open-api path template: https://api.netlify.com/api/v1/sites/{site_id} +0ms
  netlify:open-api replacing {site_id} with 6d3e3c39-8988-4331-9b92-b52d5857a8be +1ms
  netlify:open-api bodyType: json +0ms
  netlify:open-api specialHeaders: {} +0ms
  netlify:open-api method: GET +23ms
  netlify:open-api Rate limit attempt 0 for https://api.netlify.com/api/v1/sites/6d3e3c39-8988-4331-9b92-b52d5857a8be +0ms
  netlify:open-api requesting https://api.netlify.com/api/v1/sites/6d3e3c39-8988-4331-9b92-b52d5857a8be +0ms
  netlify:open-api fetch done +389ms
  netlify:open-api response contentType: application/json; charset=utf-8 +0ms
  netlify:open-api parsing json +0ms
Deploy path:        ./.netlify/build
Functions path:     ./dist-1-zip
Configuration path: ./netlify.toml
Deploying to draft URL...
- Hashing files...
✔ Finished hashing 1 files and 3 functions
- CDN diffing files...
  netlify:open-api path template: https://api.netlify.com/api/v1/sites/{site_id}/deploys +4s
  netlify:open-api replacing {site_id} with 6d3e3c39-8988-4331-9b92-b52d5857a8be +0ms
  netlify:open-api bodyType: json +1ms
  netlify:open-api specialHeaders: { 'Content-Type': 'application/json' } +0ms
  netlify:open-api method: POST +2ms
  netlify:open-api Rate limit attempt 0 for https://api.netlify.com/api/v1/sites/6d3e3c39-8988-4331-9b92-b52d5857a8be/deploys +0ms
  netlify:open-api requesting https://api.netlify.com/api/v1/sites/6d3e3c39-8988-4331-9b92-b52d5857a8be/deploys +1ms
  netlify:open-api fetch done +320ms
  netlify:open-api response contentType: application/json; charset=utf-8 +0ms
  netlify:open-api parsing json +0ms
✔ CDN requesting 0 files and 2 functions
- Uploading 2 files
  netlify:open-api path template: https://api.netlify.com/api/v1/deploys/{deploy_id}/functions/{name} +45ms
  netlify:open-api replacing {deploy_id} with 5e163685bac481b9081e4070 +1ms
  netlify:open-api replacing {name} with events +0ms
  netlify:open-api qs: { runtime: 'js' } +2ms
  netlify:open-api bodyType: binary +5ms
  netlify:open-api specialHeaders: { 'Content-Type': 'application/octet-stream' } +0ms
  netlify:open-api method: PUT +0ms
  netlify:open-api Rate limit attempt 0 for https://api.netlify.com/api/v1/deploys/5e163685bac481b9081e4070/functions/events?runtime=js +0ms
  netlify:open-api requesting https://api.netlify.com/api/v1/deploys/5e163685bac481b9081e4070/functions/events?runtime=js +0ms
  netlify:open-api path template: https://api.netlify.com/api/v1/deploys/{deploy_id}/functions/{name} +3ms
  netlify:open-api replacing {deploy_id} with 5e163685bac481b9081e4070 +0ms
  netlify:open-api replacing {name} with health +0ms
  netlify:open-api qs: { runtime: 'js' } +0ms
  netlify:open-api bodyType: binary +0ms
  netlify:open-api specialHeaders: { 'Content-Type': 'application/octet-stream' } +2ms
  netlify:open-api method: PUT +0ms
  netlify:open-api Rate limit attempt 0 for https://api.netlify.com/api/v1/deploys/5e163685bac481b9081e4070/functions/health?runtime=js +0ms
  netlify:open-api requesting https://api.netlify.com/api/v1/deploys/5e163685bac481b9081e4070/functions/health?runtime=js +0ms
  netlify:open-api fetch done +29s
  netlify:open-api response contentType: application/json; charset=utf-8 +1ms
  netlify:open-api parsing json +0ms
  netlify:deploy Upload failed... retrying +0ms
  netlify:deploy { status: '500', error: 'Internal Server Error' } +0ms
  netlify:open-api fetch done +110ms
  netlify:open-api response contentType: application/json; charset=utf-8 +0ms
  netlify:open-api parsing json +1ms
  netlify:deploy Upload failed... retrying +110ms
  netlify:deploy { status: '500', error: 'Internal Server Error' } +0ms

- What is the expected behavior?
It deploys correctly
- Local Environment Information
 netlify status --verbose
──────────────────────┐
 Current Netlify User │
──────────────────────┘
Name:  Chris
Email:
Teams:
  Chris's team: Collaborator

────────────────────┐
 Environment Info   │
────────────────────┘

  System:
    OS: macOS 10.14.6
    CPU: (4) x64 Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz
  Binaries:
    Node: 13.5.0 - /usr/local/bin/node
    Yarn: 1.17.3 - /usr/local/bin/yarn
    npm: 6.9.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 79.0.3945.117
    Firefox: 70.0
    Safari: 13.0.4

────────────────────┐
 Netlify Site Info  │
────────────────────┘
Current site: staging-bot-slack
Netlify TOML: ./netlify.toml
Admin URL:    https://app.netlify.com/sites/staging-bot-slack
Site URL:     https://staging-bot-slack.netlify.com
Site Id:      6d3e3c39-8988-4331-9b92-b52d5857a8be

Netlify version 2.27.0

@seawatts where do the built assets of your site live?
It looks like deploy is trying to deploy the Deploy path:        ./.netlify/build folder.
What does your netlify.toml config file look like?

Sorry I redacted the full path.
# netlify.toml
[build]
  command = "yarn build"
  functions = "dist"

Looks like you need a publish dir in that config file. This should* fix the problem.
# netlify.toml
[build]
  publish = "dist"
  command = "yarn build"
  functions = "dist"

Or pass that along as a CLI arg. https://cli.netlify.com/commands/deploy
netlify deploy --dir dist

Looks like that worked. Thanks!
Would be nice to have some better error handling around this. Currently if you don't do DEBUG=* then it just hangs there without any feedback.


mypy: fix "key" attribute

It cannot be None and with this patch definition of key in
subclasses/plugins is enforced by python itself
Signed-off-by: Martin Basti<EMAIL_ADDRESS>Maintainers will complete the following section

[x] Commit messages are descriptive enough
[x] Code coverage from testing does not decrease and new code is covered
[n/a] JSON/YAML configuration changes are updated in the relevant schema
[n/a] Changes to metadata also update the documentation for the metadata
[n/a] Pull request has a link to an osbs-docs PR for user documentation updates
[n/a] New feature can be disabled from a configuration file

no release notes, just update in annotations and internal code change


FIX: VTK Version

Closes #10398
Feel free to try it @hoechenberger

Will have a look

I tested this locally on macOS, reproduced the bug on main, and it's fixed by this. So I'll go ahead and merge for 1.0


Update README.md

remove enable purchases on iOS 11

Cydia store purchase is a thing that should be enabled. Isn’t it? As users don’t need to rely on the Dev to process purchases

They have to be on third party repos because the actual Cydia store has
been disabled
On Thu, Feb 14, 2019 at 9:35 AM Tashfeen<EMAIL_ADDRESS>wrote:

Cydia store purchase is a thing that should be enabled. Isn’t it? As users
don’t need to rely on the Dev to process purchases
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pwn20wndstuff/Undecimus/pull/363#issuecomment-463672177,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AXzofCtcy8MJcGQjOQgEouGDE74AK6_kks5vNYIygaJpZM4a7pyv
.

Also you can suggest and discuss the changes on Pull Request #340 . Which is currently under development by the participants

And that’s the reason it’s on the pending list

Please combine this with #340


backfill the status and route columns of the exchanges table

Right now the exchanges.status column has NULL values. It would be great to constrain that column to non-NULL, but it would take some effort with the Balanced API to determine the proper value for previous exchanges. IRC

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/4689324-backfill-the-status-and-route-columns-of-the-exchanges-table?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github).

Added to the Balanced shutdown milestone.

We have a user getting bitten by #3302, which we decided to fix with this ticket.
https://gratipay.freshdesk.com/helpdesk/tickets/1867

I'm working on manually fixing this for the 1867 case, and am blocking payday (#3307) on it.

IRC

I believe the destination parameter here identifies the route for the most recently failing deposit:
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EVfe85f7b4d70211e4b84a0663eb3e293e
However, the same parameter on the previous nine deposits (some successful, some not) is not present in exchange_routes.address:
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV3c627cc8d15011e496a0065bec47279b
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV9610ff60c92111e4b8dc02da29bdd175
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV864e9340c69111e4a8dc02da29bdd175
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV58831afac2bb11e49819065bec47279b
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV2f43cbb2b89811e4a512065bec47279b
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV501740b0b31811e4adbd065bec47279b
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV4decb9d4ae3911e4962c02da29bdd175
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV54ec1920a82e11e4ad9702da29bdd175
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EV54966e04a29711e4972f065bec47279b

We are expecting one of the old deposits to fail, at which time Balanced will post back so we can update exchanges.status. We'll need to insert the old exchange_route before we can connect the exchange to it. For now I'm only worrying about the most recent failure.

I set exchanges.route for the exchange in question and clicked "Replay" on the Balanced dashboard:
https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/events/EVfe85f7b4d70211e4b84a0663eb3e293e
It's still in the "retrying" state, and I'm not seeing the status updated on the user's History page.

Here's the error in Sentry (cf. #3302):
https://app.getsentry.com/gratipay/gratipay-com/group/60437761/

I'm not getting good visibility into the status of the webhook postback. From Sentry it looks like we haven't generated any errors since I hit "Replay," but Sentry samples (right? how?), so not every request is recorded. If the postback works then there also won't be a Sentry entry. The absence of a Sentry entry either means it worked or it didn't but Sentry isn't showing us the specific failure instance.
All we get in the Balanced dashboard is this:

Using the Chrome inspector I've found the json endpoint for callbacks, which includes this:
      "attempts_remaining": 9,
      "attempts": 1,
      "attempt_limit": 10,

the callback will be retried up to 10 times with an initial retry delay of 10 minutes which will increase exponentially with each failure for the next 7 days.

https://docs.balancedpayments.com/1.1/api/callbacks/

Weeeeeeeelllllll, I clicked Replay more than 10 minutes ago, and it's still reporting as retrying with 9 attempts left.

So I could update the user's balance manually, but I don't see that I can cancel the callback. :-(

If I run record_exchange_result manually and then start payday, and the callback succeeds while payday is running, then we will send the user twice as much money as we should. That'd be a mess to clean up.

But at this point we're down to the wire on payday.

then we will send the user twice as much money as we should

No, we won't.

Of course you also have to update the status of the exchange, not just the balance.

from gratipay.main import Participant
from gratipay.billing.exchanges import record_an_exchange

sudoroom = Participant.from_username('sudoroom')
record_an_exchange( Participant.db
                  , 67813
                  , 'failed'
                  , 'Funds unavailable due to action by RDFI or other legal action'
                  , sudoroom
                   )

IRC

[gratipay] $ heroku config -s | honcho run -e /dev/stdin ./env/bin/python unblock.py
Traceback (most recent call last):
  File "unblock.py", line 12, in <module>
    , foo
  File "/Users/whit537/personal/gratipay/gratipay.com/gratipay/billing/exchanges.py", line 318, in record_exchange_result
    assert route
AssertionError
[gratipay] $

I clicked "Replay" and still get the "retring" message.

Succeeded! :-)

Okay!
So as part of this ticket we'll need to look for any further failed webhooks in Balanced and replay them.

 I'm tempted to just mark all of them as 'succeeded', and focus on just the routes instead.

If we're making the API calls anyway it might make sense to do #2443 together with this one.

See https://github.com/gratipay/inside.gratipay.com/issues/323#issuecomment-135588541 for another instance of the above one-off fix.

Over at https://github.com/gratipay/inside.gratipay.com/issues/308#issuecomment-143008427 I made a download of the JSON for all of our Balanced transactions. Might be a discrepancy but we're close. Use the scripts over there as a basis for backfilling routes here .

Do all of these necessarily have to be present in Balanced?

No.

Could they also be old exchanges from Stripe or so?

Yes.

We're at 77,360 exchanges, still with 37,557 with NULL status.
Here's what a record looks like in our database:
=# select * from exchanges where id=[n];
┌─────┬───────────────────────────────┬────────┬──────┬─────────────┬──────────┬──────┬────────┬───────┐
│ id  │           timestamp           │ amount │ fee  │ participant │ recorder │ note │ status │ route │
├─────┼───────────────────────────────┼────────┼──────┼─────────────┼──────────┼──────┼────────┼───────┤
│ [n] │ 2012-07-27 20:51:27.828555+00 │   9.32 │ 0.68 │ [username]  │ ¤        │ ¤    │ ¤      │     ¤ │
└─────┴───────────────────────────────┴────────┴──────┴─────────────┴──────────┴──────┴────────┴───────┘
(1 row)

And here's what we want to match it up with from transactions/*.json:
(Pdb) pp thing
{u'amount': 1000,
 u'appears_on_statement_as': u'BAL*www.gittip.com',
 u'created_at': u'2012-07-27T20:51:27.505700Z',
 u'currency': u'USD',
 u'description': u'[username]',
 u'failure_reason': None,
 u'failure_reason_code': None,
 u'href': u'/debits/[]',
 u'id': u'[]',
 u'links': {u'card_hold': u'[]',
            u'customer': u'[]',
            u'dispute': None,
            u'order': None,
            u'source': u'[]'},
 u'meta': {},
 u'status': u'succeeded',
 u'transaction_number': u'[]',
 u'updated_at': u'2012-07-27T20:51:27.532539Z'}
(Pdb)

Start on this, got a hot mess of a script atm:
#!/usr/bin/env python -u
from __future__ import absolute_import, division, print_function, unicode_literals

import os, json
from gratipay import wireup
from decimal import Decimal as D

db = wireup.db(wireup.env())
dirname = os.path.join(os.path.dirname(__file__), 'transactions')
i = 0
I = 500000
class Done(Exception): pass

with db.get_cursor() as cur:
    try:
        for filename in reversed(sorted(os.listdir(dirname))):
            data = json.load(open(os.path.join(dirname, filename)))

            for key in data:
                if key in ('links', 'meta'): continue
                things = reversed(data[key])

                if key == 'card_holds':
                    continue  # We don't track these.

                if key != 'debits':
                    continue  # Let's start with credit card charges.

                for thing in things:
                    route = cur.one( "SELECT * FROM exchange_routes WHERE address='/cards/'||%s"
                                   , (thing['links']['source'],)
                                    )
                    if route is None:
                        import pdb; pdb.set_trace()
                        continue
                    username = cur.one( "SELECT username FROM participants WHERE id=%s"
                                      , (route.participant,)
                                       )

                    params = dict( status   = thing['status']
                                 , amount   = D(thing['amount']) / 100
                                 , ts       = thing['created_at']
                                 , route_id = route.id
                                 , username = username
                                  )
                    mogrified = cur.mogrify("""\
                    WITH rows AS (
                        UPDATE exchanges
                           SET status = %(status)s
                             , route = %(route_id)s
                         WHERE participant = %(username)s
                           AND amount + fee = %(amount)s
                           AND ((   (%(ts)s::timestamptz - "timestamp") < interval '0'
                                AND (%(ts)s::timestamptz - "timestamp") > interval '-1s'
                                ) OR (
                                    (%(ts)s::timestamptz - "timestamp") > interval '0'
                                AND (%(ts)s::timestamptz - "timestamp") < interval '1s'
                                ))
                     RETURNING participant
                    ) SELECT * FROM rows;
                    """, params)
                    res = cur.one(mogrified)
                    print(params['ts'], res, username)

                    if i > I:
                        raise Done()
                    i += 1
    except Done:
        pass
    print()

grep
wc -l

": 1\.$"
8,145

"Need to insert route for"
11,079

"Need to link"
8

"ExchangeMismatch: "
28,078

("ExchangeMismatch: .*got None"
28,078)

"TooMany"
526

Sum
47,836

`"Z
"`

$ grep "raise" backfill.log | sort | uniq
    raise ExchangeMismatch(nupdated)
    raise TooMany(self.rowcount, lo, hi)
$

#!/usr/bin/env python -u
from __future__ import absolute_import, division, print_function, unicode_literals

import json, os, sys, traceback
from gratipay import wireup
from decimal import Decimal as D
from pprint import pprint

db = wireup.db(wireup.env())
dirname = os.path.join(os.path.dirname(__file__), 'transactions')
i = 0
I = float('inf')
class Done(Exception): pass

class ExchangeMismatch(Exception):
    def __str__(self):
        return "Expected 1 update, got {}.".format(self.args[0])

def process_thing(cur, thing):
    customer = cur.one( "SELECT participants.*::participants "
                        "FROM participants "
                        "WHERE balanced_customer_href='/customers/'||%s"
                      , (thing['links']['customer'],)
                       )
    route = cur.one( "SELECT * FROM exchange_routes WHERE address='/cards/'||%s"
                   , (thing['links']['source'],)
                    )

    if customer is None and route is None:
        print("Need to link {} to ... {}(?) and create a route for {}."
              .format( thing['links']['customer']
                     , thing['description']
                     , thing['links']['source']
                      ))
        return
    elif participant is None:
        print("Need to link {} to ... {}?"
              .format(thing['links']['customer'], thing['description']))
        return
    elif route is None:
        print("Need to insert route for {} and {}."
              .format(participant.username, thing['links']['source']))
        return

    # Use the username that is current as of this transaction.
    username = cur.one( "SELECT username FROM participants WHERE id=%s"
                      , (route.participant,)
                       )

    params = dict( status   = thing['status']
                 , amount   = D(thing['amount']) / 100
                 , ts       = thing['created_at']
                 , route_id = route.id
                 , username = username
                  )
    mogrified = cur.mogrify("""\
    WITH rows AS (
        UPDATE exchanges
           SET status = %(status)s
             , route = %(route_id)s
         WHERE participant = %(username)s
           AND amount + fee = %(amount)s
           AND ((   (%(ts)s::timestamptz - "timestamp") < interval '0'
                AND (%(ts)s::timestamptz - "timestamp") > interval '-1s'
                ) OR (
                    (%(ts)s::timestamptz - "timestamp") > interval '0'
                AND (%(ts)s::timestamptz - "timestamp") < interval '1s'
                ))
     RETURNING 1 AS count
    ) SELECT * FROM rows;
    """, params)
    print("Updating exchange(s) for {}: ".format(username), end='')
    nupdated = cur.one(mogrified)
    if nupdated != 1:
        raise ExchangeMismatch(nupdated)
    print("{}.".format(nupdated))

with db.get_cursor() as cur:
    try:
        for filename in reversed(sorted(os.listdir(dirname))):
            data = json.load(open(os.path.join(dirname, filename)))

            for key in data:
                if key in ('links', 'meta'): continue
                things = reversed(data[key])

                if key == 'card_holds':
                    continue  # We don't track these.

                if key != 'debits':
                    continue  # Let's start with credit card charges.

                for thing in things:
                    print("{} | ".format(thing['created_at']), end='')

                    try:
                        process_thing(cur, thing)
                    except KeyboardInterrupt:
                        raise
                    except:
                        print("Exception! Thing:")
                        pprint(thing)
                        print('-'*78)
                        traceback.print_exc(file=sys.stdout)
                        print('='*78)

                    if i > I: raise Done()
                    i += 1
    except Done:
        pass
    print()

Alright, we've got duplicate exchange routes ((network, address) is not unique), and we've also got duplicates in participants.balanced_customer_href.

Whoops :)

I'm working on https://github.com/gratipay/gratipay.com/pull/3356.

Before we start filling the route values for previous exchanges, we should make sure that no new exchanges are created with a NULL route. As of now, record-an-exchange.spt is the only place where this happens (MassPay uses it, so that too). That's what this PR aims to fix.

!m @rohitpaulk :dancer:

Some of both kinds of duplicates are due to merge/takeover. Not sure if all, but that seems likely.

Spot-checking the None matches suggests that a 1-second time interval is too tight to match. We should loosen that to 2 seconds and see if we overmatch any (we also constrain by username and amount, so I wouldn't expect so).

This one is off by almost thirty seconds.

We have a bunch with non-null status, but null route:
=> select count(*) from exchanges where route is null and status is not null;
┌───────┐
│ count │
├───────┤
│ 31620 │
└───────┘
(1 row)

I upped the threshold to one minute, and I'm not seeing any NoExchanges except for a handful of special cases where we charged people manually, out-of-band from payday. It looks like if we up the threshold 15 minutes we should be able to pick these off. Again, that shouldn't introduce duplicates because we only charge once per week.

Alright, so I guess we need to resolve duplicate routes and customers before proceeding here. Reticketed as #3806.

#!/bin/sh
patterns=(": 1\.$"
          "No route for"
          "NoExchange"
          "TooMany")
for pattern in "${patterns[@]}"; do
    printf "%-14s %5s\n" "\"$pattern\"" `grep "$pattern" backfill.log | wc -l`
done

$ balanced/tally-backfill.sh
": 1\.$"       33918
"No route for" 11186
"NoExchange"    2308
"TooMany"        322
$

We want 47,573 of the first, and zero of the rest.
I believe most of the NoExchange are in fact already linked exchanges, from after when we started recording status and route.

[ ] deduplicate routes and customers—#3806
[ ] backfill participants.balanced_customer_href—we're missing a few, I think?
[ ] create missing routes—definitely missing a bunch of these
[ ] modify the exchanges schema to store a URL and a JSON payload

Haven't even touched refunds, or bank deposits or reversals yet. We don't actually need those in order to be able to issue refunds against Balanced this week, which is our primary objective atm.

Though conceptually it makes sense to get this done for all transaction types while we're here.

Still seeing one NoExchange: a special case where the manual exchange was recorded wrong, with a second exchange to correct. The amount of the first exchange is out relative to Balanced.

Okay, cool. We're down to one NoExchange, with two others picked off by expanding the time window to ±15m, and the rest falling under Already linked.
": 1\.$"            33920
"Already linked"     2305
"NoExchange"            1
"AmbiguousExchange"     0
"No route for"      11186
"TooMany"             322

I think we can backfill missing customers and routes as part of this script, once both are deduplicated.

Got the numbers to line up:
": 1\.$"              33920
"Already linked"       2305
"NoExchange: "            1
"AmbiguousExchange: "     0
"No route for"        11186
"raise TooMany"         161
---------------------------
                      47573

#!/bin/sh
patterns=(": 1\.$"
          "Already linked"
          "NoExchange: "
          "AmbiguousExchange: "
          "No route for"
          "raise TooMany")
N=0
for pattern in "${patterns[@]}"; do
    n=`grep "$pattern" backfill.log | wc -l`
    N=`expr $n + $N`
    printf "%-21s %5s\n" "\"$pattern\"" $n
done
echo "---------------------------"
printf "%-21s %5s\n" "" $N

"raise TooMany" should go to zero with #3806, and we'll handle "No route for" by creating routes on the fly while looping.

#!/usr/bin/env python -u
from __future__ import absolute_import, division, print_function, unicode_literals

import json, os, sys, traceback
from gratipay import wireup
from decimal import Decimal as D
from pprint import pprint

db = wireup.db(wireup.env())
dirname = os.path.join(os.path.dirname(__file__), 'transactions')
i = 0
I = float('inf')
class Done(Exception): pass

class ExchangeMismatch(Exception):
    def __str__(self):
        return "{} {} {}\n\n{}".format(*self.args)
class NoExchange(ExchangeMismatch): pass
class AmbiguousExchange(ExchangeMismatch): pass

def process_thing(cur, thing):
    print('{} -> '.format(thing['description']), end='')

    participants = cur.all( "SELECT username FROM participants "
                            "WHERE balanced_customer_href='/customers/'||%s"
                          , (thing['links']['customer'],)
                           )
    print('{} | '.format(', '.join(participants)), end='')

    route = cur.one( "SELECT * FROM exchange_routes WHERE address='/cards/'||%s"
                   , (thing['links']['source'],)
                    )
    if route is None:
        if not participants:
            print("No route for {}.".format(thing['links']['source']))
        else:
            print("No route for {}.".format(thing['links']['source']))
        return

    # Use the username that is current as of this db transaction.
    username = cur.one( "SELECT username FROM participants WHERE id=%s"
                      , (route.participant,)
                       )

    params = dict( status   = thing['status']
                 , amount   = D(thing['amount']) / 100
                 , ts       = thing['created_at']
                 , route_id = route.id
                 , username = username
                  )

    nalready = cur.one("""\
        SELECT count(*) FROM exchanges
         WHERE status = %(status)s
           AND route = %(route)s
           AND participant = %(username)s
           AND amount + fee = %(amount)s
           AND ((   (%(ts)s::timestamptz - "timestamp") < interval '0'
                AND (%(ts)s::timestamptz - "timestamp") > interval '-15m'
                ) OR (
                    (%(ts)s::timestamptz - "timestamp") > interval '0'
                AND (%(ts)s::timestamptz - "timestamp") < interval '15m'
                ))
    """, dict(params, route=route.id))
    if nalready == 1:
        print("Already linked!")
        return

    mogrified = cur.mogrify("""\
    WITH rows AS (
        UPDATE exchanges
           SET status = %(status)s
             , route = %(route_id)s
         WHERE (status is NULL or status = %(status)s)
           AND route is NULL
           AND participant = %(username)s
           AND amount + fee = %(amount)s
           AND ((   (%(ts)s::timestamptz - "timestamp") < interval '0'
                AND (%(ts)s::timestamptz - "timestamp") > interval '-15m'
                ) OR (
                    (%(ts)s::timestamptz - "timestamp") > interval '0'
                AND (%(ts)s::timestamptz - "timestamp") < interval '15m'
                ))
     RETURNING 1 AS count
    ) SELECT * FROM rows;
    """, params)
    print("Updating exchanges for {}: ".format(username), end='')
    nupdated = cur.one(mogrified, default=0)
    Error = None
    if nupdated == 0:
        Error = NoExchange
    elif nupdated > 1:
        Error = AmbiguousExchange
    if Error:
        raise Error(params['username'], params['amount'], params['ts'], mogrified)
    print("{}.".format(nupdated))

with db.get_cursor() as cur:
    try:
        for filename in reversed(sorted(os.listdir(dirname))):
            data = json.load(open(os.path.join(dirname, filename)))

            for key in data:
                if key in ('links', 'meta'): continue
                things = reversed(data[key])

                if key == 'card_holds':
                    continue  # We don't track these.

                if key != 'debits':
                    continue  # Let's start with credit card charges.

                for thing in things:
                    print("{} | ".format(thing['created_at']), end='')

                    try:
                        process_thing(cur, thing)
                    except KeyboardInterrupt:
                        raise
                    except:
                        print("Exception! Thing:")
                        pprint(thing)
                        print('-'*78)
                        traceback.print_exc(file=sys.stdout)
                        print('='*78)

                    if i > I: raise Done()
                    i += 1
    except Done:
        pass
    print()
    raise Exception  # trigger rollback

Once this is done we can drop column exchanges.participant, since we'll have a reference through exchange_routes. That'll knock one out for #835.

Alright, so after some poking on #3806, I'm now thinking that we should pick one of the duplicates rather than deduplicating.

Working on this at Blue Canary with @kaguillera. Caught him up to speed, and now diving back in. We just discovered that later transactions in Balanced have an exchange_id and a participant_id. Looks like it starts around August, 2014. Let's see how much ambiguity that removes ...

We just verified that every transaction with a exchange_id also has a participant_id.

GitHub comments as version control. :cat:
#!/usr/bin/env python -u
from __future__ import absolute_import, division, print_function, unicode_literals

import json, os, sys, traceback
from gratipay import wireup
from decimal import Decimal as D
from pprint import pprint
from gratipay.models.exchange_route import ExchangeRoute
from gratipay.models.participant import Participant

db = wireup.db(wireup.env())
dirname = os.path.join(os.path.dirname(__file__), 'transactions')
i = 0
I = float('inf')
class Done(Exception): pass

class ExchangeMismatch(Exception):
    def __str__(self):
        return "{} {} {}\n\n{}".format(*self.args)
class NoExchange(ExchangeMismatch): pass
class AmbiguousExchange(ExchangeMismatch): pass

def get_participant(cur, customer_id, old_username):
    participant = cur.one("SELECT participants.*::participants FROM participants "
                          "WHERE balanced_customer_href='/customers/'||%s", (customer_id,))
    if participant is None:
        participant = cur.one("SELECT participants.*::participants FROM participants "
                              "WHERE username=%s", (old_username,))
        #XXX Check for username changes!
    if participant is None:
        raise Exception("Unable to get a participant.")
    return participant

def create_route(cur, thing):
    participant = get_participant(cur, thing['links']['customer'], thing['description'])
    route = ExchangeRoute.insert(participant, 'balanced-cc', '/cards/'+thing['links']['source'])
    return route

def pick_route(cur, routes, thing):
    import pdb; pdb.set_trace()
    route = None
    return route

def process_thing(cur, thing):
    print(thing['description'], end='')
    route, participant = None, None

    # Try to pick a route.
    routes = cur.all( "SELECT * FROM exchange_routes "
                      "WHERE network='balanced-cc' and address='/cards/'||%s"
                    , (thing['links']['source'],)
                     )
    if not routes:
        print("No route for {}.".format(thing['links']['source']), end='')
        route = create_route(cur, thing)
    elif len(routes) > 1:
        print("{} routes for {}.".format(len(routes), thing['links']['source'], end=''))
        route = pick_route(cur, routes, thing)
    else:
        route = routes[0]

    # Ack.
    if type(route.participant) is long:
        route.set_attributes(participant=Participant.from_id(route.participant))

    # Bail if we don't have a route.
    if not route:
        print()
        return

    # Use the username that is current as of this db transaction.
    username = cur.one( "SELECT username FROM participants WHERE id=%s"
                      , (route.participant,)
                       )

    params = dict( status   = thing['status']
                 , amount   = D(thing['amount']) / 100
                 , ts       = thing['created_at']
                 , route_id = route.id
                 , username = username
                  )

    nalready = cur.one("""\
        SELECT id FROM exchanges
         WHERE status = %(status)s
           AND route = %(route)s
           AND participant = %(username)s
           AND amount + fee = %(amount)s
           AND ((   (%(ts)s::timestamptz - "timestamp") < interval '0'
                AND (%(ts)s::timestamptz - "timestamp") > interval '-15m'
                ) OR (
                    (%(ts)s::timestamptz - "timestamp") > interval '0'
                AND (%(ts)s::timestamptz - "timestamp") < interval '15m'
                ))
    """, dict(params, route=route.id))
    if nalready == 1:
        print("Already linked!")
        return

    mogrified = cur.mogrify("""\
    WITH rows AS (
        UPDATE exchanges
           SET status = %(status)s
             , route = %(route_id)s
         WHERE (status is NULL or status = %(status)s)
           AND route is NULL
           AND participant = %(username)s
           AND amount + fee = %(amount)s
           AND ((   (%(ts)s::timestamptz - "timestamp") < interval '0'
                AND (%(ts)s::timestamptz - "timestamp") > interval '-15m'
                ) OR (
                    (%(ts)s::timestamptz - "timestamp") > interval '0'
                AND (%(ts)s::timestamptz - "timestamp") < interval '15m'
                ))
     RETURNING 1 AS count
    ) SELECT * FROM rows;
    """, params)
    print("Updating exchanges for {}: ".format(username), end='')
    nupdated = cur.one(mogrified, default=0)
    Error = None
    if nupdated == 0:
        Error = NoExchange
    elif nupdated > 1:
        Error = AmbiguousExchange
    if Error:
        raise Error(params['username'], params['amount'], params['ts'], mogrified)
    print("{}.".format(nupdated))

with db.get_cursor() as cur:
    try:
        for filename in reversed(sorted(os.listdir(dirname))):
            data = json.load(open(os.path.join(dirname, filename)))

            for key in data:
                if key in ('links', 'meta'): continue
                things = reversed(data[key])

                if key == 'card_holds':
                    continue  # We don't track these.

                if key != 'debits':
                    continue  # Let's start with credit card charges.

                for thing in things:
                    print("{} | ".format(thing['created_at']), end='')

                    try:
                        process_thing(cur, thing)
                    except KeyboardInterrupt:
                        raise
                    except:
                        print("Exception! Thing:")
                        pprint(thing)
                        print('-'*78)
                        traceback.print_exc(file=sys.stdout)
                        print('='*78)

                    if i > I: raise Done()
                    i += 1
    except Done:
        pass
    print()
    raise Exception  # trigger rollback

@whit537 - https://gist.github.com/ :stuck_out_tongue:

Yeah, yeah. :P

@rohitpaulk et al. Moving to a PR, #3807.

Got this far enough for the Balanced Shutdown, taking off that milestone now.

Done for the three TransferWise exchanges to date.

Moving to a new PR for the remaining exchanges: #3912.


Use medium+ instance for e2e tests

bors try

bors try

Can't do medium+ with free open source :(


Non deterministic error in tests

 Message:
Npgsql.PostgresException : 40P01: deadlock detected

DETAIL: Detail redacted as it may contain sensitive data. Specify 'Include Error Detail' in the connection string to include this information.

  Stack Trace:
NpgsqlConnector.<ReadMessage>g__ReadMessageLong|226_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
NpgsqlCommand.ExecuteNonQuery(Boolean async, CancellationToken cancellationToken)
Respawner.ExecuteDeleteSqlAsync(DbConnection connection)
Respawner.ExecuteDeleteSqlAsync(DbConnection connection)
Respawner.ExecuteDeleteSqlAsync(DbConnection connection)
Respawner.ResetAsync(DbConnection connection)
Respawner.ResetAsync(DbConnection connection)
<<AddResettableSqlDatabase>b__6>d.MoveNext() line 36
--- End of stack trace from previous location ---
SqlFixtureExtensions.UsingDbConnection(ITestWebService webService, AsyncAction`1 action) line 45
SimpleAsyncEvent`1.Emit(T value)
WebServiceTestsFixture.ResetAsync(CancellationToken cancellationToken) line 66
WebServiceIntegrationTest`1.InitializeAsync() line 68

Consider using 'Include Error Detail' in the connection string during integration tests

https://www.postgresql.org/message-id/1125336357.2589.8.camel@linda

The problem are Rebus background tasks interfering with Respawn database purge

The solution would be to pause all the service background tasks and dispose all RebusTestBus before resetting test fixtures


wf-recorder: Remove myself and CrazedProgrammer as maintainers

Description of changes

I don't use wf-recorder anymore.
@CrazedProgrammer is this fine for you as well?  If you don't plan on actively updating/maintaining this package again we should ensure that the maintainer list is empty so that it is clear that this package needs new maintainers.
Anyway, thanks for packaging wf-recorder.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

I'm curious: Are you now using something else for screen recording? wf-recorder was what I used last time, so if there is no better alternative I would take over maintainership.

Hey @yu-re-ka, sorry for the delay and thanks for that offer.
No, I'm not using some other software, I currently just don't need screen recording anymore (which makes regular updates and testing a bit too annoying as I already lack time/priority to maintain my other packages). wf-recorder is AFAIK still the best option for wlroots-based compositors (depending on the use case).
Should I add you as new maintainer then?


Version?

Is there a version of Quark Open Publishing that will work on grav v1.69, as I can not upgrade my php in order to get grav 1.7x, is there as I really would love to use this theme?

Hi @BigDC , the current version should work fine as it should only need Grav 1.6 or greater. Are you running into an issue?

Yes I am. I download the current version for this repo,I upload the inner grav-theme-quark-open-publishing-master directory and its contents to my grav themes folder, but when I go to activate it in grav admin it doesn't even show the theme as even being there, please help???

I would suggest you install the theme via the Admin Panel or the GPM (as described on this page): https://getgrav.org/downloads/themes

I tried renaming the the theme folder to quark-open-publishing still no luck
So I deleted the entire folder from themes and I actually tried doing it via the Admin Panel but it gave me an error saying one of the packages requires grav 1.7.21

Oops, my bad. I forgot to remove the word master from the name when I renamed it, I just did that now and it has installed manually.


Instant view of a website?

Is there any way to make a website or link to open in instant view when the user presses on a button of inline_keyboard?

No, you can't force this behavior.

Thanks