Inquiry into ippe_py

Hi Roy,
I'm doing research on fiducial marker-based navigation:
https://www.youtube.com/watch?v=3jVkphKte30
However, when the camera is more than 2 meters away from the tag, the pose estimation becomes very unstable. Then I employed the Kalman-filter and low pass filter but the result was still not promising.
I checked the history of the 2D image points and found that they were stable since I was using Apriltags. So I think the main source of the unstable estimation is the PnP method itself. And my case is similar to the following demo:
https://www.youtube.com/watch?v=Efb0zux-_hU
So I hope that I can use the IPPE instead of the default iterative method and I looked into this package.
I’m using python 3.6.9+ Opencv 4.4.1 where cv2.solvepnp() doesn’t have the flag of SOLVEPNP_IPPE.   And I’m also wondering if Opencv 4.4.1 works for the package.
I really appreciate any help

Hi, this IPPE was original developed for python2.7 environment, I think it might have some conflicts towards python3.6+ and latest version of opencv, numpy, I could spend some time this weekend, update it to py3 version

Thank you so so much. I feel like the IPPE is available in solvePnPGeneric because I can find an attribute called cv2.SOLVEPNP_IPPE, but I don't know how to use it. Here is what I tried:
import numpy as np
import cv2
import cv2.cv2 as cv

points_3d = np.array(([-8.6, 147, 31.2],
                  [8.6, 147, 31.2],
                  [8.6, 147, 14],
                  [-8.6, 147, 14]),
                 dtype=np.double)

points_2d = np.array(([354,114],
 [454,106],
 [467,202],
 [353,206]),dtype=np.double)

cam1_matrix = np.array(([330.6,0,326.8336],[0,330.6,251.8424],[0,0,1]),dtype=np.double)

dist_coefs1 = np.array([-0.302587121740377,0.068024115878877,0.001645011282553,0.002976587367681,0],dtype=np.double)

# found, rvec, tvec = cv2.solvePnP(points_3d, points_2d, cam1_matrix, dist_coefs1)
# print(rvec, tvec)
# found, rvec, tvec = cv.solveP3P(points_3d, points_2d, cam1_matrix, dist_coefs1,5,rvecs = None,tvecs =None)
# print(rvec[0], tvec[0])
found, rvec, tvec = cv.solvePnPGeneric(points_3d, points_2d, cam1_matrix, dist_coefs1,rvecs = None,tvecs =None, useExtrinsicGuess=None,flags=6,rvec = None,tvec =None,reprojectionError=None)
print(rvec[0], tvec[0])

Sorry to pull this request but all the tutorials I can find is about the default method, something like found, rvec, tvec = cv2.solvePnP(points_3d, points_2d, cam1_matrix, dist_coefs1).

did you tried with C++ version of opencv?

Hi, I found the following code worked. I wclose the issue now. Thanks!
found, rvec4, T0, repro = cv2.solvePnPGeneric(points_3d, points_2d, cam1_matrix, dist_coefs1, rvecs = None, tvecs = None, useExtrinsicGuess= None,flags = cv2.SOLVEPNP_IPPE_SQUARE ,
                    rvec= None, tvec = None,reprojectionError= 3)


Update pwsh to 7.1.14, separate image minting from vmss minting, and update VMs.

/azp run

/azp run

/azp run


[BUG] lora 训练后，想合并权重到原模型，发现不能成功。

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior
合并代码如下。
import torch
from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForCausalLM, LlamaTokenizer
def apply_lora(model_name_or_path, output_path, lora_path):
print(f"Loading the base model from {model_name_or_path}")
base = AutoModelForCausalLM.from_pretrained(
model_name_or_path, torch_dtype=torch.float16, low_cpu_mem_usage=True
)
base_tokenizer = LlamaTokenizer.from_pretrained(model_name_or_path)
print(f"Loading the LoRA adapter from {lora_path}")

lora_model = PeftModel.from_pretrained(
    base,
    lora_path,
    torch_dtype=torch.float16,
)

print("Applying the LoRA")
model = lora_model.merge_and_unload()

print(f"Saving the target model to {output_path}")
model.save_pretrained(output_path)
base_tokenizer.save_pretrained(output_path)

if name == "main":
model_name_or_path = '/models/Qwen-7B-Chat'
output_path = '../merge_out'
lora_path = '/Qwen-7B/output_qwen/checkpoint-500'
apply_lora(model_name_or_path, output_path, lora_path)
报错信息如下：The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'QWenTokenizer'.
The class this function is called from is 'LlamaTokenizer'.
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. If you see this, DO NOT PANIC! This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
Traceback (most recent call last):
File "/llm_projects/Qwen-7B/finetune/merge_lora.py", line 33, in
apply_lora(model_name_or_path, output_path, lora_path)
File "/llm_projects/Qwen-7B/finetune/merge_lora.py", line 11, in apply_lora
base_tokenizer = LlamaTokenizer.from_pretrained(model_name_or_path)
File "/root/anaconda3/envs/qwen1/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1854, in from_pretrained
return cls._from_pretrained(
File "/root/anaconda3/envs/qwen1/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2017, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/root/anaconda3/envs/qwen1/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 156, in init
self.sp_model = self.get_spm_processor()
File "/root/anaconda3/envs/qwen1/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 166, in get_spm_processor
tokenizer.Load(self.vocab_file)
File "/root/anaconda3/envs/qwen1/lib/python3.10/site-packages/sentencepiece/init.py", line 905, in Load
return self.LoadFromFile(model_file)
File "/root/anaconda3/envs/qwen1/lib/python3.10/site-packages/sentencepiece/init.py", line 310, in LoadFromFile
return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
TypeError: not a string
进程已结束,退出代码1
期望行为 | Expected Behavior
No response
复现方法 | Steps To Reproduce
No response
运行环境 | Environment
- Transformers: 4.33.1

备注 | Anything else?
No response

请参考https://github.com/QwenLM/Qwen#finetuning部分说明试一下。
目前已知merge后保存不会存储QWenTokenzier和其它代码，请暂时参考https://github.com/QwenLM/Qwen/issues/486处理，把文件复制过去就好（或者AutoTokenizer加载再保存，不能用LlamaTokenizer的）。


truncate seems to be lagging behind the size of the box

Reproducable in onTruncate callback case from the README
When shrinking the box, text goes to the second line even though lines={1} was passed into the props.

When expanding, text is truncated even though there is enough room.

Seems like something to do with the setState called in onTruncate, when I removed it things were fine.

Hey @yuxuansean1, if you are referring to what you can see here https://cl.ly/3ae9f8d1f164 - meaning the screenshot you took is just a temporary state before RTM updates it again, then you are correct.
RTM by itself does a few setStates in order to find the longest "substring" that still fits the container, but it still works fine and you don't notice any lag when rendering something once (most cases probably).
What happens here when resizing is that RTM does its thing on every resize event which is a lot of setStates, plus, as you mentioned, the onTruncate callback sets state as well. So I think it's reasonable to expect when resizing the box and then setting state based on the RTM result, that there will be some lag. But it's just a demo case - I've added the resize box to the examples just to demonstrate how the callback functionality works, but I don't expect that setup to be used in real apps.

Hi @patrik-piskay, thanks for your response.
The issue i'm seeing is that those screenshots are not a temporary state, they are what is displayed after the resize has finished.

Interesting! what browser is that? Can you verify that it happens in other browsers as well?

This was on Chrome, I reproduced this in Firefox, Edge and Safari as well.


Rework Deployment Section

Describe the issue or suggestion
During triage we discussed making some changes to the deployment section. @alexwolfmsft might have additions - feel free to make comments and we'll re-work.

[ ] Rework the ACA article to provide options for deploying via command line with AZD and via Visual Studio
[ ] Make sure the 2 CI/CD articles are up-to-date (GitHub Actions and AzDO Pipelines)
[ ] Add an article on customizing one's deployment that goes into using azd infra synth for IAC generation and customization
[ ] Add a new "production considerations" article that walks through a few of the suggested changes one would need to make post-azd infra synth to production-ready their apps. One thing we'd want to call out here is how to add a managed identity and role assignments per container app rather than using 1 single managed identity for the whole app
[ ] Cover the upcoming changes in the App Host-located provisioning code - those specific to the Construct approaches that are beginning to percolate throughout preview 5 - and explain how developers can use the RunAs and/or PublishAs methods to customize the deployment scenarios

cc @mitchdenny and @glennc to see if you had any addendums to the production section above or any additional concerns you think we'd need to mitigate. We're marking this with preview-5 but I feel confident we'll extend this as the updates come in for the provisioning changes.

Maybe another TODO is how to address in docs any commentary on k8s (and AKS) deployment at this time.

A couple thoughts here @bradygaster
Right now we have AZD, Bicep and CLI steps for ACA deployment. I agree we should prioritize AZD and Visual Studio (which we currently don't have) - but I'm assuming we also still want Azure CLI and Bicep. These approaches are much more manual, so my recommendation would be to keep them but reduce the size of those flows by extracting the CLI commands and Bicep code into completed scripts in the aspire-samples repo and provide an overview of them rather than keeping the tedious copy/paste approach of those currently. That way the simpler approaches are front and center and the other approaches are there if people want them.
I also think we should have some sort of (short) AZD + Aspire overview doc that is non specific to any one scenario that explains concepts such as:

Which Azure services are supported for auto provisioning/deployment
The relationship between AZD and .NET Aspire - basically extracting most of the content out of the existing "in-depth" tutorial into it's own section about how AZD consumes the resource info to "auto provision" via in-memory. I think the in-depth tutorial should go away personally, since that content would be handled by this new doc plus the improved standard aca deployment tutorial
Basics of infra synth
Basics of CI/CD option through AZD

The goal would be for these topics to provide context for the other docs that use AZD for specific scenarios since these are shared concepts.

Your summary looks good Brady. I think calling out the managed identity thing is good. It really depends on your architecture to determine whether that is something you would be concerned about.  Beyond that it just comes down to your preference around blast radius.

I am late to noticing this but seems good to me.

@alexwolfmsft did we lose the "production considerations" item? I didn't see that in the doc to which you linked and this is coming back up as a big deal for 9.0 we'd like to have implemented.


updated readme.txt

This fixes the issue  #1049

good innovatoin

@Vinit-bisht I know, its not a very big issue. But its Hacktober fest.So every PR matters:) I was not able to find any other major issues while testing this app.


Add functionality to run model inside place

Some projects like rbx-dom need to send a more complicated script to run inside the place. Instead of generating the place with all of that code, it would be useful to generate a model whose top-level instance is a ModuleScript and have run-in-roblox execute that script.
Alternatively, we could have the run-in-roblox runtime search the model for all Script instances and run them?

I don't know if this feature is super valuable since it makes run-in-roblox much more confusing.


Dropdown

There is something wrong with my multiselect.
After I linked my API into multiselect, when I open the dropdown all the data is already selected and cant even clicked the datas.

I Fixed it!

How did you solve? I can't figure out what I'm doing wrong. Exact same problem on this usage:
<MultiSelect :options="[{composicao: 'fck 89'},{composicao: 'fck 86'},{composicao: 'fck 84'}]"
    track-by="composicao" label="composicao" />


remove vendor prefix from preset

avoid duplication of property, remove vendor prefix from preset css definition

:tada: This issue has been resolved in version 1.0.0-beta.40 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


SepBot broken?

Not working, latest Telegram update? Is bot down? /start does nothing?

fixed

Nvm... working now, thanks!


[Accessibility][Screen Readers-Filter Dialog] : Focus should move to the chart section after selecting any dialog.

User Experience
Users will find it difficult to navigate through the application if focus order is incorrect. Here, focus should move to the flow chart section after selecting any of the dialog otherwise user would have to navigate through all the dialogs which will be time consuming.
Test Environment
Browser: Microsoft Edge Canary Version: 82.0.425.0 (Official build)
OS build: 2004 (19564.1005)
Screen Reader: Narrator
URL: Bot Framework Composer
Prerequisite Here
Repro Steps:

Open the above mentioned URL in EDGE and set the zoom at 400%.
Select 'Design Flow' from left navigation.
Now, select any dialog available in the filter dialog pane.
Observe the issue.

Actual:
When dialog gets selected, and user presses tab key, focus moves to the next dialog available in list item instead of the flow chart section.
Expected:
On selecting any of the dialog, the focus should move to the flow chart of the dialog so that the user knew about the opened chart.
Note:

Same issue found throughout the page, on activating any of the node property pane gets opened but the focus remains on the chart and when user press next tab new property pane gets opened. Due to this the keyboard user will not be able to reach property pane. I suggest the role should be assigned to the node or any shortcut should be defined to reach directly to the property pane.
MAS2.4.3_Property pane.zip

MAS Impacted: MAS2.4.3
Attachment:
MAS2.4.3_Focus should move to the first interactive element.zip

@cwhitten As checked the issue is not fixed yet as well as now the focus is not going inside the chart section with Tab key. On pressing tab from 'Show Code' the focus should come inside the tree pane but the focus stuck on the whole pane. As well as when we use mouse to take focus inside the pane the keyboard focus get tapped inside the chart section.
Please refer the video's for the same "Issue.zip"
I downloaded below Master build file last updated on 23rd April.
Test Environment
Browser: Microsoft Edge Dev {Version 84.0.488.1 (Official build) dev (64-bit)}
OS build: 2004 (19562.1000)
Screen Reader: Narrator
URL: Bot Framework Composer
Prerequisite Here

I think what's going on here is a confusion of two different issues. The original one was about the focus moving into the visual editor after selecting a dialog, and that has in fact been fixed.
However, there's a separate problem where Tab and Shift-Tab no longer work inside the visual editor because those keys have been forcibly remapped in ways that break at least Narrator - the visual-editor extension has a whole KeyboardZone component that overwrites the usual Tab button behavior with its own.

@beyackle The original issue is not fixed yet! Please check and validate.

I think there's some confusion here over the word "select". I had been interpreting it to mean "move to and hit Enter", selecting a dialog for display in the main window, and that case does move focus over. However, moving through the list of dialogs (without hitting Enter, just going back and forth through the list) does in fact use Tab instead of the arrow keys, and you are correct that this is unexpected behavior, given how things like the sidebar nav menu use the arrow keys.
I'll go ahead and fix that, because that is a problem; is that also the behavior you were referring to?

Yup! Sounds cool. The user should be able to navigate with arrow key and once hit enter the focus should jump to the opened chart.

#Nonbenchmark;


Cannot read property 'CheckOnce' of undefined

After updating to 0.5.6, got this error:

Cannot read property 'CheckOnce' of undefined

And shows a blank page.

It is because of angular2-meteor-auto-bootstrap. Moved there.


feat: symbol update command

Add a symbol update command that updates all props at the same time.
fixes #637
fixes #638

:tada: This PR is included in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


"Bonus to all pets" stat unnecessary spacing

Hi
The "bonus to all pets" statement always had a blank line before and after. Even in game.
But in game, Prefix/Suffix/BaseItem properties are mixed and the "bonus to all pets" have a fixed position in the end.
That said, in TQVaultAE, when the "Detailed tooltip view" is enabled and the "bonus to all pets" is the only property in its category, i agree it looks a bit weird (picture on the left).
I'm not shocked by the location of the property on the right picture (looks like in game).

Wild hunt spear: triple combo

racial tooltip color
all pets spacing
empty granted skill

can you share the item ?

Vault with items with one or another way weird displayed stats.
weird-stats.zip

Let's keep in mind that part of this is reported in #265


Update spark_executor.py

update

While I personally have nothing against comments in Chinese, these additions are not actually updates to this code, merely comments. Furthermore these comments seem to be related to basic features of python and not really about anything concerning what the code actually does, or expressing general confusion.
Furthermore this fails flake8. Unless there are substantial additions to the code you wish to make, I will close this PR with merging in 2 days.


Can not See / Connect to Stream

Hi,
i successfully Configurated everything. Everything is working... Chat Commands, Scene Switching and Connecting to rtmp Server via Streamelemtents App on Android and also GoPro.
But in my OBS isnt any Stream showing up. And in VLC it shows me: avcodec error: Failed to open rtmp://localhost:1935/publish/live: Operation not permitted
What i am doing wrong?
Best wishes,
Nevs08

What URL are you using to pull the stream into OBS? Also are you using VLC Video Source or the default Media Source in OBS?

I "fixed" it...
In the config.json File i changed the rtmp Key from live to someting else and this didnt let me connect to rtmp://localhost:1935/publish/live
I just changed it back to live and its working now.

Copy that, you should be able to change the Key in the config and in OBS and it should work. just FYI.


Comments on the logo

The zulip logo can be changed to make it unique. The zulip program can not be changed.
The old look is normal.

Do you have any suggestions for a better logo then? Feel free to join us on the #design stream to discuss

Closing for now. We'd be happy to chat on chat.zulip.org!


endless loop

Hi,
i use the odbc with the "AR System ODBC Driver".  it seems that dthe driver did not respond with the correct info if is select some columns.
In my case there is an endless loop when i select a char column. I did an odbc protocoll, there is the data, but it seems that the fetchall have a problem. Any idea?
This is the part of an endless loop:
node  --harmony 1778-2530	EXIT  SQLGetData  with return code 0 (SQL_SUCCESS)
		HSTMT               0x001328C8
		UWORD                        3
		SWORD                       -8 <SQL_C_WCHAR>
		PTR                 0x059D0020 [      20] "Zugewiesen"
		SQLLEN               1048575
		SQLLEN *            0x003DF514 (20)

node  --harmony 1778-2530	ENTER SQLGetData
		HSTMT               0x001328C8
		UWORD                        3
		SWORD                       -8 <SQL_C_WCHAR>
		PTR                 0x059D0020
		SQLLEN               1048575
		SQLLEN *            0x003DF514

node  --harmony 1778-2530	EXIT  SQLGetData  with return code 0 (SQL_SUCCESS)
		HSTMT               0x001328C8
		UWORD                        3
		SWORD                       -8 <SQL_C_WCHAR>
		PTR                 0x059D0020 [      20] "Zugewiesen"
		SQLLEN               1048575
		SQLLEN *            0x003DF514 (20)

node  --harmony 1778-2530	ENTER SQLGetData
		HSTMT               0x001328C8
		UWORD                        3
		SWORD                       -8 <SQL_C_WCHAR>
		PTR                 0x059D0020
		SQLLEN               1048575
		SQLLEN *            0x003DF514

node  --harmony 1778-2530	EXIT  SQLGetData  with return code 0 (SQL_SUCCESS)
		HSTMT               0x001328C8
		UWORD                        3
		SWORD                       -8 <SQL_C_WCHAR>
		PTR                 0x059D0020 [      20] "Zugewiesen"
		SQLLEN               1048575
		SQLLEN *            0x003DF514 (20)

the problem is the loop in odbc.cpp. in my case the one column index will be read again and again. In the moment i remove the loop - but this cannot be the solution


Character "&" before text in tabs

Describe the bug
When installing Ferdi and launching it I can see some "&" before some text.
To Reproduce
Steps to reproduce the behavior:

Install Ferdi
Open Ferdi
Sign / Create account
See error in the top of Ferdi

Expected behavior
It supposed to be only File, Edit, View and not &File &Edit &View
Screenshots

(screenshot taken in beta)
Desktop:

OS: Windows 10
Version: 5.3.3 and 5.3.4-beta.1

Additional context
The Todo tabs and windows tabs don't have this problem.

This issue got introduced through #27. The & should only get used internally and not get shown. Will have to see why the & get shown.

Thanks for the info.

@rseitbekov @vantezzen found a comment stating the accelerator is different based on the platform: https://github.com/electron/electron/issues/9584#issuecomment-304050674

Hmm, but it states that & is the right accelerator for windows - which this issue is referencing, so it still doesn't explain it.
If we can't find a solution we can also declare the accelerator separately.

Ah my bad, I don't know why I thought it was related to Ubuntu somehow 😅

Not sure if related: https://github.com/meetfranz/franz/pull/1323#issuecomment-482507454

@vantezzen can i take up this issue? I would like to fix this bug for my first contribution

@g-ravity Yes, its still up for grabs.

@g-ravity Are you woking on this?

Yes, I'm working on it. I was stuck at work this week, so I wasn't able to send the PR this week. But I'll surely send a PR within next week

Ok, no problem. I just didn't want this issue to go stale.

The project is not working properly. I don't why but suddenly many errors came up when I tried to start the project. I thought I'll be able to fix the issues and work on the bug, but it took me whole day and still I couldn't figure out why the project wasn't starting. I guess I'll leave this issue for someone else, as I don't have much time to work on this now

Is even the feature of doing #27   Alt + (First letter of the tabs) is even working? If not maybe the best will be to revert the pull request that caused this and then find another way to do it this time.

Is even the feature (#27 ) of doing Alt + (First letter of the tabs) is even working? If not maybe the best will be to revert the pull request that caused this and then find another way to do it this time.

@rseitbekov?

Ok I tested on Xubuntu 19.10 x64 and ALT+F or W etc is working and does not show "&". I think the bug happen only in Windows (and macOS X ??)

Doesn't show "&" on macOS either, just Windows.


Mirage is intercepting asset requests

Hello
I had previously issue to mock the proper request, but it's now nearly fixed.
I'm using ember-cli-rails.
When I run my test I got application.js:13 Uncaught ReferenceError: I18n is not defined :
Acceptance | user list: visiting /users (2, 0, 2) 60152 ms
Uncaught ReferenceError: I18n is not defined@ 165 ms
Source:
http://localhost:7357/assets/front.js:309
Test timed out

So I add to my config.js
this.pretender.get('http://localhost:7357/assets/**', this.pretender.passthrough);

But I still have this new error. It's strange because http://localhost:7357/ is my ember-test running

Would like to also upvote this issue.

@alexmasita I've finally move my translation code to ember. i18n is now not anymore with i18njs but with ember-18n.
If you have error like this one, you should check how to run test if you're using ember-cli-rails but especially add dependencies to test/index.html.

For me it can be close.


RM-616: Use project_id not versions_key to list apps

This PR fixes the deletion of projects/application from the dashboard.

OCD run https://ocd.appscale.com:8080/job/Daily Build/4850/

Merged master to pull in fixes for cassandra version: OCD build https://ocd.appscale.com:8080/job/Daily Build/4887/

re-run of the blobstore test that didn't have the instance coming up: https://ocd.appscale.com:8080/job/Blobstore/3411/


Flux Standard Actions compatibility

It would be cool to have a support for SFA.
In that case it will be much easier to combine redux-thunk with some nice projects like redux-actions and redux-promise-middleware.
Example of  using a combination of redux-actions, redux-promise-middleware and redux-thunk :
export const login = createAction(LOGIN, (credentials) => async (dispatch, getState) => {
    if (getState().user.isLoggedIn) {
      await dispatch(logout());
    }

    return api.login(credentials);
});

// LOGOUT_PENDING
// LOGOUT_FULFILLED
// LOGIN_PENDING
// LOGIN_FULFILLED

There are even two PR's (you can choose any): https://github.com/gaearon/redux-thunk/pull/130 and https://github.com/gaearon/redux-thunk/pull/105

Redux Thunk was always meant to be super simple. I’m not convinced we need to add more to it. Adding this is likely to help some people, but is also likely to break some other projects in weird ways where middlewares previously relied on Thunk ignoring objects. So I’d encourage you to fork redux-thunk as redux-thunk-fsa or something. If it becomes popular we can reconsider.

There are already several projects provide such functionality like
https://github.com/jtpalmer/redux-fsa-thunk
https://github.com/makeomatic/redux-thunk
https://github.com/ShMcK/redux-action-thunk
Existing of such project plus PR https://github.com/gaearon/redux-thunk/pull/105 from @bshy522 show the necessity of compatibility with FSA.


HTTPRouteMatchingAcrossRoutes sometimes fails

Description:
HTTPRouteMatchingAcrossRoutes sometimes fails
--- FAIL: TestGatewayAPIConformance (31.84s)
    --- FAIL: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes (1.05s)
        --- PASS: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/3_request_to_example.com/example_with_headers_should_go_to_infra-backend-v1 (0.02s)
        --- PASS: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/0_request_to_example.com/_should_go_to_infra-backend-v1 (0.02s)
        --- PASS: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/1_request_to_example.com/example_should_go_to_infra-backend-v1 (0.02s)
        --- PASS: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/5_request_to_example.net/v2_should_go_to_infra-backend-v1 (0.02s)
        --- PASS: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/2_request_to_example.net/example_should_go_to_infra-backend-v1 (0.02s)
        --- FAIL: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/4_request_to_example.com/v2_should_go_to_infra-backend-v2 (30.00s)
        --- FAIL: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/6_request_to_example.com/v2/example_should_go_to_infra-backend-v2 (30.00s)
        --- FAIL: TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/7_request_to_example.com/_with_headers_should_go_to_infra-backend-v2 (30.00s)

Repro steps:
seen in https://github.com/envoyproxy/gateway/pull/480
Logs:
Here's the HTTPRoutes spec https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/tests/httproute-matching-across-routes.yaml
Here's the Request/Response
ending Request:
< GET /v2 HTTP/1.1
< Host: example.com
< User-Agent: Go-http-client/1.1
< Accept-Encoding: gzip
<
<

Received Response:
< HTTP/1.1 200 OK
< Content-Length: 473
< Content-Type: application/json
< Date: Mon, 03 Oct 2022 18:56:08 GMT
< Server: envoy
< X-Content-Type-Options: nosniff
< X-Envoy-Upstream-Service-Time: 0
<
< {
<  "path": "/v2/example",
<  "host": "example.com",
<  "method": "GET",
<  "proto": "HTTP/1.1",
<  "headers": {
<   "Accept-Encoding": [
<    "gzip"
<   ],
<   "User-Agent": [
<    "Go-http-client/1.1"
<   ],
<   "X-Envoy-Expected-Rq-Timeout-Ms": [
<    "15000"
<   ],
<   "X-Forwarded-Proto": [
<    "http"
<   ],
<   "X-Request-Id": [
<    "1ee97601-099c-4889-9fa1-7e833983b2f6"
<   ]
<  },
<  "namespace": "gateway-conformance-infra",
<  "ingress": "",
<  "service": "",
<  "pod": "infra-backend-v1-7bdb567c9f-94nft"
< }

=== CONT  TestGatewayAPIConformance/HTTPRouteMatchingAcrossRoutes/6_request_to_example.com/v2/example_should_go_to_infra-backend-v2
    http.go:159: Response expectation failed, not ready yet: expected pod name to start with infra-backend-v2, got infra-backend-v1-7bdb567c9f-94nft

From the IR, we can see that that the first match is a prefix match to / for example.com which goes to infra-backend-v1 and there is also a /v2 prefix match for example.com that goes to infra-backend-v2
gateway-conformance-infra-same-namespace:
  HTTP:
  - Address: <IP_ADDRESS>
    Hostnames:
    - '*'
    Name: gateway-conformance-infra-same-namespace-http
    Port: 10080
    Routes:
    - AddRequestHeaders: null
      BackendWeights:
        Invalid: 0
        Valid: 0
      Destinations:
      - Host: <IP_ADDRESS>
        Port: 8080
        Weight: 1
      DirectResponse: null
      HeaderMatches:
      - Exact: example.com
        Name: :authority
        Prefix: null
        SafeRegex: null
      Name: gateway-conformance-infra-matching-part1-rule-0-match-0-example.com
      PathMatch:
        Exact: null
        Name: ""
        Prefix: /
        SafeRegex: null
      QueryParamMatches: null
      Redirect: null
      RemoveRequestHeaders: null
    - AddRequestHeaders: null
      BackendWeights:
        Invalid: 0
        Valid: 0
      Destinations:
      - Host: <IP_ADDRESS>
        Port: 8080
        Weight: 1
      DirectResponse: null
      HeaderMatches:
      - Exact: example.com
        Name: :authority
        Prefix: null
        SafeRegex: null
      - Exact: one
        Name: version
        Prefix: null
        SafeRegex: null
      Name: gateway-conformance-infra-matching-part1-rule-0-match-1-example.com
      PathMatch:
        Exact: null
        Name: ""
        Prefix: /
        SafeRegex: null
      QueryParamMatches: null
      Redirect: null
      RemoveRequestHeaders: null
    - AddRequestHeaders: null
      BackendWeights:
        Invalid: 0
        Valid: 0
      Destinations:
      - Host: <IP_ADDRESS>
        Port: 8080
        Weight: 1
      DirectResponse: null
      HeaderMatches:
      - Exact: example.net
        Name: :authority
        Prefix: null
        SafeRegex: null
      Name: gateway-conformance-infra-matching-part1-rule-0-match-0-example.net
      PathMatch:
        Exact: null
        Name: ""
        Prefix: /
        SafeRegex: null
      QueryParamMatches: null
      Redirect: null
      RemoveRequestHeaders: null
    - AddRequestHeaders: null
      BackendWeights:
        Invalid: 0
        Valid: 0
      Destinations:
      - Host: <IP_ADDRESS>
        Port: 8080
        Weight: 1
      DirectResponse: null
      HeaderMatches:
      - Exact: example.net
        Name: :authority
        Prefix: null
        SafeRegex: null
      - Exact: one
        Name: version
        Prefix: null
        SafeRegex: null
      Name: gateway-conformance-infra-matching-part1-rule-0-match-1-example.net
      PathMatch:
        Exact: null
        Name: ""
        Prefix: /
        SafeRegex: null
      QueryParamMatches: null
      Redirect: null
      RemoveRequestHeaders: null
    - AddRequestHeaders: null
      BackendWeights:
        Invalid: 0
        Valid: 0
      Destinations:
      - Host: <IP_ADDRESS>
        Port: 8080
        Weight: 1
      DirectResponse: null
      HeaderMatches:
      - Exact: example.com
        Name: :authority
        Prefix: null
        SafeRegex: null
      Name: gateway-conformance-infra-matching-part2-rule-0-match-0-example.com
      PathMatch:
        Exact: null
        Name: ""
        Prefix: /v2
        SafeRegex: null
      QueryParamMatches: null
      Redirect: null
      RemoveRequestHeaders: null
    - AddRequestHeaders: null
      BackendWeights:
        Invalid: 0
        Valid: 0
      Destinations:
      - Host: <IP_ADDRESS>
        Port: 8080
        Weight: 1
      DirectResponse: null
      HeaderMatches:
      - Exact: example.com
        Name: :authority
        Prefix: null
        SafeRegex: null
      - Exact: two
        Name: version
        Prefix: null
        SafeRegex: null
      Name: gateway-conformance-infra-matching-part2-rule-0-match-1-example.com
      PathMatch:
        Exact: null
        Name: ""
        Prefix: /
        SafeRegex: null
      QueryParamMatches: null
      Redirect: null
      RemoveRequestHeaders: null
    TLS: null

This gets translated to this Envoy route config and based on the Envoy route matching rules https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_conn_man/route_matching the first match gets picked
[name:"route_gateway-conformance-infra-same-namespace-http" virtual_hosts:{name:"route_gateway-conformance-infra-same-namespace-http" domains:"*" routes:{match:{prefix:"/" headers:{name:":authority" string_match:{exact:"example.com"}}} route:{cluster:"cluster_gateway-conformance-infra-matching-part1-rule-0-match-0-example.com"}} routes:{match:{prefix:"/" headers:{name:":authority" string_match:{exact:"example.com"}} headers:{name:"version" string_match:{exact:"one"}}} route:{cluster:"cluster_gateway-conformance-infra-matching-part1-rule-0-match-1-example.com"}} routes:{match:{prefix:"/" headers:{name:":authority" string_match:{exact:"example.net"}}} route:{cluster:"cluster_gateway-conformance-infra-matching-part1-rule-0-match-0-example.net"}} routes:{match:{prefix:"/" headers:{name:":authority" string_match:{exact:"example.net"}} headers:{name:"version" string_match:{exact:"one"}}} route:{cluster:"cluster_gateway-conformance-infra-matching-part1-rule-0-match-1-example.net"}} routes:{match:{prefix:"/v2" headers:{name:":authority" string_match:{exact:"example.com"}}} route:{cluster:"cluster_gateway-conformance-infra-matching-part2-rule-0-match-0-example.com"}} routes:{match:{prefix:"/" headers:{name:":authority" string_match:{exact:"example.com"}} headers:{name:"version" string_match:{exact:"two"}}} route:{cluster:"cluster_gateway-conformance-infra-matching-part2-rule-0-match-1-example.com"}}}]

api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.HTTPRouteMatch suggests the below logic when there is a tie
If ties still exist across multiple Routes, matching precedence MUST be determined in order of the following criteria, continuing on ties:

The oldest Route based on creation timestamp.
The Route appearing first in alphabetical order by “{namespace}/{name}”.
If ties still exist within an HTTPRoute, matching precedence MUST be granted to the FIRST matching rule (in list order) with a match meeting the above criteria.

When no rules matching a request have been successfully attached to the parent a request is coming from, a HTTP 404 status code MUST be returned.

Here are the HTTPRoutes
kubectl get httproute -A -o yaml
apiVersion: v1
items:
- apiVersion: gateway.networking.k8s.io/v1beta1
  kind: HTTPRoute
  metadata:
    creationTimestamp: "2022-10-03T18:55:38Z"
    generation: 1
    name: matching-part1
    namespace: gateway-conformance-infra
    resourceVersion: "7403"
    uid: db23e763-61ab-40f4-b157-a6b592b7e72c
  spec:
    hostnames:
    - example.com
    - example.net
    parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: same-namespace
    rules:
    - backendRefs:
      - group: ""
        kind: Service
        name: infra-backend-v1
        port: 8080
        weight: 1
      matches:
      - path:
          type: PathPrefix
          value: /
      - headers:
        - name: version
          type: Exact
          value: one
        path:
          type: PathPrefix
          value: /
  status:
    parents:
    - conditions:
      - lastTransitionTime: "2022-10-03T18:55:38Z"
        message: Route is accepted
        observedGeneration: 1
        reason: Accepted
        status: "True"
        type: Accepted
      controllerName: gateway.envoyproxy.io/gatewayclass-controller
      parentRef:
        group: gateway.networking.k8s.io
        kind: Gateway
        name: same-namespace
- apiVersion: gateway.networking.k8s.io/v1beta1
  kind: HTTPRoute
  metadata:
    creationTimestamp: "2022-10-03T18:55:38Z"
    generation: 1
    name: matching-part2
    namespace: gateway-conformance-infra
    resourceVersion: "7408"
    uid: fe79d86b-552f-451f-8db0-b805b75d9d82
  spec:
    hostnames:
    - example.com
    parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: same-namespace
    rules:
    - backendRefs:
      - group: ""
        kind: Service
        name: infra-backend-v2
        port: 8080
        weight: 1
      matches:
      - path:
          type: PathPrefix
          value: /v2
      - headers:
        - name: version
          type: Exact
          value: two
        path:
          type: PathPrefix
          value: /
  status:
    parents:
    - conditions:
      - lastTransitionTime: "2022-10-03T18:55:38Z"
        message: Route is accepted
        observedGeneration: 1
        reason: Accepted
        status: "True"
        type: Accepted
      controllerName: gateway.envoyproxy.io/gatewayclass-controller
      parentRef:
        group: gateway.networking.k8s.io
        kind: Gateway
        name: same-namespace
kind: List
metadata:
  resourceVersion: ""

based on the tie break rules, the logic in EG is correct and the request should be sent to infra-backend-v1 but conformance tests are expecting the request to be sent to infra-backend-v2.

@skriss is my understanding of the issue correct ?

another minor issue I see is  the Hostnames field is set to * by the Gateway API translator instead of the specific Hostname/virtual host such as example.com

I think this is a similar issue to a couple of the other conformance issues -- essentially, we need to sort routes so that those with longer (more specific) path match criteria come first. I.e. in this case, since "/v2" is a longer path match than "/", it should be the route that gets chosen when a request's path starts with "/v2". This is described in https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io%2Fv1beta1.HTTPRouteRule in the matches section, just above the part that you quoted above. Fixing this should fix a number of the failures, it just hasn't been implemented yet.
Re: hostnames, the way it's structured right now is that the IR routes have :authority header matches with the specific hostname from the Gateway API HTTPRoute. The IR listener itself has the wildcard hostname because it comes from the Gateway API Listener's hostname, which in this case was empty. We could structure this differently, by computing all of the different hostname intersections between Gateway API Listeners and HTTPRoutes, and then creating a unique IR Listener for each unique hostname, but it's more complicated and you lose the nice 1:1 mapping between Gateway API Listener and IR Listener. Can definitely discuss this further if folks have opinions on it.

For reference, here's Contour's logic for sorting routes: https://github.com/projectcontour/contour/blob/main/internal/sorter/sorter.go#L163


adding emulation code needed to create a docker container emulationg …

Hello hello 👋
Adding rover emulation, this creates a docker image that uses Ubuntu and ROS to control emulated roboclaws. More details in the readme.

Feels weird merging my own pull request even for something as benign as this. Going to leave it to someone else for now


Decision on editorializing

What's the plan?  Do we just want a brain dump that is welcoming?  Or do we want to curate this a bit?  If we go later, it'll be more useful to new comers, but then we have to make editorial judgements on people's contributions more.  And maybe rank software packages to some extent.

OK, agree


Applying the LM optimizer for PINNs

Hi, I would like to apply the LM optimizer for the training of physics informed neural networks (PINNs), I have to write a custom loss function so I followed one example that was mentioned in another issue and then I customized it to my problem. Here I am trying to solve a very simple ODE. As far as I understood I have to create a custom loss class and then I have to call the custom loss function in the call method. Here is my code:
import sys
import os.path
import tensorflow as tf
import math as m
import numpy as np
import time
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Layer, Activation, Dense, BatchNormalization
from tensorflow.keras.optimizers import Adam
import levenberg_marquardt as lm
import matplotlib.pyplot as plt

pi = tf.constant(m.pi)

inputs = tf.keras.Input(shape=(1,),dtype='float32')
layer1 = Dense(units=50, activation='tanh',dtype='float32')(inputs)
layer2 = Dense(units=50, activation='tanh',dtype='float32')(layer1)
predictions = Dense(units=1, activation='linear',dtype='float32')(layer2)
model = tf.keras.Model(inputs=inputs, outputs=predictions)
model.summary()

t0 = tf.Variable([[0.0]], shape=[1,1])

# Define custom loss
def custom_loss():

    # Write here the loss function
    t = tf.random.uniform(shape=[1000,1],maxval = 1.5)
    uIC = model_wrapper.predict(t0)
    with tf.GradientTape() as tape1:
        u = model_wrapper.predict(t)
    u_t = tape1.gradient(u, t)[:,0]

    tf.print("\n IC Loss is : ", tf.reduce_mean(tf.square(uIC - 1)), output_stream=sys.stdout)
    tf.print("\n ODE Loss is : ", tf.reduce_mean(tf.square(u_t-4*pi*tf.math.cos(2*pi*t))), output_stream=sys.stdout)

    loss = tf.reduce_mean(tf.square(uIC - 1)) + \
    10.0 * tf.reduce_mean(tf.square(u_t-4*pi*tf.math.cos(2*pi*t)))

    # Return a function
    return loss

class CustomLoss(tf.keras.losses.Loss):
    def __init__(self,
                 reduction=tf.keras.losses.Reduction.AUTO,
                 name='custom_loss'):
        super(CustomLoss, self).__init__(
            reduction=reduction,
            name=name)

    def call(self, y_true, y_pred):

        loss = custom_loss()

        return loss

    def residuals(self, y_true, y_pred):

        loss = custom_loss()
        eps = tf.keras.backend.epsilon()
        residuals = tf.math.sqrt(eps+loss)

        return residuals
        # you have to write here the code according to how you custom loss is defined
        # so that: loss = mean(residuals^2)

input_T = tf.linspace([0.0], [1.0], 1000) # We define a dummy input in order to use the fit method
output_T = tf.linspace([0.0], [1.0], 1000)  # We define a dummy output

model.summary()

# model.compile(
#     optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
#     loss=custom_loss())

model_wrapper = lm.ModelWrapper(
    tf.keras.models.clone_model(model))

model_wrapper.compile(
    optimizer=tf.keras.optimizers.SGD(learning_rate=1.0),
    loss=CustomLoss())

model_wrapper.fit(input_T,output_T, epochs=100)

tTest = tf.Variable(tf.linspace([0.0], [1.5], 1000))
tInit = tf.Variable([[0.0]])

with tf.GradientTape(persistent=True) as tape1:
    uIC = model_wrapper.predict(tInit)
u = model_wrapper.predict(tTest)

u_t = tape1.gradient(u, tTest)

As you can see the loss function contains two terms, I have the following error
'
uIC = model_wrapper.predict(t0)
RuntimeError: Method requires being in cross-replica context, use get_replica_context().merge_call()
'

Does it work using a standard Keras optimiser (eg. Adam)?
Can you provide me the math equations of the loss that you have to implement?
I find the code confusing, the loss that you implemented does not depend on y_true or y_pred.

Does it work using a standard Keras optimiser (eg. Adam)? Can you provide me the math equations of the loss that you have to implement? I find the code confusing, the loss that you implemented does not depend on y_true or y_pred.

Yes the code works with Adam optimizer or any Keras optimizer, the math equations which I try to solve are:
$$ u_t - 4\pi cos(2 \pi t) =0, \quad u(0) = 1 $$
The loss term does not depend on y_true or y_pred since we are solving the equations in data-free regime. I generate random points for the input variable $t$ and then I construct the operator of the math equation and finally I define the loss term which is a combination of two losses, the one coming from the ODE and the other from the initial condition. I hope this make it clearer. Thank you very much.

Hi @saddamhijazi I found a way to make it work. It is a bit tricky. My LM optimiser inside can call the model with a different batch size than the one set in the fit method in order to reduce memory usage. I had to fix proper size of the jacobian_max_num_rows, because I could not figure out a way to compute a gradient with respect to t for different sizes of t.
I organised the code so that you do no need to define a custom loss, but you can use the MeanSquaredError that is already implemented.
Usually LM works better in the overdetermined case (num_residuals >= num_parameters). But it is not mandatory, you can also try it in the underdetermined case.
For the following code:
num_residuals = num_steps * 2 (num_steps = 1000)
So as long as your model as less than 2000 parameters you are in the overdetermined case.
If you want to increase the model size then you can increase the num_steps, of course at a certain point you will run into memory issues.
In this experiment I trained a model with 321 parameters and it get's already very small training losses.
I hope it helps.
Here is the code:
import tensorflow as tf
import math as m
import levenberg_marquardt as lm

num_steps = 1000

t = tf.Variable(tf.zeros(shape=(num_steps, 1)))

class PINN(tf.keras.Model):
    def __init__(self):
        super().__init__()

        self.model = tf.keras.Sequential([
            tf.keras.layers.Dense(units=16, activation='tanh'),
            tf.keras.layers.Dense(units=16, activation='tanh'),
            tf.keras.layers.Dense(units=1, activation='linear')
        ])

    def call(self, inputs, training=None, mask=None):
        pi = tf.constant(m.pi)
        t0 = tf.constant([[0.0]])

        u_ic = self.model(t0)

        t_random = tf.random.uniform(shape=(num_steps, 1), maxval = 1.5)
        t.assign(t_random)

        with tf.GradientTape() as tape1:
            u = self.model(t)
        u_t = tape1.gradient(u, t)

        out0 = tf.broadcast_to(u_ic - 1.0, shape=(num_steps, 1))
        out1 = u_t - 4.0 * pi * tf.math.cos(2.0 * pi * t)

        k0 = 1.0 / num_steps
        k1 = 10.0

        out = tf.concat([k0 * out0, k1 * out1], axis=-1)

        return out

def main():
    x = tf.zeros((num_steps, 1))  # dummy input with one sample
    y = tf.zeros(shape=(num_steps, 2))

    pinn = PINN()

    yt = pinn(x)

    pinn.summary()

    # pinn.compile(
    #     optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
    #     loss=tf.keras.losses.MeanSquaredError(), run_eagerly=False)

    # pinn.fit(x, y, batch_size=num_steps, epochs=100000)

    model_wrapper = lm.ModelWrapper(pinn)

    model_wrapper.compile(
        optimizer=tf.keras.optimizers.SGD(learning_rate=1.0),
        loss=lm.MeanSquaredError(),
        jacobian_max_num_rows=num_steps * 2)

    model_wrapper.fit(x, y, batch_size=num_steps, epochs=100000)

if __name__ == '__main__':
    main()

Hi @fabiodimarco , thank you very much for your time and for the code. I tried to add the following lines at the end of the main method in order to visualize the results:
tTest = tf.Variable(tf.linspace([0.0], [1.5], 1000))
pi = tf.constant(m.pi)
y_PINN = pinn(tTest)
y_GT = 2*tf.math.sin(2*pi*tTest) + 1
print('tTest shape is ' + str(tTest.shape))
print('y_GT shape is ' + str(y_GT.shape))
print('y_PINN shape is ' + str(y_PINN.shape))

I notice that the PINN output has 2 columns, I don't understand what is the reason of that. How one should do a simple forward test using the trained PINN, is it like what I wrote above or is it using the model_wrapper ?
I also notice that by re-running the code several times, the loss sometimes converge to low values and sometimes not. Thank you very much.

I had to use 2 columns because the LM optimiser that I implemented support only one output tensor. For that reason, you can not have a tensor with shape (1000, 1) and one with shape (1, 1). To overcome the problem I repeated the second tensor 1000 times and concatenated the two to obtain just one tensor with shape (1000, 2).
out0 = tf.broadcast_to(u_ic - 1.0, shape=(num_steps, 1))
out1 = u_t - 4.0 * pi * tf.math.cos(2.0 * pi * t)

k0 = 1.0 / num_steps
k1 = 10.0

out = tf.concat([k0 * out0, k1 * out1], axis=-1)

This is how the first and second column are computed.
out1 is like in your case (1000, 1)
out0 is also (1000, 1) but it is just a repetition of the same value 1000 times, to get a (1,1) tensor you can just do: out0[0, :]
Now that I look again at the code I noticed that the correct values for k0 and k1 should be:
k0 = tf.math.sqrt(1.0 / num_steps)
k1 = tf.math.sqrt(10.0)

In order obtain the correct scaling values when they are squared by the MeanSquaredError.
Using pinn or the model_wrapper is the same, they will return the same values. The model_wrapper inside will call pinn.
The fact that the loss sometimes converge better than other times might depends on the initial random initialisation of the network. In some cases it might get stuck into some local minimum.

Thank you very much for the explanation @fabiodimarco , now I understand better the formulation, basically when I was calling the pinn method on the input values of $t$ I was getting an output of two columns which corresponds to the residuals of the two losses. As for the PINN solution $y$ it should be obtained by calling 'model' which is a member of the class PINN, as it is done below
tTest = tf.Variable(tf.linspace([0.0], [1.5], 1000)) // Generate uniform grid points for testing the model
pi = tf.constant(m.pi)
residuals_PINN = pinn(tTest) // compute the residuals of the two losses, residulas_PINN will be [1000 X 2]
y_PINN = pinn.model(tTest) // compute the PINN approximation of the ODE at the generated points
y_GT = 2*tf.math.sin(2*pi*tTest) + 1

Are you sure that a division by num_steps is needed for k1 ? isn't that a mean operation is already done by the loss function therefore you we don't need to divide by num_steps .
Finally you may imagine that I would like to solve more complicated equations using the PINNs and I understand that if we have more equations and more boundary conditions to be met then I must add more variables in the call function, for example assume that we have 3 equations and 10 boundary conditions then the function call must return a vector of 11 dimension correct me please if am wrong.
Thank you very much.

s for the PINN solution
it should be obtained by calling 'model' which is a member of the class PINN, as it is done below

Correct.

Are you sure that a division by num_steps is needed for k1 ? isn't that a mean operation is already done by the loss function therefore you we don't need to divide by num_steps .

You are right since the loss is the mean then division by num_steps is not needed.
Original loss:
loss = out0^2 + 10 * sum(out1^2) / 1000
New loss:
loss = k0^2 * out0^2 * 1000 / 2000 + k1^2*sum(out1^2) / 2000
so to be identical:
k0 = sqrt(2)
k1 = sqrt(2 * 10)

Finally you may imagine that I would like to solve more complicated equations using the PINNs and I understand that if we have more equations and more boundary conditions to be met then I must add more variables in the call function, for example assume that we have 3 equations and 10 boundary conditions then the function call must return a vector of 11 dimension correct me please if am wrong.

In the example above you have 1 equation and 1 boundary condition, the return dimension is 2.
Then I think if you have 3 equations and 10 boundary conditions the output dimension should be 13. Why do you think it is 11?

Thank you very much for the explanation @fabiodimarco , now I understand better the formulation, basically when I was calling the pinn method on the input values of $t$ I was getting an output of two columns which corresponds to the residuals of the two losses. As for the PINN solution $y$ it should be obtained by calling 'model' which is a member of the class PINN, as it is done below
tTest = tf.Variable(tf.linspace([0.0], [1.5], 1000)) // Generate uniform grid points for testing the model
pi = tf.constant(m.pi)
residuals_PINN = pinn(tTest) // compute the residuals of the two losses, residulas_PINN will be [1000 X 2]
y_PINN = pinn.model(tTest) // compute the PINN approximation of the ODE at the generated points
y_GT = 2*tf.math.sin(2*pi*tTest) + 1

Are you sure that a division by num_steps is needed for k1 ? isn't that a mean operation is already done by the loss function therefore you we don't need to divide by num_steps .
Finally you may imagine that I would like to solve more complicated equations using the PINNs and I understand that if we have more equations and more boundary conditions to be met then I must add more variables in the call function, for example assume that we have 3 equations and 10 boundary conditions then the function call must return a vector of 11 dimension correct me please if am wrong.
Thank you very much.

s for the PINN solution
it should be obtained by calling 'model' which is a member of the class PINN, as it is done below

Correct.

Are you sure that a division by num_steps is needed for k1 ? isn't that a mean operation is already done by the loss function therefore you we don't need to divide by num_steps .

You are right since the loss is the mean then division by num_steps is not needed.
Original loss: loss = out0^2 + 10 * sum(out1^2) / 1000
New loss: loss = k0^2 * out0^2 * 1000 / 2000 + k1^2*sum(out1^2) / 2000
so to be identical:
k0 = sqrt(2)
k1 = sqrt(2 * 10)

Finally you may imagine that I would like to solve more complicated equations using the PINNs and I understand that if we have more equations and more boundary conditions to be met then I must add more variables in the call function, for example assume that we have 3 equations and 10 boundary conditions then the function call must return a vector of 11 dimension correct me please if am wrong.

In the example above you have 1 equation and 1 boundary condition, the return dimension is 2. Then I think if you have 3 equations and 10 boundary conditions the output dimension should be 13. Why do you think it is 11?

Thank you very much for your answer, for the last point you are right it was a typo, the number of outputs is 13 in that case. I will try to see what are the advantages and limitations of using the LM optimizer for more complex problems. Thanks again for being responsive and for the code.

Hi @fabiodimarco , sorry for reopening the issue again, I would like to ask you one thing about training of the model using the LM optimizer after a pre-training procedure with the Adam optimizer. I do that with the aim of getting the model weights to a point that facilitates the training with the LM optimizer and consequently avoid getting stuck in a local minima. I have modified the code to the one below in order to do that. I have noticed that the LM optimizer does not benefit from the pre-training procedure of the Adam optimizer and it diverges often from the correct solution after the first epoch. Do you think that there is something wrong ?
Here is the code:
import tensorflow as tf
import math as m
import levenberg_marquardt as lm
import matplotlib.pyplot as plt

num_steps = 1000

t = tf.Variable(tf.zeros(shape=(num_steps, 1)))

class PINN(tf.keras.Model):
    def __init__(self):
        super().__init__()

        self.model = tf.keras.Sequential([
            tf.keras.layers.Dense(units=16, activation='tanh'),
            tf.keras.layers.Dense(units=16, activation='tanh'),
            tf.keras.layers.Dense(units=1, activation='linear')
        ])

    def call(self, inputs, training=None, mask=None):
        pi = tf.constant(m.pi)
        t0 = tf.constant([[0.0]])

        u_ic = self.model(t0)

        t_random = tf.random.uniform(shape=(num_steps, 1), maxval = 1.5)
        t.assign(t_random)

        with tf.GradientTape() as tape1:
            u = self.model(t)
        u_t = tape1.gradient(u, t)

        out0 = tf.broadcast_to(u_ic - 1.0, shape=(num_steps, 1))
        out1 = u_t - 4.0 * pi * tf.math.cos(2.0 * pi * t)

        k0 = tf.math.sqrt(2.0)
        k1 = tf.math.sqrt(2.0)

        out = tf.concat([k0 * out0, k1 * out1], axis=-1)

        return out

def main():
    x = tf.zeros((num_steps, 1))  # dummy input with one sample
    y = tf.zeros(shape=(num_steps, 2))

    pinn = PINN()

    yt = pinn(x)

    pinn.summary()

    pinn.compile(
        optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
        loss=tf.keras.losses.MeanSquaredError(), run_eagerly=False)

    # Pre-train the model using the Adam optimizer
    pinn.fit(x, y, batch_size=num_steps, epochs=2000)

    tTest = tf.Variable(tf.linspace([0.0], [1.5], 1000))
    pi = tf.constant(m.pi)
    y_PINN = pinn.model(tTest)
    y_GT = 2*tf.math.sin(2*pi*tTest) + 1

    # Visualize the pre-training results
    fig = plt.figure(figsize=(100, 50))
    ax = fig.add_subplot(111)
    ax.plot(tTest.numpy(),y_GT,'r',tTest.numpy(),y_PINN,'--b',markersize=10, linewidth=10)
    plt.xlabel(r'$t$', fontsize = 50)
    plt.ylabel(r'$u$', fontsize = 50)
    plt.grid(linewidth=2)
    plt.tick_params(axis='x',labelsize=30)
    plt.tick_params(axis='y',labelsize=30)
    plt.legend([r'$GT$', r'$PINN$'],fontsize = 30)
    plt.show()

    model_wrapper = lm.ModelWrapper(pinn)

    model_wrapper.compile(
        optimizer=tf.keras.optimizers.SGD(learning_rate=0.001),
        loss=lm.MeanSquaredError(),
        jacobian_max_num_rows=num_steps * 2)

    # Train using the LM optimizer
    model_wrapper.fit(x, y, batch_size=num_steps, epochs=100000)

    residuals_PINN = pinn(tTest)
    y_PINN = pinn.model(tTest)
    print('y Error is ' + str(tf.reduce_mean(tf.square(y_GT-y_PINN))))

    # Visualize the training results
    fig = plt.figure(figsize=(100, 50))
    ax = fig.add_subplot(111)
    ax.plot(tTest.numpy(),y_GT,'r',tTest.numpy(),y_PINN,'--b',markersize=10, linewidth=10)
    plt.xlabel(r'$t$', fontsize = 50)
    plt.ylabel(r'$u$', fontsize = 50)
    plt.grid(linewidth=2)
    plt.tick_params(axis='x',labelsize=30)
    plt.tick_params(axis='y',labelsize=30)
    plt.legend([r'$GT$', r'$PINN$'],fontsize = 30)
    plt.show()

if __name__ == '__main__':
    main()

I do not think pre-training with Adam will help you into not getting stuck into a local minima. Something that I see here which is not ideal is the LM learning rate which is too low (0.001).
You should try to get it to 1, you can set a bit less than 1 if you find it to be too unstable.
LM is already an adaptive optimiser which will decrease the learning rate if it is not able to reduce the loss.
Performing small steps will give you more possibility of getting stuck on local minima. If you perform larger steps and a more "noisy" training in general it will be easier to escape from local minima.
You can also try to reduce the number of attempts of the LM optimiser, by default attempts_per_step is set to 10. This means that LM will compare the loss before performing the training step and the loss after performing the step, if the loss is lower then perform the step and go to the next batch, if the loss is higher it "reduce the learning rate" (not exactly like that) and try
again for a maximum of attempts_per_step. After that number of attempts it just perform the step even if it does not decrease the loss.
Optimisers like Adam will always perform the step even if they increase the loss. So you could try to reduce attempts_per_step to make the training more "noisy" and explore more the solution space.
Let me know if it helps.

Thank you @fabiodimarco for your answer and for the tips, in fact changing the attempts_per_step to value of 2 has improved the rate of success of the algorithm, it succeeded in converging in 6 tries out of 10.
However, I am still surprised that you are saying that pre-training with the Adam optimizer will not necessary help the LM in converging. I have even tried a pre-training which is basically enough to converge to the correct solution, yet the LM optimizer would diverge on the very first step which I find counter intuitive if the goal of the algorithm is to minimize the loss.
I am now considering a more complex case with 14 different losses and when I run the code I got memory issues and the code was aborted not even after doing one epoch, I don't know if there are ways to alleviate this problem. Thank you very much.

Thank you @fabiodimarco, I will read the section you referred to and hopefully I will be able to improve the efficiency in my specific testcase. Thank you very much for being helpful.


Why does #authorize return true?

Hey there! I love the library, but this got me thinking.
Why does #authorize return true? If it raises an error, there’s really no use checking whether its return value is truthy.
If it returned the record that was passed into it, it could be used for one-liners like this:
render json: authorize(Foo.all)

I don't think there was any real intention behind returning true besides it seeming sane.
Please be aware that the example you posted probably doesn't work as you expect. Passing a scope to authorize will pass the entire scope to the policy, not authorize each record in the scope individually. See more thorough explanation.
One could make a point that forcing authorize to stand on its own would make it more explicit, but I'm not sure that I buy my own argument in this case.
Nevertheless, you're just a little late to the party. Had you come with this issue literally earlier the same day that you reported it, we could have considered it, but unfortunately this is a backwards compatibility breaking change, and we have just cut Pundit 1.0. Given semver, we can't consider this change before 2.0. Given Pundit's simplicity, I am doubtful that a 2.0 is going to happen for a really long time, if ever.

Yeah that timing was pretty unfortunate. I understand your reasoning completely.
I resorted to overriding the method in my ApplicationController:
def authorize(record, query = nil)
  record if super(record, query)
end

And about the scopes: I’m handling those cases in my policies so that they work as expected.
Anyway, thanks for giving it a thought! :+1:

Closing this since we won't tackle it for the foreseeable future.

Looks like this has been changed after all:
https://github.com/elabs/pundit/commit/21d4c1bd2d0a2d69b8c32653870cd63bf0ca0eae
Was a bit confused as this is only changed on master and I was wondering why the latest release wasn't behaving as documented in the master README!

Yes, master will be released as 2.0 at some point.


Add npub search

Hi mate can we add npub search to find creators, we need to be able to find plebs not just recipes. Thanks

Hello,
There is a user page at https://nostr.cooking/user/npub12gu8c6uee3p243gez6cgk76362admlqe72aq3kp2fppjsjwmm7eqj9fle6
Real search does not really work with nostr tho.


HIVE-25570: Hive should send full URL path for authorization for comm…

…and 'insert overwrite location'

What changes were proposed in this pull request?
Made changes such that hive will now be sending full url path for authorization for the command insert overwrite location

Why are the changes needed?
Otherwise, hive is only sending the URI specified in the input command which isn't consistent with other commands like Create external table with directory (where we send full URL for authorization).

Does this PR introduce any user-facing change?
No.

How was this patch tested?
Local machine, remote cluster

Pending tests.

Hey @nrg4878  @saihemanth-cloudera there seems to be an issue with this commit on master: https://github.com/apache/hive/commit/a3d302ff7047898745bb556ebe8d2e1cb49549b2
Only SemanticAnalyzer.java changes where pushed and query.out files are missing -- this is currently breaking upstream tests, can you please take a look?


Add TOC to portal page (2nd challenge)

About

Return of Portal's TOC

Fixed #80

We'll implement this again in future.


GA: Users understand that Cody will not use any code as context, unless explicitly configured.

done via enhanced context selector


can't modify angular2 state from grid events ?

I am using ag-grid for Angular2 and tried to do the following:
    onCellClick: e => {
      console.debug('cell clicked ', e);
      this.MyObject.foo = 'bar 22222';
    }

basically modifying some variable in my class and then showing it in my view.html.
MyObject: {{MyObject.foo}}

I can see the original value, but the modified value does not trigger UI update.
does ag-grid have any events that wire into Angular2 state?

ok so it looks like global grid events work in this fashion, it's just the cell click handler of the column does not..
i think i could get around this using global events.

You have two options available to you here - use the context object:
Parent: https://github.com/ceolter/ag-grid-angular-example/blob/master/angular-cli/src/app/from-component.component.ts
Child: https://github.com/ceolter/ag-grid-angular-example/blob/master/angular-cli/src/app/child-message.component.ts
Or the angular suggested way to do this, which is to use services.

hey thanks!
not sure how services apply here though? it just seems that the column click handler is not executed within angular context, so it wouldn't have any effect on services just like it doesn't on class variables.
also not sure how the parent/child ways can be leveraged from column click component.
i am now working on doing this via the global grid click handler which seems to be executed within angular2 context.
these are just my findings/assumptions, i may be wrong!


[request] add notification when the alarm is set

What do you think?

This would be notification clutter for me. There is already a system status bar icon when the alarm is set (although there are other apps that might cause that to be set, too).


CleverTap Android 12 Fix

Description
Push notifications sent while the app is open aren't working on Android 12. This PR aims to fix that.
Other changes
N/A
Tested
Tested locally - WIP
How others should test
This does not need to be tested by QA.
Related issues

Fixes #2087

Backwards compatibility
Yes

Closing this pull request as there are still issues with Android 12 using clevertap-react-native - 0.8.1. When this library is fixed a new pull request can be created with the updated dependency.


[SPARK-25852][Core] we should filter the workOffers with freeCores>=CPUS_PER_TASK for better performance

What changes were proposed in this pull request?
we should filter the workOffers with freeCores>CPUS_PER_TASK  for better performance
How was this patch tested?
Exist tests

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Yes, that is the comment I have been referring to. So it seems you can't filter, right? it's not scheduling work here. @jiangxb1987 do you know this part of the code?

What do you mean by "better performance" ? If that means we can spend less time on TaskSchedulerImpl.resourceOffers() then I agree it's true, but AFAIK it's never reported this can be a bottleneck of the whole cluster, so maybe the perf gain is trivial here. If you expect better task distribution over existing executors then I don't see any case can be improved by this proposed change. Please correct me if I'm wrong.

It may happen that a busy executor is marked as lost and later it re-register to the driver, in that case currently we call makeOffers() and that will add the executor into TaskSchedulerImpl.hostToExecutors. This is bad implementation here since it shall not have depend on the makeOffers() function to update a unrelated protected val, but that's what we have for now.


Error generating social_share_button

Include the gem in Gemfile , run bundle install and run rails generate social_share_button:install, it returns an error:
/config/application.rb:10:in <module:SocialShareButton>': uninitialized constant SocialShareButton::Rails::Application (NameError) 	from /SocialShareButton/config/application.rb:9:in <top (required)>'
from /home/user/.rvm/gems/ruby-2.2.1/gems/spring-1.4.1/lib/spring/application.rb:82:in require' 	from /home/user/.rvm/gems/ruby-2.2.1/gems/spring-1.4.1/lib/spring/application.rb:82:in preload'
from /home/user/.rvm/gems/ruby-2.2.1/gems/spring-1.4.1/lib/spring/application.rb:143:in serve' 	from /home/user/.rvm/gems/ruby-2.2.1/gems/spring-1.4.1/lib/spring/application.rb:131:in block in run'
from /home/user/.rvm/gems/ruby-2.2.1/gems/spring-1.4.1/lib/spring/application.rb:125:in loop' 	from /home/user/.rvm/gems/ruby-2.2.1/gems/spring-1.4.1/lib/spring/application.rb:125:in run'
from /home/user/.rvm/gems/ruby-2.2.1/gems/spring-1.4.1/lib/spring/application/boot.rb:18:in <top (required)>' 	from /home/user/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in require'
from /home/user/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in require' 	from -e:1:in '
how to solve?

You have use SocialShareButton with you Rails application name, that name is conflict with social-share-button!


REST API for resource specific metadata update

The existing endpoint supports only core metadata update. This needs to be extended to support each resource type specific metadata update.

@pkdash Can this be closed?

@aphelionz Closing.


feat: implement eth_get_transaction_count

https://github.com/kkrt-labs/kakarot-ssj/blob/b1f7f8c13be38edc081f0738c188561240c098b4/crates/contracts/src/kakarot_core/eth_rpc.cairo#L128-L130
Given an evm address, this will return the transaction count (none) of this address.
Use it where required in the codebase, for example:
https://github.com/kkrt-labs/kakarot-ssj/blob/b1f7f8c13be38edc081f0738c188561240c098b4/crates/evm/src/backend/validation.cairo#L35-L38
reference:
https://github.com/kkrt-labs/kakarot/blob/main/src/kakarot/eth_rpc.cairo#L43-L50

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged

Hey! I'm Gerson. 👋
Member of Dojo Coding. I have contributed before to other Kakarot projects like:

kakarot-rpc
ef-tests

Can I work on this issue?
Check my OnlyDust profile, I've contributed to many projects: Profile
How I plan on tackling this issue

Plan to Solve the Issue:

Study the codebase to understand the current structure and functionality.
Understand the problem that needs to be solved by thoroughly reading the issue details.
Leverage all provided resources within the issue to get a better grasp of the task at hand.
If I encounter roadblocks, I will ask questions ASAP in the Telegram group to move forward effectively.
I will open a PR and wait for ur feedback

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
I am a Experience Cairo smart contract developer with experience working on projects such as Just Art Peace, Dojo, Kart, TBA, and Shinigami. Before transitioning to Cairo development, I was a backend developer specializing in Rust.
My recent work with cairo starknet

https://github.com/keep-starknet-strange/raito/pull/128
https://github.com/keep-starknet-strange/raito/pull/143
https://github.com/horuslabsio/TBA/pull/45
https://github.com/horuslabsio/karst-core/pull/60
https://github.com/keep-starknet-strange/art-peace/pull/121
https://github.com/keep-starknet-strange/zkramp/pull/33
https://github.com/keep-starknet-strange/art-peace/pull/32
https://github.com/horuslabsio/TBA/pull/53

My recent work with rust

https://github.com/mubarak23/axum-user-auth
https://github.com/antoyo/rustc_codegen_gcc/pull/1
https://github.com/antoyo/rustc_codegen_gcc/pull/5
https://github.com/fedimint/fedimint/pull/5994
https://github.com/mubarak23/soroban-bank

How I plan on tackling this issue
Am going to

implement the function
Look through the codebase where ever the function is call and make sure the code build successfully
Write unit test for the function
ETH: 18hrs

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
I am a Experience Cairo smart contract developer with experience working on projects such as Just Art Peace, Dojo, Kart, TBA, and Shinigami. Before transitioning to Cairo development, I was a backend developer specializing in Rust.
My recent work with cairo starknet

https://github.com/keep-starknet-strange/raito/pull/128
https://github.com/keep-starknet-strange/raito/pull/143
https://github.com/horuslabsio/TBA/pull/45
https://github.com/horuslabsio/karst-core/pull/60
https://github.com/keep-starknet-strange/art-peace/pull/121
https://github.com/keep-starknet-strange/zkramp/pull/33
https://github.com/keep-starknet-strange/art-peace/pull/32
https://github.com/horuslabsio/TBA/pull/53

My recent work with rust

https://github.com/mubarak23/axum-user-auth
https://github.com/antoyo/rustc_codegen_gcc/pull/1
https://github.com/antoyo/rustc_codegen_gcc/pull/5
https://github.com/fedimint/fedimint/pull/5994
https://github.com/mubarak23/soroban-bank

How I plan on tackling this issue
Am going to

implement the function
Look through the codebase where ever the function is call and make sure the code build successfully
Write unit test for the function
ETH: 18hrs


Bump version up to 1.0.2.

Codecov Report

Merging #231 (e320e2d) into main (64a756c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #231   +/-   ##
=======================================
  Coverage   96.92%   96.92%
=======================================
  Files          32       32
  Lines        2468     2468
  Branches      222      222
=======================================
  Hits         2392     2392
  Misses         75       75
  Partials        1        1

Flag
Coverage Δ

unittests
96.92% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed
Coverage Δ

src/cipherSuiteNative.ts
100.00% <ø> (ø)


Allow custom retry_decorator to be passed to the LLM

The retry decorator for ChatOpenAI is hardcoded here
Allow the user to supply a custom retry_decorator.

Users many want specific types of exceptions to be handled in specific ways, the current retry_decorator is limiting especially when different LLM hosts have different rate limits and exponential decay for subsequent requests.

@AnishPimpley Any update regarding this? Has this been addressed in future versions? I'm running through the codebase now to check myself


Failed to open file /mine/153, -57/11/405

I have been using this script with 16 turtles 8 miners and 8 chunky turtles and at minimum 5 days non stop mining.
But yesterday i ran into this problem, and i fixed it by removing all of the turtle files. but today the system broke in the same way.
Steps to reproduce:
Setup system with 8 mining turtles 8 chunky turtles, leave them on for a few days, and hope it breaks
Pressing reboot on the main controll screen or reboot might of have created the issue.
or update.
Turtles all display Nil.
if you cant reproduce it i could give you my server ip so you can check it out?

My Discord: PanGeo_#5337

Fixed! Hub was opening mine files for reading but not closing them, and it hit a maximum limit.


Auth all paths

Changes proposed in this pull request:

Fix auth_all_paths for openapi3
Code cleanup

Cool, thanks for this fix @cziebuhr !
I'm curious about the change from get(..., {}) to setdefault(...). Any particular technical reason for this, or just style? I'm happy either way, just want to learn if I missed something.
Can you split your first commit into (1) fix the bug, (2) change the style? It took me longer than it should have to find the bug (not getting securitySchemes in oas3, test not applying to both specs) in the many lines that were changed.

It's hard to split, because the reason it failed is that properties have been saved in AbstractAPI which only worked for swagger 2. security_definitions is the only property which gets accessed outside of add_operation. For the other ones, one could also omit the setdefault lines and use get with default value directly in add_operation. The difference is, that a new dict or list object would get created for each operation when the respective key is missing in the specification instead of sharing them among all operations. Maybe i'm thinking too much in memory optimization.
For long-term, one could argue if it makes sense to have two different classes (swagger2/openapi3) also for the whole spec and not only for each operation.

It's hard to split, because the reason it failed is that properties have been saved in AbstractAPI which only worked for swagger 2.

Wait, isn't the bug in the title fixed by the following minor change?
_security_schemes = self.specification.get('components', {}).get('securitySchemes', {})
self.security_definitions = self.specification.get('securityDefinitions', _security_schemes)

FWIW I absolutely agree with the refactor, my feedback was just that it was hard find that change, since your PR changes a lot more than what's in the title. Sorry if I am missing something obvious - it's getting late here. Admittedly I haven't tried to pull your diff and split it myself. I'll give that a go tomorrow so I know what I'm talking about.
I really like your idea of making spec classes. Now that raw_spec and specification exist as separate things (specification has been ref-resolved). You could move the setdefaults to the constructor, and use the Spec object to build the correct Operation as well. It would be nice to have as few if version < ... statements around as possible.
Great idea! Maybe I'll do a proof of concept diff tomorrow. :thinking:

Wait, isn't the bug in the title fixed by the following minor change?
_security_schemes = self.specification.get('components', {}).get('securitySchemes', {})
self.security_definitions = self.specification.get('securityDefinitions', _security_schemes)

You're right.
If you do a proof of concept: For #686 I also had in mind to move parts of security_decorator into AbstractAPI, because 80% of the code (especially calling the verify_* wrappers) are only depending on the security definition, but not on the security object of each operation. It would save memory to reuse that wrappers. I just wasn't sure on how to cleanly implement it, because the current structure e.g. passes components from oas3 directly to the operation objects. I thought about some kind of preprocessing or caching, but that would have increased complexity. If we can get an object which represents a swagger/openapi specification, we can do better preprocessing or use object oriented methods for accessing parts of the specification instead of passing bunches of dicts around.

hey @cziebuhr , are you happy to close this?
#712 should have addressed the bug and #713 and #726 include refactor and cleanup.

The only thing which is not handled is the resolver error cleanup, but that's only minor.


Proposal: Another type of arrayFormat

Hi,
I think that would be good to add another type of arrayFormat in both stringify and parse methods for joining array values with a comma.
The reason of my proposal is that with the OpenAPI 3 specification they allow to do that (see: OpenAPI spec > fixed fields style and explode fields).
I can understand that you want to keep the functionality of this package as standard as possible (based in the HTTP spec) but I think that would be a nice feature for those who use this package to build API-related url query strings.
Of course this would only provide a code style improvement, since you can reach the same result with the current functionality by doing something like this:
queryString.stringify({
    foo: [1,2,3].join(`,`),
    bar: ['a','b','c'].join(`,`)
});

But I think is much more clear to do it this way:
queryString.stringify({
    foo: [1,2,3],
    bar: ['a','b','c']
}, { arrayFormat: "noExplode"});

I would like to submit a PR if you accept my proposal.
PD:
Another option could be to allow the user build its own arrayFormatter by passing a callback instead of a string... Maybe?
Looking forward your feedback!!

I second the need for this.
Currently when handling dynamic requests we need to iterate over the payload and manually join any arrays before passing them to query-string. This defeats the purpose of using a library to abstract away the URL parsing.

PR welcome. Should include docs and tests :)

+1 from me as well. I actually submitted issue #122 and PR #124 trying to solve a problem related to this issue.
In my scenario, I'm working with an API url that uses the following array format for tags:
tags=postal%20office,burger%2C%20fries%20and%20coke

And I was resorting to something similar to what @d-asensio describes:
let tags = ['postal office', 'burger, fries and coke'];

let params = {
  tags: tags.map(t => encodeURIComponent(t)).join(',') // encode manually
}

let qs = queryString.stringify(params, { encode: false }); // disable encoding

This worked just fine for creating the query string, but I stumbled into limitations when parsing it. Since the API url allows commas inside tags, and those are decoded automatically by parse, I was getting the following:
let params = queryString.parse(qs)
// => { tags: 'postal office,burger,fries and coke' }

let tags = params.tags.split(',');
// => ['postal office', 'burger', 'fries and coke'

Therefore my issue suggesting adding the decode option to parse, which lets me do:
let params = queryString.parse(qs, { decode: false })
// => { tags: 'postal%20office,burger%2C%20fries%20and%20coke' }

let tags = params.tags.split(',').map(t => decodeURIComponent(t));
// => ['postal office', 'burger, fries and coke'

I share my specific scenario here in case it might bring more light to the usefulness of this issue and #122.

I think this issue was addressed with #167


Update travis cache

Sometimes travis appears to fail (see here) because AmplNLWriter is in the cache but then it runs an old version.
Let's just revert to the old Travis of only caching Ipopt.

Codecov Report

Merging #74 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #74   +/-   ##
=======================================
  Coverage   81.36%   81.36%
=======================================
  Files           4        4
  Lines         676      676
=======================================
  Hits          550      550
  Misses        126      126

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a81ccb...660efcd. Read the comment docs.


Update 301 links

Addresses #1815
Remaining 301's to review
aka.ms
https://aka.ms/nupkgservicing
redirected to: https://blogs.msdn.microsoft.com/dotnet/2014/01/22/net-4-5-1-supports-microsoft-security-updates-for-net-nuget-libraries/
status code: 301 (object permanently moved)
linked from page(s):
https://docs.microsoft.com/en-us/dotnet/articles/core/tools/dotnet-pack
https://aka.ms/sdkimplicititems
redirected to: https://github.com/dotnet/core/blob/master/release-notes/1.0/sdk/1.0-rc3-default-compile-items.md
status code: 301 (object permanently moved)
linked from page(s):
https://docs.microsoft.com/en-us/dotnet/articles/core/tools/project-json-to-csproj
https://aka.ms/vstest-filtering
redirected to: https://github.com/Microsoft/vstest-docs/blob/master/docs/filter.md
status code: 301 (object permanently moved)
linked from page(s):
https://docs.microsoft.com/en-us/dotnet/articles/core/tools/dotnet-test
fwlink
https://go.microsoft.com/fwlink/?linkid=843444
redirected to: https://download.microsoft.com/download/8/F/9/8F9659B9-E628-4D1A-B6BF-C3004C8C954B/dotnet-dev-osx-x<IP_ADDRESS>.pkg
status code: 302 (object temporarily moved)
linked from page(s):
https://docs.microsoft.com/en-us/dotnet/articles/core/macos-prerequisites
localization redirects
There are 14 that redirect for localization and have this pattern:
https://docs.microsoft.com/nuget/consume-packages/configuring-nuget-behavior
redirected to: https://docs.microsoft.com/en-us/nuget/consume-packages/configuring-nuget-behavior
status code: 301 (object permanently moved)
linked from page(s):
https://docs.microsoft.com/en-us/dotnet/articles/core/tools/dotnet-nuget-push
There are ~43 that I can't characterize with this pattern:
https://github.com/dotnet/core-docs/blob/master/docs/core/deploying/index.md
redirected to: https://github.com/dotnet/docs/blob/master/docs/core/deploying/index.md
status code: 301 (object permanently moved)
linked from page(s):
https://docs.microsoft.com/en-us/dotnet/articles/core/deploying/index

For your remaining 301's:

I wouldn't change aka.ms and FwLink links, especially to installers. I'd also probably roll back the change to the https://aka.ms/dotnetcoregs. But we can change the Fwlinks to have a /p option to make them a 301 redirect instead of 302. e.g. https://go.microsoft.com/fwlink/p/?linkid=843444
for the localization redirects, we'd have to decide which behavior is better: being able to open the links on GitHub preview or not having a 301, because those links need to be locale-agnostic.
So they either look like https://docs.microsoft.com/nuget/consume-packages/configuring-nuget-behavior or /nuget/consume-packages/configuring-nuget-behavior (but then don't work on GH preview)
changing from core-docs to just docs is great!

LGTM! Let's wait for the engineering team to fix the issue with our builds before we merge this one.

roll back the change to the https://aka.ms/dotnetcoregs

:+1:

change the Fwlinks to have a /p option

:+1:

[IList<T>]<xref:System.Collections.Generic.IList-1> -> <xref:System.Collections.Generic.IList%601> and other spots with - that need %60

:+1:

not having a 301

I'd vote for customers not getting redirected

from core-docs to just docs

Looks like those are coming from the contributors section (see the last
 element):

all

Thanks @tdykstra ... I'm having fun with datetime docs. I'll get these changes in as soon as datetime docs are lov'in life :heart: again.


Config files for best reported CoNLL results

Hi,
Can you provide your config files for the reported CoNLL numbers (under section 3. Performance)?
Thanks,
Yi

The hyperparameters are listed in  this paper. Generally, the final results are various in different computer or environment, you can change the seed number to get different results.

results are not reproducible by using params written in paper. Can you better provide config file for your results.

@dhwajraj please give me your log file in your experiments. I will help you checking if you use the wrong setting or data format.


Need support on Spark 3.2.x, Spark 3.3.x

I found currently, ZparkIO Test cannot work with Spark 3.2.x and Spark 3.3.x
I simply created a project from demo (sbt new leobenkel/zparkio.g8), fixed some issues to make it work. I need to read a delta table so I added delta-core in my sbt file.
When I run the application I found delta-core is not working with 3.1.x spark. And if I upgrade to 3.2.x and run again I found the following issue.
[error] stack trace is suppressed; run 'last update' for the full output
[error] stack trace is suppressed; run 'last ssExtractDependencies' for the full output
[error] (update) lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo1.maven.org/maven2/org/apache/avro/avro-mapred/1.10.2/avro-mapred-1.10.2-hadoop2.jar: not found: https://repo1.maven.org/maven2/org/apache/avro/avro-mapred/1.10.2/avro-mapred-1.10.2-hadoop2.jar
[error] (ssExtractDependencies) lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo1.maven.org/maven2/org/apache/avro/avro-mapred/1.10.2**/avro-mapred-1.10.2-hadoop2.ja**r: not found: https://repo1.maven.org/maven2/org/apache/avro/avro-mapred/1.10.2/avro-mapred-1.10.2-hadoop2.jar
[error] Total time: 4 s, completed Dec 10, 2022, 9:37:14 AM

I will look into it

Done. Please confirmed that it is working :)


fix: JS and dart toString name conflict

[ ] Closes #10477

ISSUE (up to flutter 3.7.3)
JavaScript toString conflict with dart toString, just use @JS('toString') annotation.

Fixed in https://github.com/firebase/flutterfire/pull/10476


⚠ Bookkeeper to the Smart People has degraded performance

In 721f915, Bookkeeper to the Smart People (https://bksp.com.au) experienced degraded performance:

HTTP code: 200
Response time: 7706 ms

Resolved: Bookkeeper to the Smart People performance has improved in 48afd11.


fix summary information from MUQ about found depedencies

using the latest master with updates to cmake to use find_package, now we have some output that is wrong:
#  Compiler definitions used for this build:
#        COMPILE_DEFINITIONS:
#
#  Required dependencies:
#        EIGEN3 ----------------> Met with existing library:
#                                 Include Directory:
#                                   /Users/fnrizzi/Desktop/test/eigen/install/include/eigen3
#
#        BOOST -----------------> Met with existing library:
#                                 Include Directory:
#
#
#        HDF5 ------------------> Met with existing library:
#                                 Include Directory:
#                                   /opt/homebrew/Cellar/hdf5/1.14.3_1/include
#                                 Libraries:
#                                   hdf5-shared
#                                   hdf5_cpp-shared
#
#        NANOFLANN -------------> Met with existing library:
#                                 Include Directory:
#
#
#        SUNDIALS --------------> Met with existing library:
#                                 Include Directory:
#
#
#        NLOPT -----------------> Met with existing library:
#                                 Include Directory:
#
#                                 Libraries:
#                                   NLopt::nlopt
#
#        PARCER ----------------> Not required for selected compile groups.
#
#        SPDLOG ----------------> Not required for selected compile groups.
#
#        OTF2 ------------------> Not required for selected compile groups.
#
#        STANMATH --------------> Met with internal build -- Could not find library.
#

which we need to fix

fixed in #70


add workflow for dependabot to automerge based on allowlist

Summary
For dependabot rotations, often many of them are just approving + merging the PR. Trying to set up some automation here using an allow-list for automerges to save some time.
adapted from https://blog.somewhatabstract.com/2021/10/11/setting-up-dependabot-with-github-actions-to-approve-and-merge/ and https://github.com/dependabot/fetch-metadata
This PR sets up a github action that will start approving and auto-merging dependabot PRs that are updating a package in an allow list if CI passes AND if it is not a major update (under the assumption that major updates may cause more breaking changes that we would want to manually validate)
will try to test on this branch not sure if i can manually simulate a dependabot
i only set one allow list for now to test with this PR https://github.com/coda/packs-sdk/pull/1953. If that works, can add all the @types... ones and more during dependabot rotations for packages that feel safe
@coda/packs

nvm i need to actually merge the workflow in first for it to even appear and run

yeah, I tried this before. https://github.com/coda/packs-sdk/pull/1207/files


Default repo field

Describe the problem
Currently you need to do "changelog": ["@svitejs/changesets-changelog-github-compact", { "repo": "sveltejs/eslint-config" }],. I realized though that the default changesets formatter doesn't require you to pass repo
Describe the proposed solution
I wonder if we could make it simpler to setup by just automatically getting the repo the same way as the default formatter
Alternatives considered
Status quo
Importance
nice to have

this is based on their changelog-github which also requires you to provide the repo
https://github.com/changesets/changesets/blob/main/packages/changelog-github/src/index.ts#L14
we may be able to read it from root package.json instead but that could be brittle.

Ah, I see. I didn't realize they have a separate @changesets/changelog-git and @changesets/changelog-github
Yeah, reading it from the package.json seems potentially tricky, so probably not worth trying that
Thanks!


Added New Relic

New Relic seems to fit in multiple categories, so I've added them to all I think makes sense. Please let me know if that's not best practice.
Thanks 👍

I think it makes sense 👍
Thanks for the contribution!

Thank you for putting this together 🥇


Skip runner download and install if already installed

Description
Currently the gitlab runner user data will try to download/run install script then install. This change will allow alternative installation via userdata_pre_install
Migrations required
NO
Verification
Tested without issue using default install and when already installed.
Documentation
pre-commit run fine, no change to terraform inputs or docs required.

@Swsystem-ATuin thanks aadded to my queue for PR's this weekend


fix: fix mv don't work in different mount points

issue: https://github.com/uutils/coreutils/issues/1411

Closed in favor of https://github.com/uutils/coreutils/pull/1415


[Enhancement] Make this anonymous inner class a lambda[SendSyncMessageProcessor]

Search before asking

[X] I had searched in the issues and found no similar issues.

Enhancement Request

located at:
eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendSyncMessageProcessor.java line 260
analysis and explanation:
Before Java 8, the only way to partially support closures in Java was by using anonymous inner classes. But the syntax of anonymous classes may seem unwieldy and unclear.
With Java 8, most uses of anonymous inner classes should be replaced by lambdas to highly increase the readability of the source code.
Describe the solution you'd like

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Yes I am willing to submit a PR!

@Sunstreaker Welcome to your pr.