Captive portal

With your release 1.0.3, you added simpleServerCaptive.ino and I was testing with this.
It unfortunately did not work, as it tries to open https://<IP_ADDRESS>/Setup on my smartphone; there is no https support here.
So, I tried to force the redirect string to http://<IP_ADDRESS>/Setup in the example with:
myIP = WiFi.softAPIP();
String redirect = String("http://")+myIP.toString()+String("/setup");
server.startCaptivePortal("ESP_AP", "123456789", redirect.c_str());

but that resulted in failure and following logging:
[I][CaptiverPortal.hpp:46] handleRequest():	Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:46] handleRequest():	Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:46] handleRequest():	Captive handler triggered. Requested app.rundefit.com/api/system/open/collect -> redirecting to http://<IP_ADDRESS>http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:46] handleRequest():	Captive handler triggered. Requested <IP_ADDRESS>http//<IP_ADDRESS>/setup -> redirecting to http://<IP_ADDRESS>http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:46] handleRequest():	Captive handler triggered. Requested <IP_ADDRESS>http//<IP_ADDRESS>/setup -> redirecting to http://<IP_ADDRESS>http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:46] handleRequest():	Captive handler triggered. Requested <IP_ADDRESS>http//<IP_ADDRESS>/setup -> redirecting to http://<IP_ADDRESS>http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:46] handleRequest():	Captive handler triggered. Requested <IP_ADDRESS>http//<IP_ADDRESS>/setup -> redirecting to http://<IP_ADDRESS>http://<IP_ADDRESS>/setup

So, the CaptiverPortal.hpp (Shouldn't be called CaptivePortal.hpp ?) CaptiveRequestHandler class C'tor gets changed Wifi ip and host url upon each request.
The logging looks promising:
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested play.googleapis.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested connectivitycheck.gstatic.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup
[I][CaptiverPortal.hpp:47] handleRequest():.Captive handler triggered. Requested play.googleapis.com/generate_204 -> redirecting to http://<IP_ADDRESS>/setup

All requests are redirected to the IP address of the AP, however the sign-in page on my phone stays empty and finally shows connection timeout.
Looking at the AP details, the AP now has IP address <IP_ADDRESS>, so the earlier attempt to connect to http://<IP_ADDRESS>/... might be the right way...
to be continued.

I just saw you pushed changes for version 1.0.4, testing that.

The captive portal working in version 1.0.4. The setup page is shown and new credentials can be entered.
Logging shows that the ESP connects to the new WIFI network, however the captive portal is still active, and setup page is still serviced. A forced remote restart from setup page update & FS menu is needed.
Only after that serial logging shows a restart and reconnect to setup page is possible. Only then the scan wifi networks button does something.
[I][AsyncFsWebServer.cpp:385] handleScanNetworks():.Start scan WiFi networks
[I][AsyncFsWebServer.cpp:387] handleScanNetworks():. done!
Number of networks: -2

One more point.
After entering the new wifi credentials, and this is succesful, there is a popup, there is a popup with a link to "the new LAN address", however, the new lan address is not visible, and holding my finger (on smartphone screen), the text is very tiny (~3x8mm?) and disappears under my finger.

related to wifi, but not captiveportal.
request from:
.  http://<IP_ADDRESS>/wifistatus
returns:
.  {"firmware": "22:28:00", "mode":"Station", "ip":2030217408}
the ip address is 32-bit value in little endian byte order. This is not portable; network byte order, e.g. IP stack, is big endian.
Expected: string "<IP_ADDRESS>"
Question: Could "firmware" also include the library version?

Hi @rob040 and first of all, thanks for your collaboration on this library!
I tried to include many of your suggestions (also from other issues), have you had a chance to try them?
As for the string reported as firmware, you can customize it using the method setFirmwareVersion();
The maximum length of the string is limited to 15 characters because it seems enough at the time. Eventually it is possible to add the possibility of re-allocating memory if necessary,

Hi @cotestatnt, thanks for the confidence. I will help and improve wherever I can. I have a few projects in the works and at least one older project that I like to use this library for, to get them a better responsive web page and OTA functionality.
I've tried the captive portal again; it kind of works, but not satisfied with behaviour. I think a device restart will fix that, but the client webbrowser, usually a smartphone, should have been redirected to the already known and shown IP address. The disappearance of the ESP AP will (should) cause the smartphone wifi to re-activate its normal wifi connection and connect to same LAN as the ESP is connected to.
wrt. firmware version, it would be nice if the library version was included in the default firmware version, shown in the example apps. However there is no getter function for the user application to get library version to be able to present that. The user application is indeed able to use the setFirmwareVersion() to show its version in the setup page.
Having the library version presented in any API, would mean that upon release, you have a bit more work of updating the version number on at least two places, adding the possibility of making mistakes.
Did you notice I changed strncpy into strlcpy ; it is safer in assuring zero termination the string regardless of the supplied string length. I did not check if there are multiple occurrences of that.
A fw version length of 15 chars is IMO plenty enough.

One other issue I ran into while testing with ESP8266, is that OTA download result in exception. I'll try to investigate further and get at least more details, then enter an issue here to get that tracked.

Having the library version presented in any API

It's quite easy to implement, but it would be great to find a way that automatically retrieves information from the library.properties file. If I add a simple #define in the source, I'm sure I'll forget to update it every time :pensive:
Regarding the issue with ESP8266, I've noticed the same behaviuor. I've tried to mess with ESP8266 WDT, but with no lucky.
I would like to leave support for this mcu, but I won't deny that for me it is of secondary importance because I never use it in my projects.

Hi @rob040
I've added the method const char* getVersion();
The definition of method is inside a distinct file named version.cpp.
In this way I can automatically update the content of file with a workflow which reads current library version value from library.properties on each push


Sanitize input

I'm looking for a way to sanitize input values from a mutation, lets say:
#[derive(InputObject)]
struct UserInput {
    username: String,
    email: String,
}

how can i now sanitize the email field, like trimming whitespaces?
I have tried to implement a custom directive, but when i am not sure how to use it, or if it is at all possible with InputObjects.
I tried to us it this way, but that's not working, the macro throws an Unknown field: `sanitize`  error
pub struct SanitizeDirective {
    value: String,
}

#[async_trait::async_trait]
impl CustomDirective for SanitizeDirective {
    async fn resolve_field(&self, _ctx: &Context<'_>, resolve: ResolveFut<'_>) -> ServerResult<Option<Value>> {
        resolve.await.map(|value| {
            value.map(|value| match value {
                Value::String(str) => Value::String(self.value.trim().to_string()),
                _ => value,
            })
        })
    }
}

#[Directive(location = "field")]
pub fn sanitize(value: String) -> impl CustomDirective {
    SanitizeDirective { value }
}

#[derive(InputObject)]
struct UserInput {
    username: String,
    #[graphql(sanitize)]
    email: String,
}

Currently custom directives are only used for client requests,  you can use the process_with attribute to do it.
https://github.com/async-graphql/async-graphql/blob/382d54d5817e1bd3259c01712c76f6edc50d1081/tests/input_object.rs#L614


Reduce peak memory usage of TensileCreateLibrary

Brief
This PR reduces the peak memory usage of TensileCreateLibrary via customized YAML loader and tweaks on logic loading flow.
Implementation

[x] Customized YAML loader(with pyyaml's event mechanism) for improvement on speed and memory usage.
[x] Add faster path to get child of root element in YAML file.
[x] Use above to properly filter out mismatched archs durling logic listing.
[x] Allow ParallelMap2 to return generator instead of list for overlapping logic reading and library merging.

Comparison
TensileCreateLibrary
Tested on building gfx942 with 64 workers:
Before

Peak memory usage:
mprofile_20240904190006.dat     108930.523 MiB

After

mprofile_20240906183806.dat     70725.227 MiB

Peak memory usage reduces to ~65% with ~9% build time increase.
For Building with All Archs
Here's the memory footprint for building all archs with 32 workers:

peak memory usage:
mprofile_20240907151401.dat     71030.605 MiB

Customized YAML Loader
Here's the data for loading aldebaran/110CU/GridBased/aldebaran_Cijk_Alik_Bljk_HHS_BH_Bias_AH_SAV.yaml logic in repo:

file size
peak memory
time(sec)

Customized
14 MiB
111 MiB
5.06

pyyaml built-in
14 MiB
1039 MiB
9.65

@Serge45 @jichangjichang do we know why these tests are failing? Is there a plan to resolve the failures soon?


@jdneo I don't see Delegate Test to Gradle option in my VS Code:

          @jdneo I don't see Delegate Test to Gradle option in my VS Code:

Any hints how to enable that feature?
Originally posted by @piotr-rachwalik in https://github.com/microsoft/vscode-java-test/issues/1045#issuecomment-2349249731

@jdneo I checked and this feature works on my Windows machine. On my Mac, it doesn't work. Perhaps I need to reinstall the VS Code to make this work.

@jdneo I've changed this setting

and now I can see the Delegate to Gradle test option.
But when I try to run the test, I'm getting this error:
Project is not a Gradle build server project: <my_project>
java.lang.IllegalArgumentException: Project is not a Gradle build server project: <my_project>
	at com.microsoft.gradle.bs.importer.handler.GradleDelegateCommandHandler.executeCommand(GradleDelegateCommandHandler.java:40)
	at org.eclipse.jdt.ls.core.internal.handlers.WorkspaceExecuteCommandHandler$1.run(WorkspaceExecuteCommandHandler.java:230)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:47)
	at org.eclipse.jdt.ls.core.internal.handlers.WorkspaceExecuteCommandHandler.executeCommand(WorkspaceExecuteCommandHandler.java:220)
	at org.eclipse.jdt.ls.core.internal.handlers.JDTLanguageServer.lambda$4(JDTLanguageServer.java:606)
	at org.eclipse.jdt.ls.core.internal.BaseJDTLanguageServer.lambda$0(BaseJDTLanguageServer.java:87)
	at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646)
	at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)

I also tried with on/off setting of build gradle server, but the error is still the same:

@jdneo perhaps it is because, the delegate test to gradle option does not honor the settings.gradle, where I have the settings:
rootProject.name = '<root_project_name>'
def subProjectsDir = new File(rootDir, 'subprojects')
fileTree(subProjectsDir) {
  include '**/build.gradle'
  exclude '**/bin/**/build.gradle'
  exclude '**/build/**/build.gradle'
  exclude '**/src/**/build.gradle'
}.collect {
  subProjectsDir.toPath().relativize(it.parentFile.toPath()).toString()
}.each {
  def projectName = ':' + rootProject.name + '-' + it.replace(File.separator, '-')
  include(projectName)
  project(projectName).projectDir = new File(subProjectsDir, it)
}

Please note that when I use Eclipse IDE with Buildship then I'm able to run a test using gradle.

@piotr-rachwalik Thank you for sharing.
The error that Project is not a Gradle build server project is caused by the fact that the project is not imported by gradle build server, but buildship instead. There are several cases that will lead to this.
Have you tried running Java: Clean Java Language Server Workspace? If does vscode command does not help, would you mind sharing the folder structure of your project?

@jdneo I've tried running Java: Clean Java Language Server Workspace but it didn't help.
Here is my folder structure:
/build.gradle
/settings.gradle
/.gitignore
/subprojects/
   /project1/src/
                       main/java/
                       main/resources/
                       test/java/
                       test/resources/
                       build.gradle

and so we have multiple gradle projects in the subprojects directory.

@jdneo I think the root cause might be the ./gradlew eclipse command that I'm using to make my project usable in Eclipse / VS Code. Without that, the dependencies are not resolved, etc.
I think that command builds the projects into Buildship readable format.

I guess I may found the root cause.
The gradle extension needs to register its gradle test runner to the test runner for java extension via the api exported by test runner for java. Thus, the gradle extension will try to activate the test runner extension and then call those apis.
The problem is that, the test runner extension is depend on redhat.java extension. Its initialization work needs to be done after the java extension is ready. Turns out that VS Code has some issue that when activate test runner extension via the extension api, vs code won't activate the java extension (Or, vs code won't wait until java extension is ready). So the registration is actually failed.
The fix for now can be that we manually activate java extension then the test runner extension, which guarantees that all the required components are ready.


K9 + Openkeychain = no automatic decryption and encryption as attachment

Expected behavior
Tell us what should happen
Issue 1: I suppose K9 should automatically decrypt all incoming emails once Openkeychain has been turned on in settings and private key is provided. Decryption works fine if I copy/paste the encrypted text into Openkeychain.
Issue 2: After encryption PGP raw data should come in the body of email, not as an attachment.
Actual behavior
Tell us what happens instead
Issue 1: K9 does not automatically decrypt incoming emails, I only see raw pgp data + there is an icon in K9 saying email is not encrypted.
Issue 2: When I encrypt emails with K9+Openkeychain, the recipient receives a blank email with two attachments named 'Unknown Name', with my encrypted message inside.
Steps to reproduce
Issue 1:

Install latest K9
Install latest Openkeychain and import my private key + other people's public keys
In K9 settings select Openkeychain as pgp provider and also my private key
Asked my friend to email me, got encrypted email but K9 did not automatically decrypt it.

Issue 2:
1 and 3. same as above
4. Created a new email, selected the recipient from my contacts, the lock turned green indicating there is a public key for this recipient. I taped send, it asked me to enter my password and this is it
Environment
K-9 Mail version: 5.201
Android version: 6.0.1
Account type (IMAP, POP3, WebDAV/Exchange): irrelevant, the issue is the same with any account

This is intended behavior, following the PGP/MIME standard rfc3156. This actually puts an encrypted structure in the message, rather than just having it as the text body, which shows up in clients that don't support it as an attachment.
You can get the old behavior by enabling PGP/INLINE, and this should also be used automatically if you reply to such a message. Note that PGP/INLINE is only parsed right at the top of a message.

I have enabled PGP/INLINE but the recipient on the other side cannot decrypt it. He receives an error saying he does not have a key to decrypt it. Any idea why is this?

Got the same problem. How to enable this?


Document Group management API for savings and loans in groups

Document Group management API for savings and loans in groups

Hey @awasum , I have resolved the conflicts emanating from #7 and #8 and documented updateGroup() & updateGroupDefinition() which @kengneruphine added. Please kindly take a look and give feedback.

Ok

Looks good. thanks


[TS migration] Migrate 'SignInDesktop' page

Details

[TS migration] Migrate 'SignInDesktop' page, files:

src/pages/signin/DesktopRedirectPage.js
src/pages/signin/GoogleSignInDesktopPage/index.website.js
src/pages/signin/GoogleSignInDesktopPage/index.js
src/pages/signin/DesktopSignInRedirectPage/index.website.js
src/pages/signin/DesktopSignInRedirectPage/index.js
src/pages/signin/AppleSignInDesktopPage/index.website.js
src/pages/signin/AppleSignInDesktopPage/index.js

Fixed Issues

$ https://github.com/Expensify/App/issues/31988
PROPOSAL: N/A
Tests

Pre-condition:

Render the web Sign In with Google button in Development. APPLE_GOOGLE_SIGNIN.md#render-the-web-sign-in-with-google-button-in-development
Update newExpensifyURL so we can use https://dev.new.expensify.com:8083 in place of https://new.expensify.com/
Start Dektop App (npm run desktop) at :8082.
Start Web App (npm run web) at :8083.
Manually login with any google account to the Web App (dev.new.expensify.com:8083). Don't use the Google Sign Button, it won't work.
Update the webpack.dev.js config to change host from dev.new.expensify.com to localhost and server type from https to http. APPLE_GOOGLE_SIGNIN.md#hostport-requirements

Steps to test:

Go to the Desktop App and click on the Google SignIn button.
Verify that you have been redirected to .../sign-in-with-google and the URL automatically updates to .../desktop-signin-redirect.
Click on Try Again to be redirected to the Desktop App.
Verify that we are logged into DesktopApp with the same account we used in Pre-condition Step 5.
Logout in the DektopApp.
In the Desktop App click on the Apple SignIn button.
Verify that you have been redirected to .../sign-in-with-apple and the URL automatically updates to .../desktop-signin-redirect.

[x] Verify that no errors appear in the JS console

Offline tests

N/A
QA Steps

Same as Tests above
PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

N/A

Android: mWeb Chrome

N/A

iOS: Native

N/A

iOS: mWeb Safari

N/A

MacOS: Chrome / Safari

N/A

MacOS: Desktop
https://github.com/Expensify/App/assets/95647348/7a7ab7e8-d6c7-4de3-b3e3-525c8a304c8a

Please, I need help to test this migration. I'm still not clear what the steps are, to test and ensure all sign in on all platforms works fine.

I still haven't managed to enable SSO options on my local dev.

Currently working on Steps test and POCs

@brunovjk Sorry, could not respond to your earlier pings. I've been checking other PRs. I'll check and get back in a few hours.

I think visiting these routes
https://github.com/Expensify/App/blob/3ab4e6e12d9ff38e69493321f0d31f032481b373/src/ROUTES.ts#L50-L52
and checking if they are working as expected with nothing broken and no new console warnings and errors should do.
You might want to modify this line for dekstop prompt in dev environment.
https://github.com/Expensify/App/blob/3ab4e6e12d9ff38e69493321f0d31f032481b373/src/components/DeeplinkWrapper/index.website.tsx#L65

Thank you @c3024 I saw your message before, still working on those tests. This PR will be ready for review soon. At the latest by the end of this week 02/09, but I believe sooner.

@c3024 The PR is ready for review.

In the pre-condition, at this step

Login with any google account to the Web App (dev.new.expensify.com:8083)

I am not able to login with a Google account on web. I am getting a blank white screen.
https://github.com/Expensify/App/assets/102477862/5b688cf7-2e37-4230-a832-ad4928061b15
I followed all the preceding steps in the pre-condition. Could you help me with this? @brunovjk

@c3024 Thanks for the review. My bad, I didn't make it so clear, that in step 5 I logged in with a Google account not through the GoogleSignIn Button, but I entered the account manually, sorry about that. During the tests I did, I tested several 'Step test Path', none led to accurate reproduction, but none had a failure or error in relation to migration. Actually I'm not very happy with these steps, I'll redo/improve it and let you know.

Hello @blazejkustra were you able to test using the steps I provided? Thank you: D

Hello @blazejkustra were you able to test using the steps I provided? Thank you :D

No, I only reviewed typescript-wise @brunovjk 😄

Nice, thank you. I agree, the TS migration is looking good. I will keep working on those steps today, I believe it is important for the QA team.

I think the test steps cover well enough to verify the changes made.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

@youssef-lr @c3024  Could you help with "Precondition" part for the QA team?

My bad @kavimuru . The QA steps do not require the pre-condition because the buttons are already rendered. Please follow the following steps.
QA steps
Google SignIn

Sign out of Desktop and Web apps
Go to the Desktop app and click on the Google SignIn button.
Verify that you have been redirected to the URL .../sign-in-with-google in the browser
Login with a Google Account and verify that URL automatically updates to .../desktop-signin-redirect.
Verify that it is directed to Desktop app. (If there is a prompt to open on Desktop app click on the prompt to redirect to Desktop app)
Verify that we are logged into Desktop app with the same account logged in at Step 4

Apple Signin

Sign out of Desktop and Web apps
Go to the Desktop app and click on the Apple SignIn button.
Verify that you have been redirected to the URL .../sign-in-with-apple in the browser
Login with an Apple Account and verify that URL automatically updates to .../desktop-signin-redirect.
Verify that it is directed to Desktop app. (If there is a prompt to open on Desktop app click on the prompt to redirect to Desktop app)
Verify that we are logged into Desktop app with the same account logged in at Step 4

Please suggest any corrections if required here https://github.com/Expensify/App/pull/35406#issuecomment-1943896890.
@brunovjk

True, we don't need pre-conditioning for QA tests, I hadn't realized that. @c3024 should I edit the PR 'Details' with these QA steps? They look good to me.

Done :)

🚀 Deployed to production by https://github.com/thienlnam in version: 1.4.41-12 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅


Included golang.org/x/crypto detected CVE-2024-45337

When analising an image created with paketo contributing this buildpack, the package golang.org/x/crypto is detected as vulnerable because of CVE-2024-45337.
Expected Behavior
Images created should be clear of any vulnerabilities.
Current Behavior
Images are detected as vulnerables because of that dependency.
Possible Solution
According to CVE sources (https://www.cve.org/CVERecord?id=CVE-2024-45337, https://avd.aquasec.com/nvd/cve-2024-45337) the version 0.31.0 has the vulnerabity fixed.
Steps to Reproduce

Create an image using paketo contributing this buildpack.
Use a CVE analisis tool, like Trivy,

trivy --cache-dir .cache/trivy image '--exit-code=1' '--ignore-unfixed=false' '--no-progress=true' '--offline-scan=true' '--severity=HIGH,CRITICAL' '--skip-db-update=true' '--timeout=20m0s' '--scanners=vuln' '--ignorefile=trivyignore_joined' '--vuln-type=library' my-service

...

layers/paketo-buildpacks_bellsoft-liberica/helper/helper (gobinary)
===================================================================
Total: 1 (HIGH: 1, CRITICAL: 0)

┌─────────────────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┬────────────────────────────────────────────────────┐
│       Library       │ Vulnerability  │ Severity │ Status │ Installed Version │ Fixed Version │                       Title                        │
├─────────────────────┼────────────────┼──────────┼────────┼───────────────────┼───────────────┼────────────────────────────────────────────────────┤
│ golang.org/x/crypto │ CVE-2024-45337 │ HIGH     │ fixed  │ v0.28.0           │ 0.31.0        │ Misuse of ServerConfig.PublicKeyCallback may cause │
│                     │                │          │        │                   │               │ authorization bypass in golang.org/x/crypto        │
│                     │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2024-45337         │
└─────────────────────┴────────────────┴──────────┴────────┴───────────────────┴───────────────┴────────────────────────────────────────────────────┘

Motivations
Even CVE are most of the time "false-positives", we try to have up-to-date our code running on production and with the less possibilites of being vulnerable by any reason.

Looks like it's CRITICAL now:

I don't believe that the helper is actually impacted by this. You can see that we are only using crypto libs for sha256 and x509. My quick read of that issue is that it's related to public key crypto, which is not something that we're doing in the helper.
https://github.com/search?q=repo%3Apaketo-buildpacks%2Flibjvm crypto&type=code

I cut a release of Bellsoft Liberica buildpack just now. You can use that now if you're in a hurry to get this patched, or you can wait for the composite to update. That will very likely happen Friday.

My organization is strict on CVEs even if there is no proven impact but thanks for looking and confirming there more than likely isn't any. Also, thanks for the quick update!

@miller79 if you use pack build --buildpack paketo-buildpacks/bellsoft-liberica ... you should be able to get the patch for this now. That should clear up the CVE. Let me know if you see any other buildpacks as impacted and I'll try to push out updates for those as well.

I don't believe that the helper is actually impacted by this. You can see that we are only using crypto libs for sha256 and x509. My quick read of that issue is that it's related to public key crypto, which is not something that we're doing in the helper.
https://github.com/search?q=repo%3Apaketo-buildpacks%2Flibjvm crypto&type=code

I agree on that @dmikusa - as I said, probably was a false-positive.
But as @miller79 on our company we take public CVE issues really into account. We have some workaround processes, but we all are happier with upated dependencies and no workarounds.
Thanks for taking care. And now I see your changes, I will try to provide a PR update next time.


Missing types in Enum SlackMessageEventType

Can we get types added for channel_archive and channel_unarchive? Currently unable to parse these in SlackHistoryMessage.

@abdolence Hi I actually have it on a local branch. Needed it for a project and I pointed the project to my local. I can put in a PR tonight. Is forking okay?


update to chocolatey 0.10.4

I've had the impression choco 0.10.4 is just around the corner - maybe we can add the new version to Chocolatey GUI 0.15 ?

If it's hitting in the next two weeks, makes perfect sense to me.


Entity to entity mapping table for root cause analysis

This PR defines the database bean which will hold mappings between entities, such as metric to metric, metric to service, dimension to dimension. This will help us define static mappings. The endpoints will allow users add/edit mappings. This mapping will be used by the various pipelines such as DimensionRewritePipeline, MetricToServicePipeline

Fixed all suggestions


Error on installing Tobs on GKE 1.24

What did you do?
helm install otel timescale/tobs -n tobs-otel --create-namespace --wait --timeout 25m
Did you expect to see some different?
Environment

tobs version:
14.6.0

Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.3", GitCommit:"aef86a93758dc3cb2c658dd9657ab4ad4afc21cb", GitTreeState:"clean", BuildDate:"2022-07-13T14:30:46Z", GoVersion:"go1.18.3", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.3-gke.2100", GitCommit:"25d7334511e90d0b636707059c955baebce769cd", GitTreeState:"clean", BuildDate:"2022-08-16T09:24:54Z", GoVersion:"go1.18.3b7", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes cluster kind: gke

Anything else we need to know?:

Hey, with some digging around, it looks like a bug in kubernetes (kubernetes/kubernetes#67761). There is also a issue on the helm side for this (helm/helm#9710), and an open PR fixing this as well, but it is not merged (helm/helm#9713).
Can you pleas add your helm CLI version to the issue as well? That'll help me reproduce this faster.

Hi @onprem thanks for reply
helm version
version.BuildInfo{Version:"v3.9.1", GitCommit:"a7c043acb5ff905c261cfdc923a35776ba5e66e4", GitTreeState:"clean", GoVersion:"go1.17.5"}

same happend on helm version
version.BuildInfo{Version:"v3.9.4", GitCommit:"dbc6d8e20fe1d58d50e6ed30f09a04a77e4c68db", GitTreeState:"clean", GoVersion:"go1.17.13"}

Going through the upstream issues, it looks like using helm with resource quotas enabled and a big helm chart is hit or miss. The problem occurs when helm tries to create a lot of resources (in tobs' case, it can create a lot of stuff as it bundles OTel and Kube Prometheus along with other projects) in a short amount of time. Every pod or service creation triggers an update in the remaining quota part of the ResourceQuota object and can lead to conflicts.
Currently helm does not have the retry patch merged and looks like the PR is abandoned as well due to lack of reviews over a long time.
The workaround I'd suggest is to incrementally roll out tobs.
Start with most of the components disabled. For example everything disabled apart from TimescaleDB, Promscale, and kube-prometheus (you can even disable some kube-prometheus parts as well, for example Grafana). Then update your helm release with more components enabled (like open telemetry). Even with this you might encounter the same error, but retrying the operation until it succeeds is the only workaround for now.

ohk got it  , using older version for helm would work ?

I don't think using an older version of helm would work. But if you are willing to, removing the resource quotas will do the trick here.


Prefer pip installation. Close #190

Is this what you were thinking about for issue 190?

Looks great @purplediane, thanks!


Bosh should (optionally?) warn about unused properties

The bosh CLI should print (during deploy or when explicitly asked) the list of unused properties defined in the current manifest.
Specifically:

if the value of a global property GP is not used by any job during erb rendering, bosh should emit a warning saying global property GP is unused (this should apply also if the value of GP is always overridden by job properties)
if the value of a job property JP is not used during erb rendering on the job J where it's defined, bosh should emit a warning saying job property JP is unused by job J

+100. I'd love to see this plus an option (--strict ?) that would cause a failure if unused properties exist so it could be used to lint manifests


Minor formatting proposals

Description
Some fix on formatting style on certain portions of this paragraph, intended for the sake of clarification
Fixes # (issue)
There is no previous issue related with this PR
Type of change
Please check all relevant options.

[ ] Bug fix (non-breaking)
[ ] New feature (non-breaking)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Testing
Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

[ ] Test A
[ ] Test B

Test Configuration:

MacOSX
Python version 3.6.8
ART version or commit number (not assignable)
TensorFlow / Keras / PyTorch / MXNet version (not assignable)

Checklist

[x] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[ ] I have commented my code
[X] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Missing Developer Certificate of Origin, closing for now


[Shattered relic xp] Update README.md with info & images

Additional information and images
Hope you don't mind me using your plugins to get more used to GitHub and writing documentation.

Sorry, not entirely sure what I did there.


Counting elements of base object

I am trying to count the elements in an array that is at the root of an API response. Might look like this:
[
  {"name": "john"},
  {"name": "clarice"}
]

The test I wrote uses the following test/comparer:
- test:
    - name: "Get all people"
    - url: "/api/people"
    - method: 'GET'
    - expected_status: [200]
    - validators:
        - compare: {jsonpath_mini: " ??? ", comparator: "ge", expected: 2}

I tried pretty much all variations of length(.), length("."), length("") etc. pp.
What am I doing wrong here?

I made a mistake. I was using the wrong comparer! Comparer mus be jmespath and the length of the main array can be retrieved like to length('[]').
So complete test definition must be:
- test:
    - name: "Get all people"
    - url: "/api/people"
    - method: 'GET'
    - expected_status: [200]
    - validators:
        - compare: {**jmespath : "length('[]')"**, comparator: "ge", expected: 2}


Adding warnings for the deletion commands

I also updated a few places that said Azure Lab Services instead of Azure DevTest Labs

@rosehjm
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Thank you @lalexedwards !

#sign-off


Nice formatting

Thank you very much for putting repo together. I am very novice. I forked your repo to try it out, but I cannot get the nice formatting that you use. I really like to use the minima jekyll theme but I haven't been able to get your slideshow and minima to work together. Any help you can offer me would be helpful!
https://steenblikrs.github.io/test/
https://github.com/steenblikrs/jekyll-slideshow
Thank you very much!
Steenblikrs

hello @steenblikrs
i do not maintain this anymore
i use unslider these days: https://github.com/hudsonperalta/Unslider
it's not really maintained either but it is easy to add to a jekyll site...


Uncaught SyntaxError: Unexpected token {

With Chromium 71, trying to add support for react-native-dom to a project I get the error Uncaught SyntaxError: Unexpected token { in file bootstrap.bundle. Looking at the line, the code is provided by react-native-dom itself, seems the braced import statement is not supported by Chromium... Any clue about what could be happening and how can I be able to fix it?
  import { RNDomInstance } from "react-native-dom";

This could be addressed by transpiling to ES5 using Webpack or Browserify.  Projects consuming this may need to turn on the switch to transpile source under node_modules.  That could slow down the build though.
Alternatively, react-native-dom could publish a minified & transpiled build.

Projects consuming this may need to turn on the switch to transpile source under node_modules.

What's that switch? I'm having this problem in another projects.

Alternatively, react-native-dom could publish a minified & transpiled build.

Yeah, that would be the ideal thing...

Here's a walk-through for Webpack.  There are tutorials out there for Bable and Browserify too.
https://medium.com/unhandled-exception/using-non-transpiled-modules-and-webpack-657dca27bd4a

Thank you! :-D


RKE version clarification

URL: https://rancher.com/docs/rke/latest/en/upgrades/
Issue: The description about RKE version on this page needs to be clarified. There are 2 parts on this page mention that "RKE v1.x should be used". However, there are another 6 parts mentioned about v0.2.x and  another 2 parts mention about v0.3.x. If we do assume that all RKE users are using stable or the latest version of RKE, and this upgrade doc is based on RKE v1.x, then why should we bother versions prior to v1.x here? It does not make sense at all.
Questions:

Is this document based on RKE v1.x?
Assuming this doc is based on v1.x, then what are the purposes of mentioning v0.2.x and v0.3.x here?
Are we missing a 'Release Notes' section for RKE?

Suggestion:

Create a  'Release Notes' section for RKE, fill it up with  'Release Notes' listed on Github.
Remove confusing descriptions of RKE versions on related documentations.
Inform users we write this document based on a specific version( i.e. v1.x) of RKE, prompt users to read 'Release Notes' and to upgrade RKE versions. Then we wouldn't need to specify RKE versions on each page.

clear


Global black and white list mode

Can the current pseudo black and white list mode add a global black and white list mode that is judged by the URL. For example, I have 20 common websites that need to be set globally for only these 20 websites to enable extensions. Other websites do not start extensions. Real black-and-white list mode Now every infrequently used website has to be closed by one click. It’s really troublesome. You can add such a black-and-white list.

Not a time-based black and white list, but a global full time URL-based black and white list mode

Not a time-based black and white list, but a global full time URL-based black and white list mode

If you have a list of 20 websites where you want to use Midnight Lizard and disable it for all other websites then the easies way is to use match patterns:

Select Default color scheme

Open Activation tab

Uncheck Run on this website

Add Include match patterns for all websites you want to Use Midnight Lizard on

Press Save (or Set as default if you don't have Save button)

This behavior is described in the hint over the Include match pattern field:

If you have a list of 20 websites where you want to use Midnight Lizard and disable it for all other websites then the easies way is to use match patterns:

Select Default color scheme

Open Activation tab

Uncheck Run on this website

Add Include match patterns for all websites you want to Use Midnight Lizard on

Press Save (or Set as default if you don't have Save button)

This behavior is described in the hint over the Include match pattern field:

This is exactly the kind of feature I was looking for. But for some reason the Match boxes are disabled (I can't edit the text inside of them). Is this a bug?
I'm using the latest version of Firefox, and would really love this feature.


sap_swpm: wrong defaults or check

The role fails with the following error

TASK [community.sap_install.sap_swpm : SAP SWPM Pre Install - Check if installation using SAP Maintenance Planner] ***************************************************************************
fatal: [nodea.lab.example.com]: FAILED! =>
  msg: |-
    The conditional check 'sap_swpm_mp_stack_path | length > 0' failed. The error was: Unexpected templating type error occurred on ({% if sap_swpm_mp_stack_path | length > 0 %} True {% else %} False {% endif %}): object of type 'NoneType' has no len()

    The error appears to be in '/home/student/.ansible/collections/ansible_collections/community/sap_install/roles/sap_swpm/tasks/pre_install/install_type.yml': line 37, column 3, but may
    be elsewhere in the file depending on the exact syntax problem.

    The offending line appears to be:

    - name: SAP SWPM Pre Install - Check if installation using SAP Maintenance Planner
      ^ here

The corresponding variables in the default file are defined as:
# MP Stack
sap_swpm_mp_stack_path:
sap_swpm_mp_stack_file_name:

The probe is wrong, as the variable is defined and NONE, hence the length cannot be determined.
This could be a possible probe:
(sap_swpm_mp_stack_path is none) or (sap_swpm_mp_stack_path| trim = '')

In pre_install.yml we need then:
./pre_install.yml:    sap_swpm_swpm_command_extra_args: "SAPINST_SKIP_DIALOGS=true SAPINST_START_GUISERVER=false {{ sap_swpm_swpm_command_virtual_hostname }} {{ sap_swpm_swpm_command_mp_stack|d('') }}"

Accidental commit to this dev branch and not my own, fix already in place in dev branch, closing issue


Remove ensemble split and indices in AutoMLSearch

Closes #2093

Hm. Most recent attempt passed, but noting these two random Windows failures:
https://github.com/alteryx/evalml/pull/2260/checks?check_run_id=2572337724
https://github.com/alteryx/evalml/pull/2260/checks?check_run_id=2576148761

@freddyaboulton Yes, I definitely agree with you that even here, we're not doing a fair apples-to-apples comparison. I've talked to @dsherry about this before, and he mentioned that he and @kmax12 have had some discussions about a separate "model-selection" split where we hold out some data that is then used to validate the model and determine the actual ranking of the models as they should appear on the leaderboard, rather than using the training cv score we're currently relying on.
I know work is being done for the automl algo right now, not sure if this would step on those toes but I've filed #2284 to track this, feel free to add more there! :D


DRAFT ONLY, DO NOT MERGE: Demo of what it would look like to rebase covers onto master

Steps:

git checkout covers
git rebase master
Apply the last 2 commits that start with "Covers files"

This demo worked, but there have been a handful of new changes in the last week. I'm going to close the PR and open a new clean one.


Toast designs

Toast designs
Can we move away from the current Toast design?
In Design system

Legacy design shown at bottom
Originally posted by @xraystyle1980 in https://github.com/decentdao/fractal-interface/issues/1578#issuecomment-2101329577

Yeah I asked for a easy redesign for the first pass of reskin. But these definitely can be focused in on.

I think this needs a lot more thought before we can implement. We currently don't have any "levels" applied to our toasts. They're all the same. No difference between success and error.
In the spirit of keeping this reskin just to a reskin, and completing this delivery in time, I'm comfortable with redesigning our current catch-all toast. But I think it's out of scope to implement multiple designs and then go through the app applying deciding which flavor to give each toast. That should go into the backlog.
@xraystyle1980 @tomstuart123 thoughts?

agreed.
@xraystyle1980 lets assume we'll pick up this 'personalised toaster scheme' as an issue further down the line.
Does the current 'generic toaster' still need an update in your eyes ? Or is the current fine for now

Still think this should stay deprioritised until more progress on hats / whitelisting proposals. Soon after though so I'll move it to a P1 but still in the backlog

Design required
I would say now we have hats progress, this COULD (but lets see about SHOULD) be an option to be picked up before design moves onton whitelisting


gnu-cobol: New package.

Use mingw version of cobol instead.
https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-gnu-cobol-svn


Ways to have log verbosity

Hi, I couldn't find a way to control the log verbosity. Looks like it's using INFO by default and I'd like to have WARN as the base level but also the option to have ERROR or DEBUG would be awesome.
Is this in the roadmap or if it does exist where can I find its documentation?

@danlsgiga Why do you want to dial it down? There is no DEBUG log and fabio shouldn't log that much. If it does then this usually points to something in your setup: a flapping health check or a check with volatile output like a timestamp for example. You'd want the INFO level since it logs how fabio is started including the runtime configuration.

Hey @magiconair, thanks for the quick follow up.
So, I'm getting lots of log entries with:
2017/09/06 14:45:07 [INFO] consul: Health changed to #10372843
I understand fabio should not be logging that much but it is and as you mentioned, how can I understand the above log message without more details? These details would usually be provided by a lower log level like DEBUG or in the worst cases even TRACE. And the option to be able to silence those in case I need for could be available for greater control.
IMHO having the runtime configuration dumped into the logs are not great as well. A better approach imo would be to provide an authenticated api endpoint where we could grab that. In our case, since we have central logging, the runtime configuration is readable by a lot of people and that include IP addresses and even consul tokens. If that's not going to change, I think obfuscating those sensitive data should be something to think about.

Hi @danlsgiga,
The message should probably read consul: Raft index changed to #10372843 which explains its true nature. The reason I'm logging those as INFO is that a high rate of these messages points a problem in your consul setup because you are generating lots of writes on the consul master. This can either be caused by flapping health checks or health checks which have volatile output like a timestamp. In any case it points to an issue within your setup which should be fixed. Providing you with a switch to hide this makes it more likely to just tune it out.
At eBay we had exactly that issue where my team had a stable log which wouldn't grow and the other team was asking for log rotation since their log grew somewhat unbounded. Ultimately, they found the issue in their consul integration framework which was causing excessive writes on consul. They fixed it and it was stable ever since.
Since the root cause for lots of these log messages is in almost all cases volatile output in the health check and the fix is usually trivial I'm still in favor of saying "please fix your setup".
However, I've learned that there are other log messages that do not provide much value but can appear a lot in the output. I'll have a look at those on whether we really need them. I might add a debug log level which can be enabled and disabled but I'd still leave the above messages at INFO since they point to an issue in your cluster setup that needs fixing.
As for the runtime config: I agree that the consul tokens should be obfuscated and I'll add that since it is a good idea. However, I'd argue that the API endpoint for the config should exist in addition to adding the runtime config to the log since that provides you with historical information. So you can go back in time to see whether the config has been changed which you can't do with just an API endpoint. I'm curious which IP addresses you'd like to obfuscate since I don't consider them to be sensitive but I'm open to arguments.

Hi @magiconair, thanks for the great insight! I'll surely look into my consul environment and check what is causing so much flaping on the health checks.
Really appreciated for the effort on making fabio even more awesome!
So, if the runtime config is being dumped into the logs and this is being centralized in our environment... having the consul IP address and the token there is pretty much giving access to your whole consul infra. Hence my request to have log verbosity since it is the most common way of controlling what goes or not to the logs.
Having the default logging at INFO is totally fine and I encourage it actually, but at the same time having the ability to choose the log level would be great.

If I obfuscate the token the consul ip becomes somewhat useless, doesn't it?

IMHO, not in the scenario where someone uses consul in a acl_default_policy = "allow" setup.
My position towards this is that dumping any configuration (even the ones with no secrets) should be seen as bad practice and discouraged. Configuration should never be dumped into logs for the same reasons StackTraces should never be exposed to users. Simply because it may or may not contain sensitive data.

I've just merged #366 which allows you to control the log level.


refactor(controllers): separate controllers into different files

Separate each controller/handler into its own file
Sort all of the controllers/handlers into categories according to the documentation

(Fixing the CI)


single cursor solution with structured contacts

single cursor solution with structured contacts

Just tried it and it works, and looks good to me. @rt2zz hope you merge it.

works man, thanks for this


Implement scoped tasks

This PR implements "scoped tasks", similar to scoped threads in libstd and crossbeam. It provides a safe abstraction for borrowing variables from the local stack in tasks. The goal is to enable scoped tasks in blocking.
For the time being, this relies on async-channel, which uses libstd for the time being. However, once we move it to no_std, this should not be an issue.
This PR probably needs a review for soundness. MIRI doesn't raise any red flags but it's 100% possible that I missed something. Just saw something I missed, let me take care of that.

I have simply thought about the soundness of "scoped-task", there are four kinds of condition just similar as "scoped-threads":

task is awaited in its scope, await would own the ownership of task and drop it after it is complete, so it must be safe;
task is not used in scope, because the lifetime of task is captured in scope, it would be dropped and set to cancel when it leaves this scope, once it is canceled, executor could not run it never more, so I think it is safe;
task is set to detach, this should be unsafe because Task::detach requires task and all captured variables 'static, therefore "scoped-task" must not be detached.
task moves out of the scope, it is impossible because task captures the lifetime of its scope;

task is set to detach, this should be unsafe because Task::detach requires task and all captured variables 'static, therefore "scoped-task" must not be detached.

If we keep a handle to the task stored in the Scope structure (as I do in this PR) this doesn't turn out to be an issue. Task is just a frontend to the inner structure that the handle captures.

The real issue is the fact that the outer scope future can be dropped at will. Consider this sequence of events:

User calls scope() and uses it to spawn a handful of scoped tasks and capture variables from outside of the scope.
User polls scope until it's done with the closure provided the user, and starts waiting for the remaining tasks to finish.
User stops polling scope and drops the scope future.
Since scope is dropped, user now has access to all of the variables that scope borrowed, even if there may still be tasks that simultaneously have access to these variables.
This is undefined behavior.

The only real option would be to have some kind of guard inside of the scope() function that aborts the program if there are any tasks left over when the user drops the future. We can't panic in this state, since a panic can be recovered from and the stacked borrowed rules are being flagrantly violated here, so the only option would be aborting. I'd like to avoid this if at all possible. Aborts are reserved for situations when the program is truly broken, like an overflowed reference counter or an operation that leaves memory in an invalid state. Futures are meant to be dropped often, and aborts are hard to debug. Imagine if this was in an enterprise codebase, running inside of another task that was accidentally cancelled, and it caused an abort that brought down the entire system.
I'd love to be proven wrong here, but I think that the only way to have a sound implementation of scoped tasks would be to have it be a block_on() function or variant thereof. While futures can be dropped at any point, a function execution cannot, which would ensure that the issue of interrupting the wait for the tasks to close could never happen (without the presence of other unsafe code, at least). I guess it would look like this:
pub fn block_on_scoped<'env>(f: impl FnOnce(&Scope<'env>) -> impl Future) {
    ...
}

let mut i = 1;
block_on_scoped(|scope| {
    let i = &mut i;
    let (runnable, task) = scope.spawn(async move { *i += 1; });

    async move {
        ...
    }
});

However, that would involve quite a few caveats. For instance, for large programs, you would have to pass the Scope across many function boundaries and threads. This would be very annoying. In addition, we'd probably have to parameterize it across multiple blocking strategies, which would require a substantial amount of effort to design an API for.
All in all, it's probably a job best suited for a crate outside of this one, and would require a careful hand to ensure that everything is sound.

Yes I think your are right, considering the Scope implementation of std::threads::Scoped, it always awaited all threads completed, I think scoped-task is might able to use the similar behavior: it should block the current thread until all scoped-task is finished.
As you said, maybe it is not the best way to embed this feature into async-task and blocking crates, however, it could be implemented without unchecked methods. blocking::spawn_unchecked might be helpful to make "scoped-blocking" above blocking crate.


New PR to test SignCLA at 2017-11-08 11:59:12.03528353 -0800 PST m=+79.617057018

Body of PR

corp-pass-staging: test succeeded


Increase coverage

Description

[x] Remove requirements files and put everything under setup.py
[x] Add factory-boy
[x] Some tests for RequestMaker

Closes #70
Related with #14

Codecov Report

Merging #71 into master will increase coverage by 1.97%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #71      +/-   ##
==========================================
+ Coverage   85.59%   87.57%   +1.97%
==========================================
  Files          14       14
  Lines         354      354
==========================================
+ Hits          303      310       +7
+ Misses         51       44       -7

Impacted Files
Coverage Δ

scanapi/errors.py
91.66% <0%> (+8.33%)
:arrow_up:

scanapi/requests_maker.py
100% <0%> (+19.23%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0201d60...fa41861. Read the comment docs.


Mysterious server crashes

There is a mysterious error that will crash the server from time to time. I have not been able to reproduce it reliably. it doesnt seem to be linked to specific summoners. (But i will keep investigating)
parsing http://na.op.gg/summoner/ajax/update.json/?summonerId=35488173
undefined:1
undefined
^
SyntaxError: Unexpected token u
at Object.parse (native)
at Request.parseSummonerRefresh [as _callback] (C:\op.gg-api-master\server.j
s:320:20)
at self.callback (C:\op.gg-api-master\node_modules\request\request.js:237:22
)
at Request.emit (events.js:107:17)
at ClientRequest.self.clientErrorHandler (C:\op.gg-api-master\node_modules\r
equest\request.js:342:10)
at ClientRequest.emit (events.js:107:17)
at Socket.socketOnData (_http_client.js:322:9)
at Socket.emit (events.js:107:17)
at readableAddChunk (_stream_readable.js:163:16)
at Socket.Readable.push (_stream_readable.js:126:10)
at TCP.onread (net.js:538:20)
C:\op.gg-api-master>

happened again - could it be that im calling the endpoint too often/too fast?
parsing http://euw.op.gg/summoner/userName=ImpExNaminitor
done
parsing http://na.op.gg/summoner/ajax/update.json/?summonerId=19005629
done
parsing http://euw.op.gg/summoner/userName=BEVBUSDRIVER
done
parsing http://na.op.gg/summoner/ajax/update.json/?summonerId=25266744
undefined:1
undefined
^
SyntaxError: Unexpected token u
at Object.parse (native)
at Request.parseSummonerRefresh [as _callback] (C:\op.gg-api-master\server.j
s:320:20)
at self.callback (C:\op.gg-api-master\node_modules\request\request.js:237:22
)
at Request.emit (events.js:107:17)
at ClientRequest.self.clientErrorHandler (C:\op.gg-api-master\node_modules\r
equest\request.js:342:10)
at ClientRequest.emit (events.js:107:17)
at Socket.socketOnData (_http_client.js:322:9)
at Socket.emit (events.js:107:17)
at readableAddChunk (_stream_readable.js:163:16)
at Socket.Readable.push (_stream_readable.js:126:10)
at TCP.onread (net.js:538:20)
C:\op.gg-api-master>

it seems to work fine for an undetermined amount of time until this happens.

the code that is called every minute (scheduled task)
prepare($sql);
$stmt->execute();

$players = $stmt->fetchAll(PDO::FETCH_ASSOC);
foreach ($players as $player) {

	$summ = $string = preg_replace('/\s+/', '', $player['player']);;

    $url = 'http://localhost:1337/euw/summoner/'.urlencode($summ);

	$refreshurl = 'http://localhost:1337/euw/refresh/'.$player['lkid'];

	// $url = 'http://localhost:1337/euw/summoner/B%C3%AAluga';

	echo $url.' - '.$refreshurl.'';

	file_get_contents($refreshurl);
	$response = file_get_contents($url);
	$data = json_decode($response);

	echo $player['player'].'';

	// determine which team our player is on
	$teamidentity = $player['teamidentity'];

	// touch player to update the timestamp - even if we dont actually update anything
	$sql = "UPDATE players SET last_history_update = CURRENT_TIMESTAMP WHERE player = :player";
	$stmt = $db->prepare($sql);
	$stmt->execute(array(':player' => $player['player']));

	$games = $data->data[0]->games;

	foreach($games as $game){

		// only inspect custom (= tournament matches)
		if($game->type == 'Custom'){

			echo 'custom found ';
			// check if game is already in db
			$match_identity = $game->gameId;
			$date_time = $game->datetime;
			echo $match_identity.'';
			$inner_sql = "SELECT match_identity from tournament_matches WHERE match_identity = '$match_identity'";
			$inner_stmt = $db->prepare($inner_sql);
			$inner_stmt->execute();
			if($inner_stmt->rowCount() > 0){
				echo 'match already in db ';
			} else {
				echo 'adding match '.$match_identity.'';
				// loop over the opposing team to try and find out if any of the opponents was swiss
				echo 'opposing: ';

				if($game->teamNum == 1){
					$players = (array)$game->team2;
				} else {
					$players = (array)$game->team1;
				}

				$opposing  = 0;

				$teamsize = 0;

				foreach($players as $player){
					$teamsql = "SELECT teamidentity FROM players WHERE player = :player";
					// echo 'query: '.$teamsql.'';
					$teamstmt = $db->prepare($teamsql);

					$teamstmt->execute(array(':player' => $player->name));
					$teams = $teamstmt->fetchAll(PDO::FETCH_ASSOC);
					foreach($teams as $team){
						$opposing = $team['teamidentity'];
					}
					$teamsize++;
					echo $player->name.' - Team: '.$opposing.'';
				}
				echo $teamsize.' Players';

				// determine winning team
				$result = $game->result;

				$winner_teamside = 1;

				if($result == 'Defeat'){
					$loser = $teamidentity;
					$winner = $opposing;
					if($game->teamNum == 1){
						$winner_teamside = 2;
					}

				} else {
					$loser = $opposing;
					$winner = $teamidentity;
					if($game->teamNum == 2){
						$winner_teamside = 2;
					}
				}
				echo 'winner: '.$winner.'';
				echo 'loser: '.$loser.'';

				$json = json_encode($game);
				// echo $json;
				try {
					$sql = "INSERT INTO tournament_matches(match_identity, json, winner, loser, datetime, winner_teamside) VALUES (:match_identity, :json, :winner, :loser, :datetime, :winner_teamside)";
					$insert = $db->prepare($sql);
					if($winner == $loser) echo 'winner equals loser. ignoring match. ';
					if($teamsize == 5 && $winner != $loser && $loser != 0 && $winner != 0){
						$insert->execute(
						array(
						':match_identity' => $match_identity,
						':json' => $json,
						':winner' => $winner,
						':loser' => $loser,
						':datetime' => $date_time,
						':winner_teamside' => $winner_teamside
						));
					}

				} catch(PDOException $ex) {
					echo 'Error: '.$ex->getMessage();
				}
			}

		}
	}
}
$time_end = microtime(true);

//dividing with 60 will give the execution time in minutes other wise seconds
$execution_time = ($time_end - $time_start);

//execution time of the script
echo ''.$limit.' Queries. Total Execution Time: '.$execution_time.' Secs. Timer per query ='.($execution_time/$limit).' Secs';

?>

I forgot to error check the refresh endpoint. I think it may be that you're making too many queries called too fast. Then again, they don't secure their json endpoints, so I wouldn't be surprised if they didn't throttle page queries.

problem seems fixed for refreshes - but also happens for summoner requests -
parsing http://na.op.gg/summoner/ajax/update.json/?summonerId=19036982
done
parsing http://euw.op.gg/summoner/userName=ERQAractosa
C:\op.gg-api-master\node_modules\cheerio\lib\parse.js:62
var oldParent = node.parent || node.root,
^
TypeError: Cannot read property 'parent' of undefined
at exports.update (C:\op.gg-api-master\node_modules\cheerio\lib\parse.js:62:
25)
at module.exports (C:\op.gg-api-master\node_modules\cheerio\lib\parse.js:24:
3)
at Function.exports.load (C:\op.gg-api-master\node_modules\cheerio\lib\stati
c.js:19:14)
at Request.parseSummoner [as _callback] (C:\op.gg-api-master\server.js:177:2
0)
at self.callback (C:\op.gg-api-master\node_modules\request\request.js:237:22
)
at Request.emit (events.js:107:17)
at ClientRequest.self.clientErrorHandler (C:\op.gg-api-master\node_modules\r
equest\request.js:342:10)
at ClientRequest.emit (events.js:107:17)
at Socket.socketOnData (_http_client.js:322:9)
at Socket.emit (events.js:107:17)

another one:
parsing http://na.op.gg/summoner/ajax/update.json/?summonerId=23920549
undefined:1

^
SyntaxError: Unexpected token <
at Object.parse (native)
at Request.parseSummonerRefresh [as _callback] (C:\op.gg-api-master\server.j
s:322:20)
at Request.self.callback (C:\op.gg-api-master\node_modules\request\request.j
s:237:22)
at Request.emit (events.js:110:17)
at Request. (C:\op.gg-api-master\node_modules\request\request.js:
1146:14)
at Request.emit (events.js:129:20)
at IncomingMessage. (C:\op.gg-api-master\node_modules\request\req
uest.js:1097:12)
at IncomingMessage.emit (events.js:129:20)
at _stream_readable.js:908:16
at process._tickCallback (node.js:355:11)
C:\op.gg-api-master>node server.js


RECYCLE BIN shows no items if a playthrough is not selected

Clicking on RECYCLE BIN before selecting a playthrough shows no items in the RECYCLE BIN. After selecting a playthrough, then the RECYCLE BIN displays the items marked for deletion properly

resolved with: 72f501f


Sending visibility event to the website

Let's discuss this in a call; I think this turned out to be way more complex and than it needed and not sure it's worth it.


Remove deprecated ArithmeticOperation

@MichaelBroughton

Oof, that looks nontrivial (at least for me -- I'd need to ramp up on how to work with ipynb files, which I don't have time for right now). Anybody who wants to fork from here, I know there's a shor.py that probably had the same change when I originally deprecated ArithmeticOperation: https://github.com/quantumlib/Cirq/pull/4702/files#diff-9d5e940cf487ce1f28d94c8a32b7c4c2a5da812e72b48aa6e5db733e3f511066

@MichaelBroughton I figured out how to do this. However I'm getting an error saying cirq.ArithmeticGate is not defined. But it is. And it works locally. Any idea?

However I'm getting an error saying cirq.ArithmeticGate is not defined. But it is. And it works locally. Any idea?

I am new to the notebook test machinery, but the test may be running with a released cirq.
Perhaps the shor.ipynb notebook needs to be added to NOTEBOOKS_DEPENDING_ON_UNRELEASED_FEATURES -
https://github.com/quantumlib/Cirq/blob/df99587d648e70206e5860efdbbe73a4182062f7/dev_tools/notebooks/isolated_notebook_test.py#L43-L59

@MichaelBroughton @pavoljuhas It turns out this is not ready to be removed. https://github.com/quantumlib/Cirq/pull/4702 was started in November but did not get merged until May, which was after 14.1 went out. These deprecations need to be bumped to 0.16 instead.

v0.15 is out, so reopening.


Linter result gets attached to the wrong workflow

I have 2 workflows running concurrently, build-and-test one and a linter one (using Apple's swift-format). The linter result (named swift-format) is being attached either to the linter workflow or to the build-and-test workflow,  seemingly randomly.
This is how I expect it's supposed to be where the swift-format result is attached to my linter workflow job list.

And this linter run does not have the result attached:

But instead it ends up in the build-and-test workflow:

Can you think of why this might be happening?

Hi there, this is an issue with how the GitHub checks API works.  This is nothing this action can fix.
Duplicate of #577, #401, #280, #97, #93.


Add option to use --tls-no-verify

Problem description
It would be nice to have the option to ignore the SSL certificates if they are not trusted.

Yep, its on the board! I think this would also fit with a big improvement on all of the authentication issues.

I would also like this feature

I am making a separate PR for global configuration where we can add the config options for this.


Expose data from rule component

@Bplotka
I'm writing an e2e test for the rule<>query interaction now but this code here is good for review already.

PTAL


provide release file

Hi, I recently found your Quick HDR repository and would really like to try it out. Unfortunately, I don't own Visual Studio and also don't have any experience with it, so I'm not able to build the app. Could you please provide a release file for this project? This would be much appreciated, thank you!
Best regards

Yes, please do this...


[WIP] Consolidate helpers for constructing Sierra[Bib|Item]Record in tests

Several of the fields on these we don’t really care about exact values – e.g. id, modifiedDate.
By creating some new createRecordWith helpers, we can focus on the values we actually care about, and cut down the noise. We already had lots of helpers – but repeated across the codebase, not consolidated in one place! So we can save ourselves some trouble there.
Additionally, the classes had several apply methods that only existed for tests. We can remove those now, and simplify those models. Hooray!

Replaced by #2442 and #2443.


Flutter Web: JavaScript history.pushState raises an exception in Release Build.

Steps to Reproduce

Clone repo: https://github.com/gladimdim/push_history_bug.

Switch to Flutter beta channel & enable web

flutter channel beta
flutter upgrade
flutter doctor
flutter config --enable-web

Run the debug mode:

flutter run -d chrome

It will show an app with the button. Clicking on the button will add 'google.com' at the end of the URL in address bar. js.history.pushState works OK in debug mode.

Run the release build :

flutter run --release -d chrome

Press the red button multiple times.

Expected results: the url is updated via js.history.pushState bindings from dart:js. Each button click adds 'google.com' at the end of the URL in the address bar.
Actual results:: the url is not updated. Exception is thrown in the Chrome console: Another exception was thrown: Instance of 'minified:ar'

Logs

flutter run --verbose -d chrome --release
[  +28 ms] executing: [/Users/i501633/progs/flutter/] git -c log.showSignature=false log -n 1 --pretty=format:%H
[  +48 ms] Exit code 0 from: git -c log.showSignature=false log -n 1 --pretty=format:%H
[        ] fabeb2a16f1d008ab8230f450c49141d35669798
[        ] executing: [/Users/i501633/progs/flutter/] git describe --match v*.*.* --first-parent --long --tags
[  +39 ms] Exit code 0 from: git describe --match v*.*.* --first-parent --long --tags
[        ] v1.14.6-0-gfabeb2a16
[   +7 ms] executing: [/Users/i501633/progs/flutter/] git rev-parse --abbrev-ref --symbolic @{u}
[  +10 ms] Exit code 0 from: git rev-parse --abbrev-ref --symbolic @{u}
[        ] origin/beta
[        ] executing: [/Users/i501633/progs/flutter/] git ls-remote --get-url origin
[   +9 ms] Exit code 0 from: git ls-remote --get-url origin
[        ] https://github.com/flutter/flutter.git
[  +45 ms] executing: [/Users/i501633/progs/flutter/] git rev-parse --abbrev-ref HEAD
[  +11 ms] Exit code 0 from: git rev-parse --abbrev-ref HEAD
[        ] beta
[   +6 ms] executing: sw_vers -productName
[  +16 ms] Exit code 0 from: sw_vers -productName
[        ] Mac OS X
[        ] executing: sw_vers -productVersion
[  +16 ms] Exit code 0 from: sw_vers -productVersion
[        ] 10.15.3
[        ] executing: sw_vers -buildVersion
[  +16 ms] Exit code 0 from: sw_vers -buildVersion
[        ] 19D76
[  +33 ms] Artifact Instance of 'AndroidMavenArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'AndroidGenSnapshotArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'AndroidInternalBuildArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterWebSdk' is not required, skipping update.
[   +3 ms] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerDebugSymbols' is not required, skipping update.
[  +36 ms] executing: /Users/i501633/Library/Android/sdk/platform-tools/adb devices -l
[   +7 ms] Exit code 0 from: /Users/i501633/Library/Android/sdk/platform-tools/adb devices -l
[        ] List of devices attached
[  +15 ms] executing: /Users/i501633/progs/flutter/bin/cache/artifacts/libimobiledevice/idevice_id -h
[  +63 ms] executing: /usr/bin/xcode-select --print-path
[   +8 ms] Exit code 0 from: /usr/bin/xcode-select --print-path
[        ] /Applications/Xcode.app/Contents/Developer
[        ] executing: /usr/bin/xcodebuild -version
[ +595 ms] Exit code 0 from: /usr/bin/xcodebuild -version
[   +3 ms] Xcode 11.3.1
           Build version 11C504
[   +4 ms] /usr/bin/xcrun simctl list --json devices
[ +113 ms] Artifact Instance of 'AndroidMavenArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'AndroidGenSnapshotArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'AndroidInternalBuildArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IOSEngineArtifacts' is not required, skipping update.
[   +2 ms] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerDebugSymbols' is not required, skipping update.
[ +216 ms] Generating /Users/i501633/projects/history_push_error/android/app/src/main/java/io/flutter/plugins/GeneratedPluginRegistrant.java
[ +117 ms] Launching lib/main.dart on Chrome in release mode...
[        ] Building application for the web...
[ +185 ms] Generating /Users/i501633/projects/history_push_error/android/app/src/main/java/io/flutter/plugins/GeneratedPluginRegistrant.java
[   +4 ms] Compiling lib/main.dart for the Web...
[  +13 ms] Initializing file store
[  +10 ms] Done initializing file store
[  +72 ms] invalidated build due to missing files: /Users/i501633/projects/history_push_error/packages/flutter_tools/lib/src/build_system/targets/web.dart
[  +16 ms] web_entrypoint: Starting due to {InvalidatedReason.inputMissing}
[   +2 ms] web_entrypoint: Complete
[+1827 ms] dart2js: Starting due to {InvalidatedReason.inputChanged}
[+17424 ms] dart2js: Complete
[  +66 ms] web_release_bundle: Starting due to {InvalidatedReason.inputChanged}
[  +63 ms] web_release_bundle: Complete
[  +72 ms] web_service_worker: Starting due to {InvalidatedReason.inputChanged}
[  +31 ms] web_service_worker: Complete
[   +2 ms] Persisting file store
[   +5 ms] Done persisting file store
[  +15 ms] Compiling lib/main.dart for the Web... (completed in 19.6s)
[  +13 ms] Building application for the web... (completed in 19.8s)
[ +874 ms] Warning: Flutter's support for web development is not stable yet and hasn't
[        ] been thoroughly tested in production environments.
[        ] For more information see https://flutter.dev/web
[        ] 🔥  To hot restart changes while running, press "r". To hot restart (and refresh the browser), press "R".
[   +2 ms] For a more detailed help message, press "h". To quit, press "q".

flutter analyze
Analyzing history_push_error...

   info • Avoid using web-only libraries outside Flutter web plugin packages • lib/main.dart:3:1 • avoid_web_libraries_in_flutter

1 issue found. (ran in 8.2s)

flutter doctor -v
[✓] Flutter (Channel beta, v1.14.6, on Mac OS X 10.15.3 19D76, locale en)
    • Flutter version 1.14.6 at /Users/i501633/progs/flutter
    • Framework revision fabeb2a16f (9 days ago), 2020-01-28 07:56:51 -0800
    • Engine revision c4229bfbba
    • Dart version 2.8.0 (build 2.8.0-dev.5.0 fc3af737c7)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /Users/i501633/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 29.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.3.1, Build version 11C504
    • CocoaPods version 1.8.4

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 3.5)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 43.0.1
    • Dart plugin version 191.8593
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] VS Code (version 1.41.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.8.0

[✓] Connected device (2 available)
    • Chrome     • chrome     • web-javascript • Google Chrome 80.0.3987.87
    • Web Server • web-server • web-javascript • Flutter Tools

• No issues found!

Can you please run the app in --profile mode instead of --release, and provide the exception stacktrace?

@yjbanov ,
I ran
flutter run --profile -d chrome

here is a console log from Chrome dev tools:
Uncaught TypeError: Cannot read property '_match' of undefined
at Object.StackFrame_fromStackTraceLine (stack_frame.dart:103)
at Object.FlutterError_defaultStackFilter (assertions.dart:681)
at Object.DiagnosticsStackTrace$ (assertions.dart:780)
at FlutterErrorDetailsForPointerEventDispatcher.debugFillProperties$1 (assertions.dart:448)
at _FlutterErrorDetailsNode.get$builder (diagnostics.dart:2900)
at _FlutterErrorDetailsNode.get$builder (assertions.dart:818)
at _FlutterErrorDetailsNode.getProperties$0 (diagnostics.dart:2914)
at TextTreeRenderer.render$4$parentConfiguration$prefixLineOne$prefixOtherLines (diagnostics.dart:1224)
at TextTreeRenderer.render$1 (diagnostics.dart:1107)
at Object.FlutterError_dumpErrorToConsole (assertions.dart:648)

Same issue here
stack_frame.dart:103 Uncaught TypeError: Cannot read property '_match' of undefined
    at Object.StackFrame_fromStackTraceLine (stack_frame.dart:103)
    at Object.FlutterError_defaultStackFilter (assertions.dart:681)
    at Object.DiagnosticsStackTrace$ (assertions.dart:780)
    at FlutterErrorDetails.debugFillProperties$1 (assertions.dart:448)
    at _FlutterErrorDetailsNode.get$builder (diagnostics.dart:2900)
    at _FlutterErrorDetailsNode.get$builder (assertions.dart:818)
    at _FlutterErrorDetailsNode.getProperties$0 (diagnostics.dart:2914)
    at TextTreeRenderer.render$4$parentConfiguration$prefixLineOne$prefixOtherLines (diagnostics.dart:1224)
    at TextTreeRenderer.render$1 (diagnostics.dart:1107)
    at Object.FlutterError_dumpErrorToConsole (assertions.dart:648)

Could not load content for https://app.myguide.care/flutter/packages/flutter/lib/src/foundation/stack_frame.dart : HTTP status code: 404

Not sure why it's asking for /flutter/packages/flutter/lib/src/foundation/stack_frame.dart

This is a bug in our stack trace parsing logic. I merged a fix a few days back. Can you please try again on the latest master channel and provide the stack trace?

Hi, @yjbanov ,
I used latest master.
 flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel master, v1.15.4-pre.129, on Mac OS X 10.15.3 19D76, locale en)
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
[✓] Xcode - develop for iOS and macOS (Xcode 11.3.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 3.5)
[✓] VS Code (version 1.42.1)
[✓] Connected device (3 available)

• No issues found!

Running in profile mode
flutter run --profile -d chrome

Error stack trace in chrome console:
before push state: http://localhost:49850/#/
js_primitives.dart:49 ══╡ EXCEPTION CAUGHT BY GESTURE ╞═══════════════════════════════════════════════════════════════════
js_primitives.dart:49 The following DomException object was thrown while handling a gesture:
js_primitives.dart:49   DataCloneError: Failed to execute 'pushState' on 'History': function(object) {
js_primitives.dart:49       var testRti;
js_primitives.dart:49       if (object == null)
js_primitives.dart:49         return object;
js_primitives.dart:49       testRti = this;
js_primitives.dart:49   ...... } could not be cloned.
js_primitives.dart:49
js_primitives.dart:49 When the exception was thrown, this was the stack:
js_primitives.dart:49     at JsObject.callMethod$2 (http://localhost:49850/main.dart.js:39910:42)
js_primitives.dart:49     at _UrlChangerState_build_closure.call$0 (http://localhost:49850/main.dart.js:70201:31)
js_primitives.dart:49     at InkWell.onTap$0 (http://localhost:49850/main.dart.js:47716:25)
js_primitives.dart:49     at _InkResponseState._handleTap$1 (http://localhost:49850/main.dart.js:47949:21)
js_primitives.dart:49     at _InkResponseState_build_closure.call$0 (http://localhost:49850/main.dart.js:48066:25)
js_primitives.dart:49     at TapGestureRecognizer.invokeCallback$1$3$debugReport (http://localhost:49850/main.dart.js:45861:27)
js_primitives.dart:49     at TapGestureRecognizer.invokeCallback$2 (http://localhost:49850/main.dart.js:45871:19)
js_primitives.dart:49     at TapGestureRecognizer.handleTapUp$2$down$up (http://localhost:49850/main.dart.js:46213:18)
js_primitives.dart:49     at TapGestureRecognizer._checkUp$0 (http://localhost:49850/main.dart.js:46138:13)
js_primitives.dart:49     at TapGestureRecognizer.handlePrimaryPointer$1 (http://localhost:49850/main.dart.js:46087:15)
js_primitives.dart:49
js_primitives.dart:49 Handler: "onTap"
js_primitives.dart:49 Recognizer:
js_primitives.dart:49   #a6dcc
js_primitives.dart:49 ════════════════════════════════════════════════════════════════════════════════════════════════════

/cc @vsmenon @DaveShuckerow is this the RTI problem that you mentioned before? The offending code seems to be this:
       var href = js.context['location']['href'];
          print('before push state: $href');
          js.context['history']
              .callMethod('pushState', [{}, '', '$href/google.com']);
          print('after pushstate');
          setState(() {
            url = js.context['location']['href'];
          });

The error implies we're somehow trying to pass a function as an argument to pushState.

@rakudrama - do you think this is the same rti issue?

The code
   js.context['history']
      .callMethod('pushState', [{}, '', '$href/google.com']);

is passing a Dart map {} to JavaScript and the Dart Map has a type (Rti) which contains function which is not serializable by the HTML Standard StructuredSerializeInternal operation.
The old Rti would have been [null,null] which is by serializable by dumb luck. If you had used <String,dynamic>{}, or had any interesting data in the map, it would have failed in the old Rti.
The right thing to do is pass something you know is a JavaScript object.
You can use js_utils to convert the map to a JavaScript object:
   js.context['history']
      .callMethod('pushState', [js.jsify({}), '', '$href/google.com']);

or use package:js to declare an anonymous @JS class with a constructor, and use
@JS()
@anonymous
class EmptyObject {
  external EmptyObject();
}
...
   js.context['history']
      .callMethod('pushState', [EmptyObject(), '', '$href/google.com']);

Since some conversion is involved in serializing the 'state' parameter to pushState via the somewhat limited HTML Standard StructuredSerializeInternal operation, you might consider taking responsibility for the serialization, e.g. converting the state to a JSON string and parsing the JSON from the onpopstate handler. In this case you could pass the pre-converted JSON string since you know it is '{}'.
Taking responsibility for the conversion means you could change to a different state representation or serialization, e.g. protocol buffers, or some system of State objects with their own ad-hoc serialization.

/cc @sigmundch

Reproducible with the latest master channel.

flutter doctor -v
[✓] Flutter (Channel master, 1.22.0-10.0.pre.340, on Mac OS X 10.15.6 19G2021 x86_64, locale en)
    • Flutter version 1.22.0-10.0.pre.340 at /Users/pedromassango/dev/SDKs/flutter_master
    • Framework revision 1fef5912cf (6 hours ago), 2020-09-23 21:37:08 -0700
    • Engine revision dd35b5b659
    • Dart version 2.10.0 (build 2.10.0-156.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.1)
    • Android SDK at /Users/pedromassango/Library/Android/sdk
    • Platform android-30, build-tools 30.0.1
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0, Build version 12A7209
    • CocoaPods version 1.9.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] IntelliJ IDEA Community Edition (version 2020.2.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 49.0.4
    • Dart plugin version 202.7319.5

[✓] VS Code (version 1.48.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.14.1

[✓] Connected device (4 available)
    • sdk gphone x86 arm (mobile) • emulator-5554 • android-x86    • Android 11 (API 30) (emulator)
    • macOS (desktop)             • macos         • darwin-x64     • Mac OS X 10.15.6 19G2021 x86_64
    • Web Server (web)            • web-server    • web-javascript • Flutter Tools
    • Chrome (web)                • chrome        • web-javascript • Google Chrome 85.0.4183.121

! Doctor found issues in 1 category.
Process finished with exit code 0

Verified this issue using latest versions and observed that the behavior is still replicable.

 stable, master flutter doctor -v
[✓] Flutter (Channel stable, 3.3.7, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 3.3.7 on channel stable at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision e99c9c7cd9 (2 days ago), 2022-11-01 16:59:00 -0700
    • Engine revision 857bd6b74c
    • Dart version 2.18.4
    • DevTools version 2.15.0

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[!] Flutter (Channel master, 3.5.0-12.0.pre.168, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 3.5.0-12.0.pre.168 on channel master at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision a1432a9c5d (4 hours ago), 2022-11-07 19:17:03 -0600
    • Engine revision e7d7edab98
    • Dart version 2.19.0 (build 2.19.0-374.0.dev)
    • DevTools version 2.19.0
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.


Bugfix partitioning

The number of partitions set based on the number of specified rows per partition (optional argument) did not take into account the number of available cpu cores which can be suboptimal if the # cores is larger than the inferred number of estimated partitions.
This PR addresses this bug by taking into account the number of cores before repartitioning

I'm not sure if we need to fix this @PhilippeMoussalli.
It depends on which meaning we want the rows_per_partition argument to have. Before, we were using it as a target: The user specifies the number of rows they want per partition, fondant tries to achieve that number as closely as possible.
With your change, we handle it as max_rows_per_partition instead. If we want this, we might want to change the name or at least the description of the parameter. Caveat here is that we approximate the number of rows per partition, so it might still be slightly higher than the value set by the argument.
If we do want to go this way, I think we can simplify the logic a bit to:
if max(n_paritions_from_row, n_workers) > original_partitions:
    if n_paritions_from_row > n_workers:
        # Repartition based on n_partitions_from_row
    else:
        # Repartition based on n_workers

I'm not sure if we need to fix this @PhilippeMoussalli.
It depends on which meaning we want the rows_per_partition argument to have. Before, we were using it as a target: The user specifies the number of rows they want per partition, fondant tries to achieve that number as closely as possible.
With your change, we handle it as max_rows_per_partition instead. If we want this, we might want to change the name or at least the description of the parameter. Caveat here is that we approximate the number of rows per partition, so it might still be slightly higher than the value set by the argument.
If we do want to go this way, I think we can simplify the logic a bit to:
if max(n_paritions_from_row, n_workers) > original_partitions:
    if n_paritions_from_row > n_workers:
        # Repartition based on n_partitions_from_row
    else:
        # Repartition based on n_workers

Come to think of it, I indeed don't know if we need to include it. I initially wanted to change it since it wasn't allowing me to properly benchmark but that's just because the specified number of rows resulted in partitions that are less than the amount of workers. It might be that indeed for some reason you want to enforce a certain number of partitions per row.
Perhaps we can keep it as it originally was and just return a warning if the resulting number of partitions are lower than the workers? Wdyt?

Perhaps we can keep it as it originally was and just return a warning if the resulting number of partitions are lower than the workers? Wdyt?

Sounds good :+1:


added Nigxht

`{
"id": "nigxht",
"name": "Nigxht Network",
"addresses": [
"nigxht.net"
]
},
``

Please take a look at the checklist & recreate this pull request.

You're not supposed to create a new file