Pkg cannot resolve JSON modules

Describe the bug
When doing nix-build ./release.nix -A package.linux.x64.elf or any of the executable builds, it results in:
> Warning Cannot find module './test.json' from '/build/typescript-demo-lib/dist'  in /build/typescript-demo-lib/dist/utils.js

The test.json is currently in src/test.json which is imported by src/utils.ts.
The TSC compiles it properly and we see that src/test.json is in dist/test.json.
Even if I add dist/test.json to assets or scripts it doesn't work. And the resulting executable throws a MODULE_NOT_FOUND error.
CMCDragonkai | POLYHACK-SURF2 | C:\Users\CMCDragonkai\Downloads
 > .\9i6jr0lxji4l99gc0sw8sfpp5b6inhn3-typescript-demo-lib-1.1.2-win32-x64.exe
pkg/prelude/bootstrap.js:1697
      throw error;
      ^

Error: Cannot find module './test.json'
Require stack:
- C:\snapshot\typescript-demo-lib\dist\utils.js
- C:\snapshot\typescript-demo-lib\dist\lib\test-utp-native.js
- C:\snapshot\typescript-demo-lib\dist\bin\typescript-demo-lib.js
1) If you want to compile the package/file into executable, please pay attention to compilation warnings and specify a literal in 'require' call. 2) If you don't want to compile the package/file into executable and want to 'require' it from filesystem (likely plugin), specify an absolute path in 'require' call using process.cwd() or process.execPath.
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:885:15)
    at Function._resolveFilename (pkg/prelude/bootstrap.js:1776:46)
    at Function.Module._load (internal/modules/cjs/loader.js:730:27)
    at Module.require (internal/modules/cjs/loader.js:957:19)
    at Module.require (pkg/prelude/bootstrap.js:1676:31)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (C:\snapshot\typescript-demo-lib\dist\utils.js:4:21)
    at Module._compile (internal/modules/cjs/loader.js:1068:30)
    at Module._compile (pkg/prelude/bootstrap.js:1758:32)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1097:10) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    'C:\\snapshot\\typescript-demo-lib\\dist\\utils.js',
    'C:\\snapshot\\typescript-demo-lib\\dist\\lib\\test-utp-native.js',
    'C:\\snapshot\\typescript-demo-lib\\dist\\bin\\typescript-demo-lib.js'
  ],
  pkg: true
}

To Reproduce

nix-build ./release.nix -A package.linux.x64.elf
Watch the logs
Download executable to windows and execute it

Expected behavior
The pkg should be bundling the test.json as normal.
Note that package.json does not appear to be a problem.

It appears pkg is correct in that the dist/ inside the the nix build context doesn't have the test.json file.
This is strange, why is it missing there...
I found this through adding:
        echo 'show dist'
        ls dist
        echo 'show here'
        ls .

Into the buildPhase of buildElf.
show dist
bin         index.js      lib         utils.js
index.d.ts  index.js.map  utils.d.ts  utils.js.map
show here
LICENSE      jest.config.js     pkgs.nix     tsconfig.build.json
README.md    nix                release.nix  tsconfig.json
default.nix  node_modules       shell.nix    utils.nix
dist         package-lock.json  src
docs         package.json       tests
><EMAIL_ADDRESS>> Warning Cannot find module './test.json' from '/build/typescript-demo-lib/dist'  in /build/typescript-demo-lib/dist/utils.js

This means pkg config cannot solve this problem. This has to do with the fact that dist simply doesn't have the JSON file. It may be coming from:
      src = "${utils.node2nixDev}/lib/node_modules/${utils.node2nixDev.packageName}";

It appears that npm run build that is called inside utils.nix doesn't seem to resolve the JSON file.
Even though we are using the same command: rm -r ./dist || true; tsc -p ./tsconfig.build.json, the src directory has test.json, but the dist directory does not have test.json.
  node2nixDev = (import (node2nixDrv true) { inherit pkgs nodejs; }).package.override (attrs: {
    src = src;
    buildInputs = attrs.buildInputs ++ [ nodePackages.node-gyp-build ];
    dontNpmInstall = true;
    postInstall = ''
      # The dependencies were prepared in the install phase
      # See `node2nix` generated `node-env.nix` for details.
      npm run build

      echo '-------------AFTER BUILD-----------------'
      ls .
      echo 'SHOW DIST'
      ls dist
      echo 'SHOW SRC'
      ls src
      echo 'JSON IS MISSING!?!?'

      # This call does not actually install packages. The dependencies
      # are present in `node_modules` already. It creates symlinks in
      # $out/lib/node_modules/.bin according to `bin` section in `package.json`.
      npm install
    '';
  });

> rm -r ./dist || true; tsc -p ./tsconfig.build.json

rm: cannot remove './dist': No such file or directory
-------------AFTER BUILD-----------------
LICENSE      jest.config.js     pkgs.nix     tsconfig.build.json
README.md    nix                release.nix  tsconfig.json
default.nix  node_modules       shell.nix    utils.nix
dist         package-lock.json  src
docs         package.json       tests
SHOW DIST
bin         index.js      lib         utils.js
index.d.ts  index.js.map  utils.d.ts  utils.js.map
SHOW SRC
bin  index.ts  lib  test.json  utils.ts
JSON IS MISSING!?!?

For some reason, the tsc being used inside the Nix may not be resolving the JSON modules...

I'm going to check if the tsc is the wrong version. Changing the typescript version in package.json to 4.4.4 now. This triggers Nix to install the new tsc. Maybe that is what's needed to make tsc copy over the test.json. Because at this point, it mus be the tsc not copying over, even though if I run npm run build in nix-shell it does copy over.

The tsc is in fact the same version. So that's not the problem either.
According to https://stackoverflow.com/a/59419449/582917 and also the docs for include. It would indicate that we would need:
  "include": [
    "./src/**/*",
    "./src/**/*.json",
    "./tests/**/*"
  ]

See the ./src/**/*.json. Now by adding this in, the tsc inside the Nix environment finally copies over the test.json automatically.
However this seems weird because npm run build by itself in nix-shell doesn't require this and it will copy over the test.json automatically. It's just needed by the tsc running inside the nix-build.
The other important thing to realise is that JSON files that are required or just referenced other ways will not be automatically copied over unless they are import and that import may need to be static imports. This is talked about here: https://stackoverflow.com/a/59419449/582917
This means a more robust solution for importing JSON files in src is needed. A solution is found here: https://github.com/microsoft/TypeScript/issues/30835#issuecomment-850098727
The solution ends up using https://github.com/swimauger/tsc-hooks to hook in and copy over all the other files that tsc doesn't resolve.
This actually can help solve the copying over the protobuf files. Currently we're doing:
    "postbuild": "cp -fR src/proto dist",

Inside js-polykey, and instead of having specific postbuild hooks, we hook into tsc to copy over all the relevant files.
For now, I think adding ./src/**/*.json will at least ensure that resolveJsonModule is actually working even in nix-build. This will mostly apply to us importing JSON schema files which are JSON files. But it won't work on say .txt files or .xml files. A bundler is usually used for this.
If we have other asset files that need to be copied into dist/ from src/ we can use the postbuild hook like in js-polykey or the tsc-hooks.
Another a good suggestion is to have "assets" outside of src, then tsc is not needed for it.
Of course the problem is that sometimes we would consider non-js, non-ts files to be "source" files. Other times they are liek images, which can be stored outside the ./src. If they are outside src then nothing we need to do in post-build nor tsc hooks. The tsc hooks and postbuild will be necessary for any non-js, non-ts, and now non-json files in ./src.

@tegefaulkes @joshuakarp beware of this.
To help catch future errors like this, we need automated tests that run post-CI/CD build of the executables. Which is to be addressed in https://github.com/MatrixAI/js-polykey/issues/231

Commented upstream in typescript regarding this problem: https://github.com/microsoft/TypeScript/issues/30835#issuecomment-961655139


topolvm.io

This is a merge PR for reducing the number of commits for #539

#539
#168
#589

The below commands created this PR:
$ git checkout main
$ git pull
$ git checkout topolvm.io
$ git merge main
$ git push
$ git diff main > /tmp/topolvm.io.patch
$ git checkout main
$ git co -b topolvm.io4
$ git apply /tmp/topolvm.io.patch
$ git add .
$ git commit -m "topolvm.io"
$ git push

What I usually do when I need to change something in a commit is the following:
git reset --soft HEAD~
Make the changes and git add
git commit again
git push -u origin  --force
This will overwrite the branch head so I can use the same PR. It may be a way to proceed with these changes instead of creating a new PR.

What I usually do when I need to change something in a commit is the following:

Thank you. I never tried the git reset --soft command.
I'll try these commands the next time.

LGTM. I will approve it after you squashed it.


improve ncp and ncpplots

Hey Meng -- Was there some reason you wanted this new version with
results_id? This just seems like an error-prone change that doesn't have
any benefit. If there wasn't a reason, can you revert to the old interface
with methods_names? This new interface with results_id just seems like more
bugs waiting to happen. (e.g. there is an implicit map between things, vs.
an explicit map...) If there was a reason for multiple sets of results, I'd
love to understand it, otherwise, go back and see how I did that with
method_names and the map before.
David
On Sun, Apr 15, 2018 at 2:29 PM, Meng Liu<EMAIL_ADDRESS>wrote:

You can view, comment on, or merge this pull request online at:
https://github.com/kfoynt/LocalGraphClustering/pull/41
Commit Summary

improved ncp and ncpplots
Merge remote-tracking branch 'upstream/master'

File Changes

M localgraphclustering/ncp.py
https://github.com/kfoynt/LocalGraphClustering/pull/41/files#diff-0
(62)
M localgraphclustering/ncpplots.py
https://github.com/kfoynt/LocalGraphClustering/pull/41/files#diff-1
(17)
M notebooks/examples.ipynb
https://github.com/kfoynt/LocalGraphClustering/pull/41/files#diff-2
(18162)

Patch Links:

https://github.com/kfoynt/LocalGraphClustering/pull/41.patch
https://github.com/kfoynt/LocalGraphClustering/pull/41.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kfoynt/LocalGraphClustering/pull/41, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AACEy6TI95DgeI-OJd6W7sKK4IHQqyTrks5to5GhgaJpZM4TVloh
.


host error limit cannot edit post

Issue Summary
Newsletter won't go to subs. Can't schedule/post

Steps to Reproduce
Try scheduling post. Won't allow.
Ghost Version
unsure
Node.js Version
unsure
How did you install Ghost?
pro
Database type
Other
Browser & OS version
No response
Relevant log / error output
No response
Code of Conduct

[X] I agree to be friendly and polite to people in this repository

Hey @aaronpartnerhacker  👋 Friendly reminder: help or support is not available on Github.
This is a Ghost(Pro) specific error related to your plan limits. Please contact<EMAIL_ADDRESS>where they can give you personalised help.


add realisation cfg.Unit

Description
Add a new configuration handler type cfg.Unit . It allows you to use units of measurement KB KiB, MB, MiB and others. Also cfg.Unit must support adding aliases.
Fixes # (issue)

Hi @DSmolonogov! Thanks for the PR!
Could you take a look at the contributing guidelines and make sure you follow them when creating issues/PR's?
These guidelines are here to facilitate a faster & more convenient review process for everyone.
Cheers 💕

LGTM

LGTM


WebApp:App Service - SSH Tunnel Issue (preview feature)

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
I am attempting to remotely debug an linux container running in App Service using Visual Studio 2017/2019. I have enabled SSH inside my container and am using the cli command below to create an ssh tunnel to my app service:
az webapp create-remote-connection --subscription <subscription-id> --resource-group <resource-group-name> -n <app-name> &
Found here: https://docs.microsoft.com/en-us/azure/app-service/configure-linux-open-ssh-session#open-ssh-session-from-remote-shell
When using an remote ssh extension in VSCode, I am able to remotely attach to the container. When attempting in Visual Studio 2019 via the connection manager, I get an error. I have looked into Visual Studio and it appears it requires 2 concurrent connections (ssh & sftp). From looking at the logs inside my container, I can see Visual Studio connect and the disconnect but I cannot see 2 connections.
I also validated with putty that I am not able to make 2 concurrent SSH connections to my container.
To Reproduce
This same project: https://github.com/grtn316/BlazorAppContainerLinux
Expected behavior
Visual Studio should be able to remotely connect to the container for debugging.
Environment summary
-Latest version of Visual Studio 2019.
-App service hosting a linux container
more details found in github repo.

webapp

webapp

Will check and try to repro the issue. Will Get back with details.

I showed the issue to the Visual Studio connection manager team and we discovered that the cli command:
az webapp create-remote-connection --subscription <subscription-id> --resource-group <resource-group-name> -n <app-name> &
will sometimes let me connect with putty, disconnect and then reconnect with a new putty session again. Other times, when I connect with putty and disconnect, I can no longer reconnect. This behavior is a bit unpredictable and may be related to the original issue.

@Kotasudhakarreddy Any updates here?

Issue still exits in Visual Studio 2022 (17.1.1) with SSH debugging.

Issue still exits in Visual Studio 2022 Enterprise (17.1.1) with SSH debugging.

Although not ideal...there is an ssh based debug extension in VSCode that should allow you to do remote debugging.

this is a question on VS extension & explicitly the CLI command. @btardif / @seligj95 do we have contact for ' remote ssh extension in VSCode'?

I'll try out some vs code extensions tomorrow, thanks for the tip.

There is a workaround for this problem. You need to open 2nd tunnel with in the first tunnel. The second tunnel is can be used multiple times and you can attach debugger in vs without problem. But you should delete remote fingerprints and  connections in vs if your target is changed because it is always connects to local host even if remote is different.
For example open the remote tunnel like,
 ssh -L 33333:localhost:2222<EMAIL_ADDRESS>-p 55555
55555 is your first tunnel port. 33333 is new tunnel port. 2222 remote ssh port. Note that specify the port with  -p option when you create the first tunnel.
Now you can can connect port 33333 in visual studio to debug.


feat(cli): lacework query fail_on_count

As a Lacework user I would like the "lacework query" CLI command to return a non-zero exit code when the number of results returned by my query meets a certain criteria.
I believe the cleanest way to implement this functionality would be to implement a "--fail_on_count" flag which can take on values such as ">0", "=1", "<10", etc.  This allows us to evaluate the number of results returned by the query without having to alter the semantics of the query in any way.
ALLY-485

It took a lot of Customer interviews and UX sessions to figure out that a fail flag would help, have you reached out to anyone to verify that this is indeed something we want to add to the query commands? If so, go for it! 🙌🏽

Would the previous interview and UX sessions not just apply here?   I personally haven't reached out to anyone, but we're welcome to stash this until we can validate that customers will adopt such a thing...

Should we close this PR?

I thought this was good to go we just wanted to refactor the "policy enforcement flows"...

@hazedav lets refactor both "policy enforcement" workflows.

Also captured in this thread:
https://lacework.slack.com/archives/C01RSNZ0C5V/p1649787895250409?thread_ts=1648222632.313529&cid=C01RSNZ0C5V

Updated to use new(FAIL-ON) exit terminology.


mark linux packages autoroller bringup: true

Unblock the tree while I investigate https://github.com/flutter/flutter/issues/155704

I'm merging this with a pending flutter-gold check because this will not affect goldens, and merging should fix the broken tree.


Add link to options on hero section

Related Issue(s)

Fixes #173

Description
Add the link to the option in the hero section
Changes Made
component
Provide a detailed list of the changes made in this pull request. Include specific files, components, or functionalities affected by your modifications.
https://github.com/oxiton-foundation/click-metrics/assets/124615886/61e68644-b552-4af4-81c8-ba9dbe6d9d34
Screenshots (if applicable)
Attach screenshots or Videos demonstrating the visual changes introduced by your pull request. This helps reviewers understand the user interface modifications more effectively.
Checklist
Before submitting your pull request, ensure that you have completed the following tasks:

[x] Code follows the project's style and conventions.
[x] Code has been tested thoroughly, and all tests pass successfully.
[x] Documentation has been updated to reflect any relevant changes.
[x] All new and modified code is adequately commented for clarity.
[x] Pull request title follows the specified format: [Feature/Enhancement/Bug Fix]: Brief description of the change.

Additional Notes
Add any additional information, concerns, or considerations that may be relevant to your pull request. This could include performance implications, future enhancements, or potential risks associated with the changes.

Thanks for PR
I am merging it


Propose new dependency structure for 2.0

The current Dependency definition incurs an extra layer of depth to support an uncommon case like // @deno-types, we can flatten it with these types:
export type Dependency = {
  specifier?: string; // ┐
  error?: string;     // ┴─ mutually exclusive
  range: Range;
  isTypeOnly?: boolean;
  isDynamic?: boolean;
  externalTypes?: string; // @deno-types is extracted as another dependency, the key goes here.
}

export interface EsModule {
  dependencies: Record<string, Dependency>; // made into a record to dereference externalTypes, only downside imo
  externalTypes: { // renamed from typesDependency
    key: string; // specifier text
    specifier?: string; // ┐
    error?: string;     // ┴─ mutually exclusive
    range?: Range; // defined for ts reference, not X-TypeScript-Types (currently non-optional field with strange default)
  }
  // ...
}

This module gets the following serializations:
/// <reference types="./external_types.d.ts" />
/// <reference path="./type_only.d.ts" />
import * as a from "./runtime.ts";
// @deno-types="./untyped_runtime.d.ts"
import * as a from "./untyped_runtime.js";
import * as a from "bad.js";

Currently
{
  kind: "esm",
  dependencies: [
    {
      specifier: "./runtime.ts",
      code: {
        specifier: "file:///runtime.ts",
        span: { start: { line: 3, character: 21 }, end: { line: 3, character: 35 } }
      }
    },
    {
      specifier: "./type_only.d.ts",
      type: {
        specifier: "file:///type_only.d.ts",
        span: { start: { line: 2, character: 22 }, end: { line: 2, character: 40 } }
      }
    },
    {
      specifier: "./untyped_runtime.js",
      code: {
        specifier: "file:///untyped_runtime.js",
        span: { start: { line: 5, character: 21 }, end: { line: 5, character: 43 } }
      },
      type: {
        specifier: "file:///untyped_runtime.d.ts",
        span: { start: { line: 4, character: 17 }, end: { line: 4, character: 41 } }
      }
    },
    {
      specifier: "bad.js",
      code: {
        error: 'Relative import path "bad.js" not prefixed with / or ./ or ../',
        span: { start: { line: 6, character: 21 }, end: { line: 6, character: 29 } }
      }
    }
  ],
  size: 250,
  typesDependency: {
    specifier: "./external_types.d.ts",
    dependency: {
      specifier: "file:///external_types.d.ts",
      span: { start: { line: 2, character: 23 }, end: { line: 2, character: 46 } }
    }
  },
  mediaType: "JavaScript",
  specifier: "file:///entry.js"
}

Proposed
{
  kind: "esm",
  dependencies: {,
    "./runtime.ts": {
      specifier: "file:///runtime.ts",
      range: { start: { line: 3, character: 21 }, end: { line: 3, character: 35 } }
    },
    "./type_only.d.ts": {
      specifier: "file:///type_only.d.ts",
      range: { start: { line: 2, character: 22 }, end: { line: 2, character: 40 } },
      isTypeOnly: true
    }
    "./untyped_runtime.js": {
      specifier: "file:///untyped_runtime.js",
      range: { start: { line: 5, character: 21 }, end: { line: 5, character: 43 } },
      externalTypes: "./untyped_runtime.d.ts",
    },
    "./untyped_runtime.d.ts": {
      specifier: "file:///untyped_runtime.d.ts",
      range: { start: { line: 4, character: 17 }, end: { line: 4, character: 41 } },
      isTypeOnly: true
    },
    "bad.js": {
      error: 'Relative import path "bad.js" not prefixed with / or ./ or ../',
      range: { start: { line: 6, character: 21 }, end: { line: 6, character: 29 } }
    }
  },
  size: 250,
  externalTypes: {
    key: "./external_types.d.ts",
    specifier: "file:///external_types.d.ts",
    range: { start: { line: 2, character: 23 }, end: { line: 2, character: 46 } }
  },
  mediaType: "JavaScript",
  specifier: "file:///entry.js"
}

cc @dsherret

Although it seems nicer, I'm not sure it's worth changing so drastically at this point because of the amount of work it would require to change the code using the output. I think we should only make changes to the output that do stuff like adding more detail (ex. imports) or fixing the output for bugs.

There is at least one piece of info that the current structure is missing for seemingly no reason, the relative specifier text associated with a dependency.type from // @deno-types. And there are a lot of smells like why does a ResolutionResolved object contain a range, and why is TypeDependency's range non-optional with a nonsensical default for X-TypeScript-Types? There's too much legacy surrounding Resolution and it's never had a proper overhaul, the non-extensibility of it will keep being an issue.
Although I used the json for illustration, it's really the rust API that's been frustrating me. How open are we to making rust changes and preserving the json in serialize_dependencies?

Although I used the json for illustration, it's really the rust API that's been frustrating me. How open are we to making rust changes and preserving the json in serialize_dependencies?

Very open. We should discuss major changes before doing them so I appreciate this issue. Also, we should do each change slowly one at a time.

There is at least one piece of info that the current structure is missing for seemingly no reason, the relative specifier text associated with a dependency.type from // @deno-types.

Yeah, I noticed that one, but I'm not sure how important it is to know about for people... maybe we should just fix it.

And there are a lot of smells like why does a ResolutionResolved object contain a range

Agree. I think that should be available regardless of whether it succeeds or not. Also, we should probably remove the specifier property on Range because that's duplicate information that can be figured out by looking at where you are in the graph.

why is TypeDependency's range non-optional with a nonsensical default for X-TypeScript-Types

Agree. I had previously opened #133 about this.

What do you think about making these changes one PR at a time and without breaking --json for now? Then in 2.0 we can break the JSON.
Btw, I did a bad PR last week in a rush and am reverting most of it in #249

Please include Range char coordinates (the whole source start char and end chart, not line+char)
Please include also the whole import statement coordinates (to be able to remove it completely or modify it completely) right now tools need parse out the string before Range.start to find out where the import keyword is located, or what are the local bindings)


Unsafe API endpoints

/users/{username}/role
/users/admins/{username}
/users/students/{username}
/users/lecturers/{username}
this can lead to problems, as a user named "admins", "students" or "lecturers", paired with a user named "role", can lead to ambiguous routes.

I would recommend switching to ids. thats the usual api behaviour when dealing with subelements of a larger collection (a user from users collection)
Google provides some useful api design tips and standard stuff here :)
https://cloud.google.com/apis/design/design_patterns


2nd PetDoor

I‘ve installed and added a second PetDoor in my house.
Only the first one is shown and Trigger the Pets didn‘t work anymore.
All sensors are shown as activated.

Hey @Sommi486 is this still an issue for you?
By chance have you tried resetting the plugin?

Hey @wylanswets thanks for your answer and glad to see that the plugin will be further developed!
After restarting HomeBridge the second PetDoor was shown and reacted as the first.

Veeeeeeeeeeery interesting lol. I was thinking to myself there wasn't a reason multiple wouldn't show up haha. Glad it is working out!


Display text on categories and tags screens

Add "imported from Quickbooks Online" text and link

Details

This PR:

replaces withPoliyc HOC with withPolicyConnections to get the policy object with the connections field fetched
get the policy ID from the useRoute hook
display the explanatory text when there is any accounting connection ( in this case, just QBO)

Fixed Issues

$ https://github.com/Expensify/App/issues/40660
PROPOSAL:
Tests / QA Steps

Ensure that you use an account that is on the Accounting beta list
Go to a collect workspace setting
Enable categories on the More Features page
Check the categories page and ensure that the text at the top looks like the one below

Enable accounting on the More Features page
Connect to QBO on the More Features page
Go back to the Categories page
Check the text at the top looks like the one below

Click the link in the text. Ensure you're taken back to the Accounting Page.

Offline tests

N/A
PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

Spanish translation coming...

Categorías importadas desde QuickBooks Online

Starting code review in a moment. noticed the following:

QA will not be able to test this, still facing the infinite loading indicator issue
Tags screen is not covered
QBO is not connected and the text on categories says that are imported from QBO

https://github.com/Expensify/App/assets/16493223/ac0eadb5-8769-496b-bd42-f6754c09af85

Reviewer Checklist

[X] I have verified the author checklist is complete (all boxes are checked off).
[X] I verified the correct issue is linked in the ### Fixed Issues section above
[X] I verified testing steps are clear and they cover the changes made in this PR

[X] I verified the steps for local testing are in the Tests section
[X] I verified the steps for Staging and/or Production testing are in the QA steps section
[X] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[X] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[X] I checked that screenshots or videos are included for tests on all platforms
[X] I included screenshots or videos for tests on all platforms
[X] I verified tests pass on all platforms & I tested again on:

[X] Android: Native
[X] Android: mWeb Chrome
[X] iOS: Native
[X] iOS: mWeb Safari
[X] MacOS: Chrome / Safari
[X] MacOS: Desktop

[X] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[X] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[X] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[X] I verified that comments were added to code that is not self explanatory
[X] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[X] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[X] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[X] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[X] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[X] I verified the JSDocs style guidelines (in STYLE.md) were followed

[X] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[X] I verified that this PR follows the guidelines as stated in the Review Guidelines
[X] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[X] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[X] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[X] If a new component is created I verified that:

[X] A similar component doesn't exist in the codebase
[X] All props are defined accurately and each prop has a /** comment above it */
[X] The file is named correctly
[X] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[X] The only data being stored in the state is data necessary for rendering and nothing else
[X] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[X] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[X] All JSX used for rendering exists in the render method
[X] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[X] If any new file was added I verified that:

[X] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[X] If a new CSS style is added I verified that:

[X] A similar style doesn't already exist
[X] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[X] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[X] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[X] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[X] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[X] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[X] I verified that all the inputs inside a form are aligned with each other.
[X] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[X] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[X] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/16493223/e9d829bd-5ed0-4600-8d49-ee147e8cc6fa

MacOS: Desktop

https://github.com/Expensify/App/assets/16493223/4e27ebb3-bb33-4090-bd5f-bc2aa9ec4fa2

@hayata-suenaga should I merge or do you want to wait/have @marcochavezf review as well?

I don't know why Melvin requested a review from Marco 🤔

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/hayata-suenaga in version: 1.4.65-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/mountiny in version: 1.4.65-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Running cargo make audit in a workspace does not produce useful behaviour

Problem Description
Running cargo make audit in a workspace does not produce useful behaviour —
cargo audit is attempted to be run inside each workspace member, but fails on the first because Cargo.lock is not found.
[cargo-make] INFO - Execute Command: "cargo" "audit"
error: Couldn't find 'Cargo.lock'!

Run "cargo build" to generate lockfile before running audit

The following table shows what I kind of expect when running cargo make audit:

Working directory
Expected Behaviour

workspace root
cargo audit is run once over the entire workspace

member crate
cargo audit is either not run at all, or is run, passing the workspace Cargo.lock as an argument

I was attempting to fix this in https://github.com/azriel91/cargo-make/tree/feature/cargo-audit-for-workspace, but then discovered the "workspace" task is generated in code. That's where I stopped :grimacing:
Oh ya, should the audit task also have install_crate = "cargo-audit"? I didn't check if it automatically does

Thanks I'll fix that.
For now you can workaround it by overriding the task and just adding
[tasks.audit]
cwd = ".."

it should work (I hope)

cargo-make 0.10.3 has been released with a fix which does the following:
Only if Cargo.lock exists in working directory will the audit be invoked. so for members in workspace this will be skipped. You will have to run in the workspace with the no workspace flag to run this for workspace level audit.
As for the install_crate, there is no need to add install_crate in most cases.
cargo-make already knows how to do that automatically so that was not an issue.
Forgot to mention that in my original reply.

@azriel91 can you please confirm the fixes are working good for you?

Hey, sorry for not testing sooner; on 0.10.3, cargo make --no-workspace audit is nearly a no-op on the workspace root:
$ cargo make --no-workspace audit
[cargo-make] INFO - cargo-make 0.10.3
[cargo-make] INFO - Using Build File: Makefile.toml
[cargo-make] INFO - Task: audit
[cargo-make] INFO - Setting Up Env.
[cargo-make] INFO - Running Task: init
[cargo-make] INFO - Running Task: audit
[cargo-make] INFO - Running Task: end
[cargo-make] INFO - Build Done  in 0 seconds.

I haven't checked the code why, but it may be because this is a virtual workspace:
# Cargo.toml
[workspace]
members = ["app/*", "crate/*"]

I've confirmed it doesn't run cargo audit a member crate if you happen to run it in there:
$ cargo make audit
[cargo-make] INFO - cargo-make 0.10.3
[cargo-make] INFO - Using Build File: Makefile.toml
[cargo-make] INFO - Task: audit
[cargo-make] INFO - External file not found, skipping.
[cargo-make] INFO - Setting Up Env.
[cargo-make] INFO - Running Task: init
[cargo-make] INFO - Running Task: audit
[cargo-make] INFO - Running Task: end
[cargo-make] INFO - Build Done  in 0 seconds.

Ok, I'll run few more manual tests on workspaces as well tomorrow.
On members, it should do nothing as you saw.

I found the issue and will push a fix later today/tomorrow.

fixed in 0.10.5, the issue was that I also checked dependencies exist in the cargo.toml but in the workspace level, not always the dependencies are defined there, so I now set the dependencies env var to true not just if dependencies exist in the toml file, but also if we are working in a workspace level cargo.toml file.
Hopefully that should resolve it.

Yeaps it works now, thank you!


Cross table can link to a "match result" page

The match result page can have a list of the matchups for that round between those two teams and show you the results.

This is done.


problem with multi-line prompts and vi-mode plugin

I'm a new to both zsh and oh-my-zsh.  I was working with a multi-line prompt:
> '

I also had the vi-mode plugin enabled.  However, when I go into command mode by pressing  it eats the last line of output from the previous command.  I've tracked the problem down to the line in the plugin file: zle -N zle-keymap-select
I'm using zsh version 4.3.9 (the one that ships with Mac OS X 10.6).

This is still happening to me with my multi-line prompt setup.

[This line] (https://github.com/justmytwospence/dotfiles/blob/master/home/.zshrc#L175) fixed it for me, @bezhermoso. Maybe it will work for you if you are modifying zle-keymap-select or zle-line-init.

Unfortunately it doesn't fix it, @justmytwospence. I'm just using the vi-mode plugin which has that fix but still having the problem.

@justmytwospence Even an empty definition like this cause the same thing:
function zle-line-init zle-keymap-select {
  zle reset-prompt
  zle -R
}

Thoughts?

I'm honestly in over my head with this stuff, sorry.
On Tue, Oct 27, 2015 at 8:06 PM, Bez Hermoso <<EMAIL_ADDRESS><EMAIL_ADDRESS>> wrote:
@justmytwospence [https://github.com/justmytwospence] Even an empty definition like this cause the same thing:
function zle-line-init zle-keymap-select {
zle reset-prompt
zle -R
}
Thoughts?
—
Reply to this email directly or view it on GitHub
[https://github.com/robbyrussell/oh-my-zsh/issues/387#issuecomment-151707788] .[https://github.com/notifications/beacon/ACr-lk0c8nGkPUuq3chKMbK6zZzITQUDks5pADM_gaJpZM4AS2es.gif]


PPC NTU 'KhPI'

Our college (polytechnic.poltava.ua) uses Linux as a primary OS, so we we are looking for a great .NET IDE for Linux. Unfortunately Rider has no Community edition...

@liketaurus Pull request merged. Thank you!

Thank you, @philipto !


Add Tab completion for directories

So that cd ta will tab complete to target.

This is fixed now, it completes correctly.


The submudule brain seems missing

https://github.com/AndreaCatania/godot/commit/a7acc38117de4aac25ea11f1b9aaba12d3a7b843#commitcomment-32429954

Fixed by #7d62206843be381d78d8faa11db8857a2497d3bc


Doesn't work on Latitude 7370

I tried to use it on my Latitude 7370 running Catalina, with both kexts injected, it recognizes the SD Card reader, but no SD Card shows up on Disk Utility or anything.
If someone can help..
Built in SD Card Reader:
Vendor ID:	0x10ec
Device ID:	0x525a
Subsystem Vendor ID:	0x1028
Subsystem ID:	0x071b
Revision:	0x0001
Link Width:	x1
Link Speed:	5.0 GT/s

RTS525A is my card reader

Please attach the complete kernel log as per FAQ. Issues without kernel logs or with incomplete kernel logs will be ignored and closed.


Bug: Unique constraint violation on ordered messages

There's a race condition when you send ordered messages at almost the same time that can cause more than one message to have the same after_message_id, causing the constraint
CREATE UNIQUE INDEX ON mq_msgs(channel_name, channel_args, after_message_id);

to be violated.
I have yet to create a minimal test case that reproduces the bug, but I intend to do so.

I think this is somewhat expected if you run in isolation level "READ COMMITTED" or lower (which is the default for statements run outside of a transaction).
In that case, you may get a unique constraint violation - which should be handled in this case by retrying the operation.
If you run in "SERIALIZABLE" isolation level, then you should get a serialization_failure instead, which should always be handled by retrying.
It's not really possible to handle that within sqlxmq, since we don't know if the executor is a transaction or not, and if it is a transaction, it's the entire transaction that must be retried.
However, in general you will always need this kind of retry logic around database operations.

I considered using some kind of locking (eg. SELECT FOR UPDATE) but it would likely reduce the throughput quite a bit, as well as increasing complexity. It's also not really in-line with how postgres is designed to work.
The best option might be to add a function to this crate which, given a sqlx::Error, tells the caller whether the operation should be retried. And then we can add that to the documentation and examples.

That would certainly be a big quality of life improvement, and would also mitigate library consumers doing retries incorrectly.

I've released 0.3.0 which has this function, among other improvements.


Migrating from Fabric to Firebase

Description
I wanted to request that you migrate your code from Fabric to Firebase.
Basic Information

Device type: Android
OS version:
App version:

Hi，I am new in the modular thing. When I try this tutorial，I fail at the step of setting the fabric (or firebase). So I want to ask if there has been any progress in the issue #21


Update Test Smallfile Scripts

Fix HiBench test does not use compacted small files problem
Change use hdfs dfs commands to delete existing output data to ssm delete action
Add new get_cmdlets_of_rule func in util.py

You should commit based on a new branch tracking newest trunk to avoid keeping your merged commits in PR. You can create a new branch and cherry-pick your new commits, then submit a new PR.

Please use 'git pull --rebase' to avoid a merge commit for updating.

got it

You need update the README.md, test_transparent_read.py is not valid. Better to add usage of your python scripts in document.

Should delete test_small_file.py if never be used.

Check PR #1795 for further updates.


[CELEBORN-727][TEST] Fix flaky test RssHashCheckDiskSuite

What changes were proposed in this pull request?
Fix the flaky test by enlarging celeborn.client.shuffle.expired.checkInterval
Why are the changes needed?
RssHashCheckDiskSuite:
- celeborn spark integration test - hash-checkDiskFull *** FAILED ***
  868 was not less than 0 (RssHashCheckDiskSuite.scala:83)

https://github.com/apache/incubator-celeborn/actions/runs/5396767745/jobs/9800766633
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass GA, and should observe CI,

The flaky actually occurs "shuffle key not expired, diskInfo.actualUsableSpace < 0, no space", increase "celeborn.client.shuffle.expired.checkInterval" from 10s to 20s should help, the rest are refactoring.

The flaky actually occurs "shuffle key not expired, diskInfo.actualUsableSpace < 0, no space", increase "celeborn.client.shuffle.expired.checkInterval" from 10s to 20s should help, the rest are refactoring.

Thanks for explaining this, but you mean from 5s?

right, from 5s to 20s


chore: bdd-sbhg-1575391693 to 0.0.1

chore: Promote bdd-sbhg-1575391693 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


dnstap showing extra responses

I'm seeing extra responses in the dnstap stream from unbound 1.13.0. I'm not sure if this is unbound actually generating extra responses or the dnstap stream duplicating the responses. Here is an example (decoded using https://github.com/dnstap/golang-dnstap/tree/master/dnstap) that I extracted from my dnstap logs. Note the extra client response. The timestamps also look strange. The timestamp of the client response is before the timestamp of the forwarder response.
message:
  type: CLIENT_QUERY
  query_time: !!timestamp 2020-12-09 23:15:19.20792
  socket_family: INET
  socket_protocol: UDP
  query_address: <IP_ADDRESS>
  query_port: 54582
  query_message: |
    ;; opcode: QUERY, status: NOERROR, id: 2
    ;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;pubsub.pubnub.com.	IN	 A
---
type: MESSAGE
message:
  type: CLIENT_RESPONSE
  response_time: !!timestamp 2020-12-09 23:15:19.208216
  socket_family: INET
  socket_protocol: UDP
  query_address: <IP_ADDRESS>
  query_port: 54582
  response_message: |
    ;; opcode: QUERY, status: NOERROR, id: 2
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;pubsub.pubnub.com.	IN	 A

    ;; ANSWER SECTION:
    pubsub.pubnub.com.	18	IN	A	<IP_ADDRESS>
    pubsub.pubnub.com.	18	IN	A	<IP_ADDRESS>
---
type: MESSAGE
message:
  type: FORWARDER_QUERY
  query_time: !!timestamp 2020-12-09 23:15:19.208601
  socket_family: INET
  socket_protocol: UDP
  response_address: <IP_ADDRESS>
  response_port: 53
  query_zone: "."
  query_message: |
    ;; opcode: QUERY, status: NOERROR, id: 57949
    ;; flags: rd cd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

    ;; QUESTION SECTION:
    ;pubsub.pubnub.com.	IN	 A

    ;; ADDITIONAL SECTION:

    ;; OPT PSEUDOSECTION:
    ; EDNS: version 0; flags: do; udp: 1472
---
type: MESSAGE
message:
  type: CLIENT_RESPONSE
  response_time: !!timestamp 2020-12-09 23:15:19.208623
  socket_family: INET
  socket_protocol: UDP
  query_address: <IP_ADDRESS>
  query_port: 54582
  response_message: |
    ;; opcode: QUERY, status: NOERROR, id: 2
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;pubsub.pubnub.com.	IN	 A

    ;; ANSWER SECTION:
    pubsub.pubnub.com.	18	IN	A	<IP_ADDRESS>
    pubsub.pubnub.com.	18	IN	A	<IP_ADDRESS>
---
type: MESSAGE
message:
  type: FORWARDER_RESPONSE
  query_time: !!timestamp 2020-12-09 23:15:19.20788
  response_time: !!timestamp 2020-12-09 23:15:19.228139
  socket_family: INET
  socket_protocol: UDP
  response_address: <IP_ADDRESS>
  response_port: 53
  query_zone: "."
  response_message: |
    ;; opcode: QUERY, status: NOERROR, id: 57949
    ;; flags: qr rd ra cd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

    ;; QUESTION SECTION:
    ;pubsub.pubnub.com.	IN	 A

    ;; ANSWER SECTION:
    pubsub.pubnub.com.	93	IN	A	<IP_ADDRESS>
    pubsub.pubnub.com.	93	IN	A	<IP_ADDRESS>

    ;; ADDITIONAL SECTION:

    ;; OPT PSEUDOSECTION:
    ; EDNS: version 0; flags: do; udp: 1232```

Hi,
This looks like you have the prefetch: yes option and thus the "strange" timestamps you are seeing.
The client gets an immediate reply from cache and the query is resolved in order to update the cache.
Other than that the client response at response_time: !!timestamp 2020-12-09 23:15:19.208623 may seem weird.
Can you maybe share the rest of your configuration?

@gthess I saw something similar today, where it seems we would get duplicate CLIENT_RESPONSE messages logged when only answering a single request (and only seeing a single response returned on the wire). Here is some JSON-data of the two messages:
{"type":"MESSAGE","identity":"resolver1.example.com","version":"unbound 1.19.1","message":{"type":"CLIENT_RESPONSE","response_time":"2024-04-03T08:21:11.982616Z","socket_family":"INET","socket_protocol":"UDP","query_address":"<IP_ADDRESS>","response_address":"<IP_ADDRESS>","query_port":19341,"response_port":53,"response_message":";; opcode: QUERY, status: NXDOMAIN, id: 38575\n;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1\n\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags:; udp: 1232\n; EDE: 3 (Stale Answer): ()\n\n;; QUESTION SECTION:\n;www5.example.com.\tIN\t A\n\n;; AUTHORITY SECTION:\nexample.com.\t272\tIN\tSOA\thidden-master.example.com. hostmaster.example.com.<PHONE_NUMBER> 28800 7200 604800 300\n"}}

{"type":"MESSAGE","identity":"resolver1.example.com","version":"unbound 1.19.1","message":{"type":"CLIENT_RESPONSE","response_time":"2024-04-03T08:21:11.982723Z","socket_family":"INET","socket_protocol":"UDP","query_address":"<IP_ADDRESS>","response_address":"<IP_ADDRESS>","query_port":19341,"response_port":53,"response_message":";; opcode: QUERY, status: NXDOMAIN, id: 38575\n;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1\n\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags:; udp: 1232\n; EDE: 3 (Stale Answer): ()\n\n;; QUESTION SECTION:\n;www5.example.com.\tIN\t A\n\n;; AUTHORITY SECTION:\nexample.com.\t272\tIN\tSOA\thidden-master.example.com. hostmaster.example.com.<PHONE_NUMBER> 28800 7200 604800 300\n"}}

What sticks out is that when the duplicates arrive they have EDE: 3 (Stale Answer) in them. Asking for the same thing afterwards only results in a single log entry (and no stale answer data).
For reference we are running with both
        prefetch: yes
        prefetch-key: yes

... and
        ede: yes
        ede-serve-expired: yes

What are the serve-expired* options you are using?

These:
serve-expired: yes
serve-expired-ttl: 14400

I believe I see why. Can you verify that this happens every single time for expired (or prefetched) answers? You can set something like cache-max-ttl: 2 for easy testing with expired answers every 3 seconds for example.

This looks in line with what I am experiencing. Interestingly I do notice that I sometimes (with cache-max-ttl: 2) seem to also get duplicates where the EDE info is not present, for example here is the result of two queries, where the first one got duplicate entries with EDE in it, and the second one is without EDE:
{"type":"MESSAGE","identity":"resolver1.example.com","version":"unbound 1.19.1","message":{"type":"CLIENT_RESPONSE","response_time":"2024-04-03T10:20:43.284279Z","socket_family":"INET","socket_protocol":"UDP","query_address":"<IP_ADDRESS>","response_address":"<IP_ADDRESS>","query_port":17387,"response_port":53,"response_message":";; opcode: QUERY, status: NXDOMAIN, id: 2841\n;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1\n\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags:; udp: 1232\n; EDE: 3 (Stale Answer): ()\n\n;; QUESTION SECTION:\n;www5.example.com.\tIN\t A\n\n;; AUTHORITY SECTION:\nexample.com.\t30\tIN\tSOA\thidden-master.example.com. hostmaster.example.com.<PHONE_NUMBER> 28800 7200 604800 300\n"}}
{"type":"MESSAGE","identity":"resolver1.example.com","version":"unbound 1.19.1","message":{"type":"CLIENT_RESPONSE","response_time":"2024-04-03T10:20:43.284349Z","socket_family":"INET","socket_protocol":"UDP","query_address":"<IP_ADDRESS>","response_address":"<IP_ADDRESS>","query_port":17387,"response_port":53,"response_message":";; opcode: QUERY, status: NXDOMAIN, id: 2841\n;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1\n\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags:; udp: 1232\n; EDE: 3 (Stale Answer): ()\n\n;; QUESTION SECTION:\n;www5.example.com.\tIN\t A\n\n;; AUTHORITY SECTION:\nexample.com.\t30\tIN\tSOA\thidden-master.example.com. hostmaster.example.com.<PHONE_NUMBER> 28800 7200 604800 300\n"}}

{"type":"MESSAGE","identity":"resolver1.example.com","version":"unbound 1.19.1","message":{"type":"CLIENT_RESPONSE","response_time":"2024-04-03T10:20:45.913856Z","socket_family":"INET","socket_protocol":"UDP","query_address":"<IP_ADDRESS>","response_address":"<IP_ADDRESS>","query_port":19017,"response_port":53,"response_message":";; opcode: QUERY, status: NXDOMAIN, id: 45319\n;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1\n\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags:; udp: 1232\n\n;; QUESTION SECTION:\n;www5.example.com.\tIN\t A\n\n;; AUTHORITY SECTION:\nexample.com.\t0\tIN\tSOA\thidden-master.example.com. hostmaster.example.com.<PHONE_NUMBER> 28800 7200 604800 300\n"}}
{"type":"MESSAGE","identity":"resolver1.example.com","version":"unbound 1.19.1","message":{"type":"CLIENT_RESPONSE","response_time":"2024-04-03T10:20:45.913914Z","socket_family":"INET","socket_protocol":"UDP","query_address":"<IP_ADDRESS>","response_address":"<IP_ADDRESS>","query_port":19017,"response_port":53,"response_message":";; opcode: QUERY, status: NXDOMAIN, id: 45319\n;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1\n\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags:; udp: 1232\n\n;; QUESTION SECTION:\n;www5.example.com.\tIN\t A\n\n;; AUTHORITY SECTION:\nexample.com.\t0\tIN\tSOA\thidden-master.example.com. hostmaster.example.com.<PHONE_NUMBER> 28800 7200 604800 300\n"}}

Follow-up requests seem to only be logged once however.

The second one happened to have 0 TTL at that time so not expired yet.
Thanks for verifying what I noticed, we are in the same page then. I'll look for a fix.

This should be fixed now and it also matches @fhriley's initial issue.

Hmm the new test now fails... I'll need to have a second look.

Nope, it works. Local testing gone wrong. But @eest, could you also verify to be sure? :)

OK closing then. Thanks for helping pin point the issue and the verification!


⚠ Mobitopup has degraded performance

In 3ebfde4, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 6782 ms

Resolved: Mobitopup performance has improved in 996ee69.


Stagger the checking of organisation edition links

This commit reduces the load incurred by checking all the links in editions for every organisation by staggering the calls to the CheckOrganisationLinksWorker.

It's worth noting that this should help to prevent the impact of the continuous "broken link checker in progress" box since the link reports will be marked as "in progress" much closer to the time when they are likely to get a response back from the link checker api.

Actually, I've just thought... if each worker takes 16 minutes to run, will this solve the problem? Since they're likely to run with minute gaps anyway.

I believe we have about 1000, so this would run over 17 hours.

Closing in favour of #3631


Whether the epochs is set to 200 or 300，there will be overfitting

epochs set to 300:  when training to about 200 epochs, the map starts to drop.
epochs set to 200:  when training to about 120 epochs, the map starts to drop.
Any suggestions?
thx!

You may increase this value to 0.1 or 0.2 and try again.
https://github.com/Megvii-BaseDetection/YOLOX/blob/69f1173a569f1e0cc41025146a32b5d5fa06dd26/yolox/exp/yolox_base.py#L58

@Joker316701882 thx！ I have set  self.min_lr_ratio = 0.1 according to your suggestion  in wechat  and trainning.

@xuezu29   hi ,  how your model after adjust max_epochs and min_lr_ratio,  is normal ?
i adjust max_epochs and min_lr_ratio as @Joker316701882  said, but still fast convergence and drop after some epochs, like below

and even convert to mutilstep lr schedule as yolov3 and yolov4， but still not available /(ㄒoㄒ)/~~7

@PyNancy @Joker316701882 It does work though it's not perfect.

@xuezu29 I think you have obtained a better model compared to the previous one. But I don't understand the meaning of "perfect", the mAP is not satisfactory?

@xuezu29  your num_classes is 1? i think model AP should be stable with a little jitter instead of  AUC drop, @Joker316701882  is right ?

@PyNancy It depends. If overfitting happens, the performance will drop.

@Joker316701882 Yes, it is much better than the previous model, thx a lot!  It drops a lot at the end with no data augmentation.

@PyNancy my num_classes: 9

@xuezu29 This is an interesting phenomenon. You may reduce the no_aug_epochs to 1 or 0. But anyway, congrats on getting a better model.

@PyNancy In your case, I think you did not properly use this project. You may need to trying to find the misuse of our code first.

@xuezu29 This is an interesting phenomenon. You may reduce the no_aug_epochs to 1 or 0. But anyway, congrats on getting a better model.

Sincere thanks!

@PyNancy In your case, I think you did not properly use this project. You may need to trying to find the misuse of our code first.

my datasets is single class and small object(4 pixel ~ 50 pixel) and big image (1k), maybe due to my datasets is special, but loss seem normal,

Anyway, thank your great project and answer in time

@PyNancy Did you pull

@PyNancy Did you pull our latest branch? We recently fixed a fatal bug for detecting small objects.

yes, i pull the latest branch, and fix a fatal bug that bbox w,h limit  and rectangle train

@xuezu29 hi , how your model after adjust max_epochs and min_lr_ratio, is normal ? i adjust max_epochs and min_lr_ratio as @Joker316701882 said, but still fast convergence and drop after some epochs, like below

and even convert to mutilstep lr schedule as yolov3 and yolov4， but still not available /(ㄒoㄒ)/~~7

sorry, could you please tell me how to add the pictures of F1 ,precision and recall in the Tensorboard.


pybabel fails to update under python-3.3

TypeError is being thrown:
$ pybabel update -i messages.pot -d translations/
updating catalog 'translations/en/LC_MESSAGES/messages.po' based on 'messages.pot'
Traceback (most recent call last):
  File "/home/georg/env/main/bin/pybabel", line 9, in <module>
    load_entry_point('Babel==1.3', 'console_scripts', 'pybabel')()
  File "/home/georg/env/main/lib/python3.3/site-packages/babel/messages/frontend.py", line 1151, in main
    return CommandLineInterface().run(sys.argv)
  File "/home/georg/env/main/lib/python3.3/site-packages/babel/messages/frontend.py", line 665, in run
    return getattr(self, cmdname)(args[1:])
  File "/home/georg/env/main/lib/python3.3/site-packages/babel/messages/frontend.py", line 1130, in update
    width=options.width)
  File "/home/georg/env/main/lib/python3.3/site-packages/babel/messages/pofile.py", line 444, in write_po
    _write(comment_header + u'\n')
  File "/home/georg/env/main/lib/python3.3/site-packages/babel/messages/pofile.py", line 388, in _write
    fileobj.write(text)
TypeError: must be str, not bytes

Still getting the same error when running Babel under Python 3.4

I can confirm that po-file generation with Python 3.4 is broken. Following @georgthegreat fixes this.
Does anyone know how to make a new release for Flask-Babel that contains this fix and then be able to install it with pip? I could live with having to install it by pointing to a git repo.

+1, please make a release

please make a release, it still does not work with the last release in pypi :-1:

This should be fixed now.


QueryAtom::setQuery() should delete the old query first

This is what QueryBond::setQuery() does and makes more sense than requiring the caller to remember to do the delete in order to avoid a leak
This could theoretically cause problems if there is any client C++ code that is using QueryAtom::setQuery() and being careful about deleting the existing query. This doesn't seem overly likely.

@bp-kelley : this bug is causing intermittent failures for me when doing some testing as part of prepping for the release. I'd like to merge it relatively quickly to fix those. Would you mind taking a quick look?

This looks good to me as long as we have found all the setQueries int the source code.
It might make sense to make the container a shared_ptr in the future, this might make the python API easier to implement.

I'm going to merge, be we need to make this a release note that the behavior has changed.

I did go through the code and am pretty sure I found all the instances, so I think that's ok.
Putting something in the release notes is a good idea. I'll do that.


[WIP] Download IDAKLU from pybammsolvers

Description
This will separate the IDAKLU C++ code from pybamm.
Fixes #3564
Type of change
This should speed up CI by skipping the build of the C++ code.

[x] Optimization (back-end change that speeds up the code)

Key checklist:

[x] No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
[x] All tests pass: $ python run-tests.py --all (or $ nox -s tests)
[x] The documentation builds: $ python run-tests.py --doctest (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ python run-tests.py --quick (or $ nox -s quick).
Further checks:

[x] Code is commented, particularly in hard-to-understand areas
[x] Tests added that prove fix is effective or that feature works

A new link error cropped up, but it looks like we could get a lot of savings on time with this update

The linkage error is the same one as #3783, coming from CasADi's plugin system. I am not sure if it's worth fixing it, since it was fixed by @martinjrobins for the linear interpolant case by dropping down to Python but IIRC there wasn't a way in CasADi for doing it for the cubic

@agriyakhetarpal Yeah I was looking at that issue as well. As far as I can tell CasADI sets a path for plugins. I am trying to see if there is a decent workaround since this was part of #4464
My guess is that the wheels for the next release will be broken as well, but I have not confirmed it yet

There is a workaround for Linux and macOS, but not for Windows (different toolchain); sadly, it's not decent enough to include. I think I'll raise a PR upstream in CasADi to get one part of the linkage going and see if we can migrate to a non-MSVC toolchain (which can potentially help fix this later on). It's been on my list of things to do for a while, but I've yet to do it.

This is fixed locally with this: export CASADIPATH=.venv/lib/python3.12/site-packages/casadi

This is fixed locally with this: export CASADIPATH=.venv/lib/python3.12/site-packages/casadi

Yes, won't work with Windows

I had a look at this. The linker error is the same as I came across for the case of linear interpolation. The solution there was to swap to using the direct casadi function rather than their plugin system, which won't work if the casadi function is evaluated in C++ for windows as we compile everything statically.
I think I might be able to access the direct bspline interface by calculating the spline coefficients in scipy and then use the casadi Function.bspline function to construct a bspline. Cross fingers this doesn't use the plugin system anywhere!

I had a look at this. The linker error is the same as I came across for the case of linear interpolation. The solution there was to swap to using the direct casadi function rather than their plugin system, which won't work if the casadi function is evaluated in C++ for windows as we compile everything statically.
I think I might be able to access the direct bspline interface by calculating the spline coefficients in scipy and then use the casadi Function.bspline function to construct a bspline. Cross fingers this doesn't use the plugin system anywhere!

Yeah I was going to approach this by seeing if I could just change the build itself. It is something that should work if we are compiling and delivering everything correctly. If that does not work, then I will look at workarounds for interpolation

I expect to work on this again next week, I have been caught up with other stuff

looks like there is still issues with the idaklu jax solver on windows, I can look into these?

I tried to figure this one out today but no luck :( It's crashing with a fatal exception when jax tries to jit compile, I'm still in the dark as to why. It might be a threading issue as the problem is intermittant (occurs in about 95% of test runs). It might be cause with some interaction with pytest because when I copy the test into a stand-alone script it works fine

For a stopgap solution, we can isolate these tests into their own xdist_group and allow only one worker to touch them at a time.

For a stopgap solution, we can isolate these tests into their own xdist_group and allow only one worker to touch them at a time.

Yeah that is my fallback option.
I want to take a closer look at the linking/delivery as well. We have failures on windows when you download the wheels:

pybammsolvers has some crashes workers
my i5 (without AVX-512 instructions) has ~45 test failures on both 24.9.0 and 24.11.0 when running tests with the wheels
A colleague's i7 (with AVX-512 instructions) has ~25 test failures on both 24.9.0 and 24.11.0 when running tests with the wheels

So it appears that the tests are working when you test in the build environment, but not in a different environment. I will be digging into this more and see what I come up with

Hi - I took a very quick look at this yesterday and agree that it seems to be a threading issue. More specifically, jaxify() can only be called once per solver instance (this is the first test), which then caches the full solve result so the jax-wrapper can query samples without repeatedly re-running the solver. My suspicion is that running these tests in parallel is causing test pollution, probably because the test script currently instantiates the solver and jax wrapper objects at the start of the test script, not as a fixture for each test (although the ubuntu tests should also be failing?). Refactoring the tests with fixtures would be good to see if that resolves things - I can take a look at that if you like but if I'm right then the xdist_group solution should also work if you need a quick fix. I can't remember the precise details as to why we can't jaxify more than once per object, but I do remember that it was more complex than just the cache issue (something to do with the jax primitives...).

Testing out the skips and test refactor now with CI. I have some docs stuff to update then this should be mostly ready to go. I will do one more update to pybammsolvers to make sure that the versions of IDAKLU match

Tests pass, just need to do some documentation fixes

Additional documentation will be added to the pybammsolvers repo

@MarcBerliner, @martinjrobins Ok I think this is finally working. I am working on tests and docs for the other repo now


fix: reset cache

The cache is currently broken, see Zulip discussion. This should refresh the bad cache files. Everyone with PRs that fail to get cache hits should rebase on top of this commit.

bors delegate+
Ping me when you remove the whitespace.

@mattrobball Actually the intent is to commit the whitespace, and only remove it in the next dependency bump. The way it works is that changing whitespace in the manifest causes the hash to change, and that ensures that cache doesn't attempt to pull the bad files existing at the old hash. Removing the whitespace would go back to the original cache and end up having no effect, but the next change to the file for a dependency bump will make the whitespace no longer necessary.
But actually I think I misdiagnosed the issue and this PR is not necessary. The problem is that the lean version used to compile the old cache files (the bad one) is not the same as the one currently in the elan toolchain directory of the runner machines (the right one), but this is also equivalent to resetting the cache; the next CI run on everyone's branches (including manual re-runs) will fail to get any cache hits and generate the new correct files.


Modal error when reloading with dirty notebook

Testing #180767

After the profile switching, reload the window immediately, test if the warning/error actions on the editor view gives you clear guidance

Before the reload occurred, this error blocked it:

This can happen if a change to the notebook and a change of the profile happened fast enough that we did not have a chance to complete the backup of the notebook to disk. In that case there is really nothing we can do to prevent data loss on shutdown and have to prevent it from happening.
To clarify, is that what was happening here? Very fast profile switch after making a notebook dirty?

I can reproduce easily.
@rebornix this is maybe notebooks could still improve next month. When I debug this I see an issue that makes the backup service bring up a dialog like this. Let me clarify what happens.
Normally, the backup service would not complain about a missing backup, when a backup exists that matches the content version of the notebook (which we track internally). Here, a backup actually exists for that content and thus I was wondering why the dialog appears anyway.

Given this content change event, the working copy backup handler will try to perform another backup, even though the content did not change. And since that fails (because in the new profile there is no notebook extension installed), the backup handler will complain when you try to exit or reload.
Is it possible to not emit the content change event here when profiles change?

@bpasero thanks for the analysis!

I think originating from a setLanguageId call for a cell.

This is a bit unexpected and should be fixed. At the same time, I agree that we shouldn't avoid content change when switching profiles.

This is an interesting issue. After switching to a new profile, which lacks of the GitHub issue notebook extension, we can't set a proper language on the cell anymore. The code cell used to have language "GitHub-issues" but now it's set to plain text as a fallback, Thus leading to the language update and triggering backup.
It's not trivial to fix and once we have https://github.com/microsoft/vscode/issues/180514, this should not happen. Let's track it over there.

@rebornix but why is a language change considered a content change? is the language part of the backup?


Add table gcp_cloudrun_service

References
Add any related links that will help us understand the resource, including vendor documentation, related GitHub issues, and Go SDK documentation.
Be able to list out metadata around Cloud Run deployed services.
https://cloud.google.com/run/docs/reference/rest#rest-resource:-v1.namespaces.services

@slagathorr Appreciate trying out Steampipe.
We will definitely look into this.
Do you use Steampipe now for any use-case and particularly looking for this table? What query do you have in mind to use? This information will help us when we start designing and prioritizing the same. Thanks for the issue.

We are looking to analyze our entire deployment against billing data, and exploring using Steampipe to access the deployment data. We use Cloud Run quite frequently, so having access to this will be helpful.

@slagathorr FYI, we are still checking how we can bring this, already engaged with the GCP support. We will keep posted.


Address data loss during block rotation when there are out of order writes

It appears when ingesters are retrying writes, sometimes the data in the same block during which the retries were happening are lost.

Fixed by #155


Husky does not seem to be working

Description

Husky should be triggered on every commit, and it should format the code that is committed. However, according to the commit logs, it does not seem to be working.
Steps to Reproduce

Add unnecessary multiple line breaks or purposely break formatting, and commit.
If the style done in step 1 was not formatted, it means the husky is not triggered.

Environment

OS: MacOS
Device: MacBook Pro M2 16-inch

Notes

I discovered the issue myself. It was caused by an incorrect Git Hook path configuration in GitKraken, a GUI git manipulating tool.


Add project descriptions for all build-logic related builds

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

OK, I've already triggered a build for you.


Use asyncio or multithreading for requests

I think requests should be called with multithreading/asyncio instead of like so: https://github.com/David-Woroniuk/Historic_Crypto/blob/53b46d50c4ec5295c2e5d2036c0187e6d20acb05/HistoricalData.py#L163
Could you elaborate on why you're doing it this way? Has this got something to with rate limits?

Whilst I would use asyncio, the Coinbase API does not handle asynchronous requests well, and has a maximum of 300 per message, as per their documentation

To limit this, I implemented using the below:
        else:
            # The api limit:
            max_per_mssg = 300
            data = pd.DataFrame()

Full docs:
https://docs.cloud.coinbase.com/exchange/docs/rest-rate-limits

Closing as no further interaction. 12:15 27/10/2022


Will an optimizer like RMSprop keep state across model.fit() calls?

I'm building my model with optimizer="rmsporp". Since my dataset is quite large, training on the entire dataset for an epoch takes around one day, so I'd like to be able to stop training at any point but still keep whatever "progress" the model has made in the last few hours. For this, I manually split the data into smaller chunks that I train on for one "epoch" at a time via model.fit(), so that validation testing (via ModelCheckpoint) is done and model weights stored more frequently.
While this seems to work fine in practice, I am now wondering whether the optimizer will do its job correctly if I train the model in this fashion. Will the optimizer be reset on every call to model.fit() ? In this case, gradient updates magnitudes would probably be suboptimal.
The alternative would be to write a model callback that does validation testing in the style of ModelCheckpoint, but does not wait for the whole epoch to finish. Is this the only "right" way to do it, or will my method work fine?

I took a look at the code, and it seems like the optimizer state is never 'reset' for a given Model instance.

Yes. The optimizer state isn't reset until you recompile the model with a new optimizer.


Use FileSystems.exist to check the remote the path

fixes: https://github.com/apache/beam/issues/30062

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

R: @ LaurenzReitsam @riteshghorse


Bug in appendix Substitution

Section The σ algebra equations contains the equation
(sub-abs)   ⟪ σ ⟫ (ƛ N)    ≡ ƛ ⟪ σ ⟫ N

This is wrong, as is the subsequent discussion ("The equations sub-app and sub-abs says that substitution is a congruence for the lambda calculus. ") Later, the correct equation is stated and proved.
sub-abs : ∀{Γ Δ} {σ : Subst Γ Δ} {N : Γ , ★ ⊢ ★}
        → ⟪ σ ⟫ (ƛ N) ≡ ƛ ⟪ (` Z) • (σ ⨟ ↑) ⟫ N

Thanks for noticing this! I've made a pull request to fix it.


🎉 2.0.2

ℹ About this release

Version: 2.0.2
Type: patch
Last Release: 2.0.1 (11/10/2021, 8:00:33 PM) [?]
Commits to merge: 9 [?]

🐛 Bug Fixes

Remove quotation marks from file name (7a0df3c)

📦 Development Dependencies

Update @types/node from ^16.11.6 to ^16.11.11
Update @typescript-eslint/eslint-plugin from ^5.2.0 to ^5.4.0
Update @typescript-eslint/parser from ^5.2.0 to ^5.4.0
Update eslint from ^8.1.0 to ^8.2.0
Update eslint-plugin-jsonc from ^1.7.0 to ^2.0.0
Update mochawesome from ^6.3.1 to ^7.0.1
Update source-map-support from ^0.5.20 to ^0.5.21
Update typedoc from ^0.22.7 to ^0.22.9
Update typescript from ^4.4.4 to ^4.5.2

:tada: This PR is included in version 2.0.2 :tada:
The release is available on:

v2.0.2
GitHub release
Docker container (sebbo2002/iubh-campus-sync:2.0.2)
v2.0.2

Your semantic-release bot :package::rocket:


build(publish): publish to Open VSX Registry

build(publish): add step to publish to Open VSX Registry
Description

Updates the publish workflow to also publish to the Open VSX Registry
This is the marketplace used by Salesforce Code Builder and will allow the analyzer to be used from Code Builder.

It is an open source alternative to vscode marketplace.
https://open-vsx.org/
Type of change

[ ] Bug fix
[X] New feature
[ ] Breaking change

Changes

Update publish.yml to publish to Open vsx registry

resolves #23

Can not merge until we have complete the setup steps
https://github.com/eclipse/openvsx/wiki/Publishing-Extensions#how-to-publish-an-extension


Issue 127: Organization Page Layout Tweaks

Issue #127
Also sorts inactive orgs last - arguable we should not show them?

@bogdankharchenko @zach2825 I notice the Organization pages have the sections showing in a different order between stage and live.
Bogdan made mention of npm build prod, above, but I'm not sure if that was intended to be run and I don't see a 'prod' in the package.json.
Do I need to run that, or some other command, to get these Orgs sections to sort the same as they are now on stage.
I think the way it sorts inactive to the bottom, as on the current stage, is the ideal.
https://stage.hackgreenville.com/orgs

https://hackgreenville.com/orgs


Remove failed push tokens

We've been seeing a rash of failed AWS SNS push failures with Android GCM. In order to better understand these we implemented a failure top subscription endpoint to remove failed device tokens.
This will hopefully reduce the number of failures and/or allow us to better understand the nature of the failures. The hypothesis is that GCM is frequently issuing new device tokens for existing devices, causing our notification service to fill up with invalid tokens.
We'll need to add the topic subscription in AWS before this gets merged.

This could use a review @jayzes. Disregard all of the annoying hound comments. They came before the mothership rubocop.yml was brought over.

@steam i adjusted a few things to be slightly more idiomatic in https://github.com/ello/ello-notifications/pull/10/commits/80fe9eb079da404100b7ea0cdcdeb6d74423a947, but LGTM otherwise!

Thanks @jayzes :metal:


Hosted Fields Instance focus is not working properly on iOS 13

General information

SDK version: 3.52.1
Environment: Sandbox and Production
Browser and OS: iPhone XS | Safari | iOS 13
"Mozilla/5.0 (iPhone; CPU iPhone OS 13_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0 Mobile/15E148 Safari/604.1"

Issue description
Hi, it looks like there is an issue with autofocus on new Safari in iOS 13.
What we have implemented is that when the user fills out the credit card number field and it is determined as valid it immediately changes the focus field to next one (ex. date, then from date to CVV).
We have successfully accomplished that and it was working as expected on previous iOS versions and desktops.
Here are the code parts that are responsible for this:
Calling a function makeFieldValid() on validity change:
hostedFieldsInstance.on('validityChange', function(event) {
    var field = event.fields[event.emittedBy];

    if (!field.isValid && !field.isPotentiallyValid) {
        // Case when there is an error
        makeFieldInvalid(event.emittedBy);
    } else if (!field.isValid && field.isPotentiallyValid) {
        // Case when it's neutral
        makeFieldNeutral(event.emittedBy);
    } else {
        // Case when it's valid
        makeFieldValid(event.emittedBy);
    }
});

makeFieldValid() function sets focus to next field if determined as valid:
function makeFieldValid(field) {
    var $fieldContainer = $(hostedFieldsInstance.getState().fields[field].container);

    $fieldContainer.next().html(getFieldLabel(field));
    $fieldContainer.next().css('color', 'green');
    $fieldContainer.css('border', '1px solid green');

    if (field == 'number') {
            hostedFieldsInstance.focus('expirationDate');
    } else if (field == 'expirationDate') {
            hostedFieldsInstance.focus('cvv');
    }
}

And here’s the on focus function for hosted field:
hostedFieldsInstance.on('focus', function(event) {
    var field = event.fields[event.emittedBy];
    $(field.container).next().addClass('focus');
});

When hostedFieldsInstance.focus is called the focus field is not changed although the on focus function is still called which can be seen when the field label changes its position and also testing with breakpoints in dev console shows the same.
Interesting is that it works as expected if "next" field has been in focus mode before, but not on the first time when that is necessary.

Thanks for the report, I was able to reproduce the issue on iOS 13.
iOS Safari has always had a quirk with how focus is handled, since you can focus an input just fine, but only a real touch event can bring up the keyboard, making it useless for that input to be focussed if the keyboard isn't displayed.
In previous iOS versions, you could focus an input as you are doing, if the keyboard is already open. I'm afraid the underlying bug is going to be in the version of Safari installed on iOS 13 devices, but we will do some investigation and see if a workaround can be provided, or if we need to submit a bug report to Apple.
In addition, it looks like Safari in iOS 13 also broke our mobile tabbing code. We'll have to look into that as well.

Any update on this?

No updates at this time. I'm afraid there's not much we can do if iOS Safari is preventing the focus of inputs inside of iframes.

Apple has updated desktop safari to prevent focussing as well. See https://github.com/braintree/braintree-web/issues/490#issuecomment-658362948
I believe it's the same underlying issue, that Apple wants to prevent inputs from being focussed without a user action, but since the user action triggers a post message to the iframe, there's no way to connect the focus request in the iframe to the original user action. 😞
We are reaching out to Apple about this to see if there's anything that can be done.

Hi @crookedneighbor  just wanted to follow up on this issue. We are seeing that when a user clicks on a label for a hosted fields, focus is not automatically sent into the input field. We are using the floating label design pattern, so this bug actually prevents users from selecting a hosted field unless the user clicks on the part of the field not covered by the label.  Do you think this is the same underlying issue focus issue?
I am able reproduce this behavior in Braintree's example codepen: https://codepen.io/braintree/full/mPgdPN.

It is.

After almost 2 years of this, we've finally found a fix! Released in v3.78.2

I'm still seeing this issue when the braintree credit card payment form is being loaded up through an iframe on an iphone 11


Request for change in YAML format

Currently, the questions are stored as:
questions:
   title: ...
   .
   .

   title: ...
   .
   .

This leads to a complicated parsing with template languages (such as Liquid used in Jekyll) due to no separation of levels between questions
A better format according to me would be:
- title: ...
  .
  .

- title: ...
  .
  .

This would have a simple parsing template as follows:
{% for question in site.data.ml %}

{{ question.text }}

{% for opt in question.opt %}

- {{ opt }}

{% endfor %}

{% endfor %}

In the proposed format, would other keys be nested inside the title key?

No. Something like this:
- title: ...
  type: ...
  text: ...

- title: ...
  type: ...
  text: ...

Well, it makes sense to do this if the parsing template simplifies. Also, it makes logical sense to do this, since each question is a list item, and each list item must begin with a -.


Carousel cleanups

A couple of minor code cleanups in the Carousel component.

Tests failed. Automated cross-browser testing via Sauce Labs and Travis CI shows that the JavaScript changes in this pull request are: BUSTED
Commit: c42f351f29d97122493936e19fababee1eea6bac
Build details: https://travis-ci.org/twbs-savage/bootstrap/builds/73552891
(Please note that this is a fully automated comment.)

Seems to have failed on OS X Chrome, but that might just be due to Sauce flakiness.

@twbs-savage retry

Tests passed. Automated cross-browser testing via Sauce Labs and Travis CI shows that the JavaScript changes in this pull request are: CONFIRMED
Commit: c42f351f29d97122493936e19fababee1eea6bac
Build details: https://travis-ci.org/twbs-savage/bootstrap/builds/73609155
(Please note that this is a fully automated comment.)

Oh ok, didn't know that. I've reverted my commit in 8e80624 now.

@fat You okay with the $.proxy change?

Tests passed. Automated cross-browser testing via Sauce Labs and Travis CI shows that the JavaScript changes in this pull request are: CONFIRMED
Commit: 8e80624e8f32ab120db495575a2a9af6b5e1494e
Build details: https://travis-ci.org/twbs-savage/bootstrap/builds/73669018
(Please note that this is a fully automated comment.)

Not sure about the proxy change. But the rest I like. Perhaps @jonathanp could split those to a separate PR?

Bootstrap 3 is no longer being officially developed or supported.
All work has moved onto our next major release, v4. As such, this issue or pull request is being closed as a "won't fix." For additional help and support, we recommend utilizing our community resources. Thanks for your understanding, and see you on the other side of v4!
<3,
@mdo and team


IsCapture flag is false for enpassant moves

For example after the moves

e4 Nf6
Nf3 Rg8
e5 d5

The enpassant move to d6 has the flag IsEnPassant true, but the move is also a capture, thus I think
it should also have IsCapture true, which atm. is false.

Hi, could you please make sure that you have updated the project to the latest version, I believe this bug was fixed in 1.15

Seems to be working in the latest, btw. latest seems to be much slower than 1.12 that I was previously using.


quirks with installing multilingual embeddings model in zep-nlp

I use a docker container install. It boots up as expected. When I pass in  jinaai/jina-embeddings-v2-base-de as ZEP_EMBEDDINGS_MESSAGES_MODEL, it tries to execute some custom code (as intended), the error message that shows up is

ValueError: The repository for /root/.cache/torch/sentence_transformers/jinaai_jina-embeddings-v2-base-de/ contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co//root/.cache/torch/sentence_transformers/jinaai_jina-embeddings-v2-base-de/.

Please pass the argument trust_remote_code=True to allow custom code to be run.

2024-09-27 12:00:05,326 - uvicorn.error - ERROR - Application startup failed. Exiting.

I tried to pass in trust_remote_code=True as environment variable and as COMMAND in the docker-compose file, without succes.
When I pass in jinaai/jina-embeddings-v3 as desired multilingual model, it gives me

You try to use a model that was created with version 3.1.0, however, your version is 2.2.2. This might cause unexpected behavior or errors. In that case, try to update to the latest version.
and
ModuleNotFoundError: No module named 'custom_st'

Please, do you see a shortcut to pass in the trust_remote_code=True parameter?
(Besides, your webserver is on a "redirect every link to the same page" trip today.)

Hi there - we're unable to support v0.x of Zep Open Source and the related NLP service is deprecated.
The documentation may now be found here: https://help.getzep.com/ce/legacy/deployment/quickstart
The parameter you mentioned needs to be added to the model loader code, not as an argument to the container.
I recommend investigating the recently released Zep Community Edition. It has a simpler setup, is more accurate, and flexible. With a proxy such as LiteLLM, Zep CE is able to connect to many embedding services, including locally hosted.

Hi Daniel,
thank you for your fast reply. I will check the community edition.

Thank you!


crash at launch

crash-2022-12-27_<IP_ADDRESS>15-quilt_loader.txt
crash-2022-12-27_02.27.05-client.txt
latest.log

Hi, please ask support questions in the forum or in discord.
Thanks!


make pick more useful

julia> c = Columns(x=[1,2],y=[2.,4])
2-element IndexedTables.Columns{NamedTuples._NT_x_y{Int64,Float64},NamedTuples._NT_x_y{Array{Int64,1},Array{Float64,1}}}:
 (x = 1, y = 2.0)
 (x = 2, y = 4.0)
julia> pick(:y)(c)
2-element Array{Float64,1}:
 2.0
 4.0

julia> pick(:y)(c)
2-element Array{Float64,1}:
 2.0
 4.0

julia> pick((:y,))(c)
2-element IndexedTables.Columns{NamedTuples._NT_y{Float64},NamedTuples._NT_y{Array{Float64,1}}}:
 (y = 2.0)
 (y = 4.0)

julia> pick((:y,:x))(c)
2-element IndexedTables.Columns{NamedTuples._NT_y_x{Float64,Int64},NamedTuples._NT_y_x{Array{Float64,1},Array{Int64,1}}}:
 (y = 2.0, x = 1)
 (y = 4.0, x = 2)

julia> pick((2,1))(c)
2-element IndexedTables.Columns{NamedTuples._NT_y_x{Float64,Int64},NamedTuples._NT_y_x{Array{Float64,1},Array{Int64,1}}}:
 (y = 2.0, x = 1)
 (y = 4.0, x = 2)

The multiple column version is not inferable in the case the columns are of different types. That can be fixed by using Base.@pure if required.

If the case of manually writing out the desired fields is important, we could do it very efficiently with a macro @pick(a,b) that expands to
x -> @NT(a=x.a, b=x.b)

To extend that to Columns, we could instead expand to
x -> wrap(x, @NT(a=x.a, b=x.b))

where
wrap(x, y) = y
wrap(x::Columns, y) = Columns(y)

or something along those lines.

@pick

That's a neat idea! Gives us a chance to generate the required output type during macro expansion time (esp. since it's  not possible with @generated functions).
We specialize map(p::Proj, c::Columns) to call p only on c.columns, this is pretty fast at the moment for this reason.

I've updated the PR. Added @pick with exactly the syntax you suggested. I've retained the old pick which has a slightly different behavior. Specifically, pick(:x)(@NT(x=1, y=2)) == 1, but @pick(x)(@NT(x=1, y=2)) == @NT(x=1).

Tests now pass except on OSX nightly (same problem of missing native library or something).

Updated. Works in the multi-process case with https://github.com/blackrock/NamedTuples.jl/pull/34 , but safe to merge here as-is.

Codecov Report

Merging #44 into master will increase coverage by 0.78%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #44      +/-   ##
==========================================
+ Coverage   91.98%   92.77%   +0.78%
==========================================
  Files           6        6
  Lines         836      844       +8
==========================================
+ Hits          769      783      +14
+ Misses         67       61       -6

Impacted Files
Coverage Δ

src/utils.jl
98.19% <100%> (+4.02%)
:arrow_up:

src/columns.jl
94.57% <100%> (+1.55%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 063d41f...25c1b62. Read the comment docs.


Rename 5.show-create-tags-edges.md to 5.show-create-tag-edge.md

Why rename this file? @HarrisChu
The file name is not displayed in the content and there are other docs referring to this topic through its file name.

There's no show create tags command, and would be confused with show tags below.
I'm not sure if it depends on the file name

Thanks for the explaination. I get your point. See if this PR resolves the issue.