Prove kill sync bug

I'm demonstrating the problem like this because I couldn't think of an easy way to create a unit test that simulates this race condition.
DON'T MERGE THIS. If you agree, I will make another PR with the fix alone.
1st commit: Prove job status synchronization bug when killing
When job is killed, it's possible that it sends a JOB_FINISHED event with some other status but is eventually marked as KILLED.
For example this code modification demonstrates how it sends JOB_FINISHED with Status.SUCCEEDED but then marks the job as KILLED.
It should be ensured that if job has finished it won't be marked as killed and on the other hand it isn't marked killed if it has finished first.
Extract from FlowRunnerTest#execAndCancel output:
2017/08/11 11:25:22.889 +0300 INFO [main] [derived-member-data] Killing 1 jobs.
kill() called..
Sleeping before marking as SUCCEEDED
!Status.isStatusFinished(finalStatus) = true
changeStatus(Status.SUCCEEDED)!!
isKilled() = false
finalStatus = SUCCEEDED
setting killed=true

2017/08/11 11:25:26.946 +0300 INFO [JobRunner-job1-1] [derived-member-data] Job job1 finished with status SUCCEEDED in 1 seconds
2017/08/11 11:25:26.948 +0300 INFO [FlowRunner-exec-1] [derived-member-data] Flow '' is set to KILLED in 4 seconds
2017/08/11 11:25:26.949 +0300 INFO [FlowRunner-exec-1] [derived-member-data] Finished Flow
2017/08/11 11:25:26.949 +0300 INFO [FlowRunner-exec-1] [derived-member-data] Setting end time for flow 1 to<PHONE_NUMBER>949
setting Status.KILLED

testJob: job1 KILLED

2nd commit: Fix job status synchronization bug when killed
Now the test output has this, as expected:
kill() called..
2017/08/11 12:42:47.845 +0300 INFO [main] [derived-member-data] Flow killed by me
Sleeping before marking as SUCCEEDED
!Status.isStatusFinished(finalStatus) = true
setting killed=true
setting Status.KILLED
2017/08/11 12:42:51.912 +0300 INFO [JobRunner-job1-1] [derived-member-data] Job job1 finished with status KILLED in 4 seconds
isKilled() = true
Setting to Status.KILLED
finalStatus = KILLED
2017/08/11 12:42:51.919 +0300 INFO [FlowRunner-exec-1] [derived-member-data] Flow '' is set to KILLED in 4 seconds
2017/08/11 12:42:51.919 +0300 INFO [FlowRunner-exec-1] [derived-member-data] Finished Flow
testJob: job1 KILLED

@HappyRay, could you or someone have a look at this?

If you agree, I will make another PR with the fix alone.

Here is my understanding of the problem you described. Please correct me if I am wrong.
Users issue kill command for a flow.
Due to race conditions, the job may finish successfully and registered event listeners are notified as such.

Thread synchronization is tricky to get right. I am afraid I don't quite follow your diagnostics and how your fix will solve this problem.

I am thinking of refactoring the DAG processing engine to reduce the complexity and reduce the need for thread synchronization.  e.g. if there is one thread that is responsible for the job/flow status transition, it would be easier to reason about.
 The final outcome may still be either a job is killed or a job finishes successfully. The user or a user program would still need to poll the status to find out the final status. And there should not be a state transition from running -> killed -> success/failure.  It should either be running -> killed or running -> success/failure.
cc @jamiesjc who has worked in this area.

How big a problem do you think this is causing?

How big a problem do you think this is causing?

I think currently the worst result is that you get a job callback that says SUCCEEDED (also could be FAILED, I suppose), although if you go and look at the execution, eventually it may be marked as KILLED instead.
With the additional KILLING status patch applied it's much worse: hitting this bug causes the flow to be stuck because a job is left stuck in KILLING status.

I am afraid I don't quite follow your diagnostics and how your fix will solve this problem.

The core source of problem here is using the field Job.killed to decide how to set job status, but not synchronizing these operations. The code branch in Job#kill was already synchronized (it reads status, possibly writes killed flag and job status). However, Job#runJob was setting job status without synchronization (depending on the killed flag), so a race condition was possible.

e.g. if there is one thread that is responsible for the job/flow status transition, it would be easier to reason about.

Definitely. May be a lot of work though, so I would appreciate having this bug fix ASAP.

 The final outcome may still be either a job is killed or a job finishes successfully. The user or a user program would still need to poll the status to find out the final status. And there should not be a state transition from running -> killed -> success/failure. It should either be running -> killed or running -> success/failure.

I suppose you describe the situation how it is without this fix. Yes, that's how I see it pretty much. Is this just you ensuring that you understand how it works, or do you mean to say that this wouldn't be a real problem?

I suppose you describe the situation how it is without this fix. Yes, that's how I see it pretty much. Is this just you ensuring that you understand how it works, or do you mean to say that this wouldn't be a real problem?

I described it as the desired state.

The lack of synchronization does seem to be a problem.
Could you describe an example sequence to better help me understand the particular race condition? And how it will interact with the killing status change?
Are you seeing this issue in your production environment now?

Could you describe an example sequence to better help me understand the particular race condition?

Actually I tried to show that in the 2 scenarios (buggy vs. synchronized) of my PR description. It might help reading it to know that Job job1 finished with status.. is the line that tells what status was fired as a JOB_FINISHED event.
Please let me know if you were looking for some other form of expressing it.

And how it will interact with the killing status change?

When the KILLING status change is applied, this line will set status to KILLING instead of KILLED:
https://github.com/juhoautio/azkaban/blob/723a46ec93d6e52f178e5d2ab445d630099d0ad6/azkaban-exec-server/src/main/java/azkaban/execapp/JobRunner.java#L833
Then it would be possible that Job.run returns initially as SUCCEEDED, while Job.kill is being executed, but then Job.kill proceeds, setting status to KILLING. If the job runner thread proceeds only at this point, this branch is skipped:
https://github.com/juhoautio/azkaban/blob/723a46ec93d6e52f178e5d2ab445d630099d0ad6/azkaban-exec-server/src/main/java/azkaban/execapp/JobRunner.java#L597-L604
As a result the job exits with status KILLING and flow won't be able to terminate.

Are you seeing this issue in your production environment now?

It was seen, but it's been patched in our fork (as in the second commit of this PR).

I described it as the desired state.

Oh, now your description is clear to me (how could I think about it that much wrong..). Indeed that's what my fix here ensures.

Yes, I have looked at your example commit. It's great to have a repeatable program to demonstrate the problem.  The current code's complexity doesn't make it easy for me to remember all the details in my head. I am not very familiar with your killing status change. It would take me some time to research the code to understand.
In the mean time, it will help me understand if you can describe the sequence of thread A does ... Thread B.... and here is the race condition. Here is how ... affect the timing...
Here is an example of how I describe race conditions. It's a style that I use.
https://github.com/azkaban/azkaban/commit/79f1f9927c151e226e9db601eaffc161df05ea8a

My local build failed when I synced to your first commit: Prove job status synchronization bug when killing juhoautio 8/11/17, 2:31 AM
I am not able to run the test.

I've read through the changes and descriptions. Correct me if I misunderstood anything. @juhoautio
The main problem here is that the kill thread is synchronized but the job run thread is not. And both threads will change the job status which causes the race condition.
One race condition happens in below scenario:

jetty thread:
azkaban.execapp.JobRunner#kill
azkaban.executor.Status#isStatusFinished -> status is not finished, does not return
switch to jobRunner thread:
azkaban.execapp.JobRunner#runJob
azkaban.execapp.JobRunner#changeStatus(azkaban.executor.Status) -> change status to SUCCEEDED
azkaban.execapp.JobRunner#fireEvent(azkaban.event.Event, boolean) -> fire JOB_FINISHED event
switch back to jetty kill thread:
set this.killed to true
azkaban.jobExecutor.Job#cancel -> do the actual kill
azkaban.execapp.JobRunner#changeStatus(azkaban.executor.Status) -> change status to KILLED

The fix is mainly to synchronize the places where #changeStatus would modify the job status incorrectly from different threads, e.g. changing it to SUCCEEDED from jobRunner thread first and then to FAILED from jettry kill thread. I think the fix makes sense. But it's definitely not easy to understand and maintain in the future. I would vote for @HappyRay's suggestion of reducing the overall complexity of the code if this is something we want to move towards finally.

I am thinking of refactoring the DAG processing engine to reduce the complexity and reduce the need for thread synchronization. e.g. if there is one thread that is responsible for the job/flow status transition, it would be easier to reason about.

Thanks @jamiesjc
I can follow the race condition you described.
How would the change fix this race condition? I don't see what's preventing the jetty thread from setting the final status to killed.

I don't see what's preventing the jetty thread from setting the final status to killed.

jetty thread returns if status is a finished status. On the other hand if jetty thread gets to setting the status, then jobRunner thread won't set it any more, because it also checks if the status is finished (or if the job is killed), before changing the status.

I didn't realize that the whole logic in the kill azkaban.execapp.JobRunner#kill method is within this lock synchronized (this.syncObject) {
Your fix makes sense to me now.
The code is still overly complex since the lock is all over the place.
Could you try to summarize your technique in preventing the race condition between the kill thread and the jobrunner thread?
My understanding is that you would check killed status before setting the final job status and place that check and set logic inside the jobrunner syncObject lock?
Thanks for your contribution and patience!

Could you try to summarize your technique in preventing the race condition between the kill thread and the jobrunner thread?
My understanding is that you would check killed status before setting the final job status and place that check and set logic inside the jobrunner syncObject lock?

Replace "check killed status" with "check killed flag" and we're there.
To sum it up in other words, I fix the problem by synchronizing in run & kill methods so that the combination of these operations are wrapped into "a single atomic operation":

checking the status and/or killed flag
based on the previous check result(s), setting the status and/or killed flag

@juhoautio I can review and merge it if you could you update the change description and update the branch.

Would you squash the commits when merging? I guess I don't need to create a new PR in that case :)

Yes, the default merge option now is to rebase.
Could you suggest the change description you would like to use?

Could you suggest the change description you would like to use?

Ok, let's see what you think about this (I edited the PR description):
https://github.com/azkaban/azkaban/pull/1330#issue-249587802
Feel free to edit it if there's anything.

Hm, looks like master branch build is failing.

It looks like there was also a random failure with FlowRunnerTest > exec1Failed.
This new PR aims to stabilize it: https://github.com/azkaban/azkaban/pull/1364


setup-kind action: add feature-gates input

Hello :wave:
First, thanks for the setup-kind action, it's very useful to remove boilerplate in all repositories requiring KinD.
For one of my projects, I was missing the featureGates parameter in my KinD config with this action. This is something possible (see docs about feature gates), so I've decided to add it here and contribute upstream to also benefit others.
With this change, we can define one or multiple (or zero) feature gates, separated by commas:
- name: Setup KinD with 1 feature gate
  uses: norbjd/actions/setup-kind@add-feature-gates-to-setup-kind
  with:
    k8s-version: 1.28.x
    kind-worker-count: 1
    feature-gates: InPlacePodVerticalScaling,HPAScaleToZero,ConsistentListFromCache # NEW!

And this will create a KinD config with these feature gates enabled:
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
# existing stuff, I have not touched that
featureGates:
  InPlacePodVerticalScaling: true
  HPAScaleToZero: true
  ConsistentListFromCache: true

I have created a test repository using this action to see if my change works correctly: https://github.com/norbjd/test-setup-kind-action/blob/main/.github/workflows/test.yaml.
I have tested the following cases:

use 1 feature gate: https://github.com/norbjd/test-setup-kind-action/actions/runs/6677787095/job/18148138768#step:2:188 :heavy_check_mark:
use multiple feature gates: https://github.com/norbjd/test-setup-kind-action/actions/runs/6677787095/job/18148138861#step:2:187 :heavy_check_mark:
don't use any feature gate (just to catch regressions): https://github.com/norbjd/test-setup-kind-action/actions/runs/6677787095/job/18148138963#step:2:186 :heavy_check_mark: (no featureGates section in yaml)

Hope this is something you can consider adding in that action.
Thanks! :pray:

Hello @cpanato, just a kind up :innocent:
Are you happy with the last changes? Otherwise, feel free to tell me! Thanks :slightly_smiling_face:


Make single-pod server setups easier

From https://github.com/solid/community-server/pull/815#discussion_r672869143:

The root container is always going to exist (and has to since it's the root folder if we use a file system). You can use that container as your personal pod. But that requires you to manually put the correct resources there to have a valid acl and webid. You would also have to manually register with the IDP. With manually I mean the registration solution where you manually add an identifying triple to your webid for the IDP. There is no way for the IDP to "generate a pod" in the root container. If you generate a pod with the IDP registration form it is always going to be in a subcontainer (either alice.test.com or test.com/alice depending on your config).
The one thing you can do is disable the IDP so no other pods can be made, but then you also can't use your own server as IDP for your own pod since those are tied together. Might be an idea for the future to have a configuration that allows having an IDP just to identify your own pod that does not allows registration for new pods. (Would be quite easy actually, just have to remove the registration handler from the list of handlers).
If we want the user to immediately have a pod with valid WebID in the root container when they boot the server, we would need a new Initializer for this. Users could then make a custom config that includes this initializer if they want to set up a server with a pod in the root container.

FYI: NSS can be run in single- and multipod mode.
The only blocker here seems to be our IDP; perhaps we need an identifier strategy for a single-pod setup (which could also block registration once that setup has been done, see https://github.com/solid/community-server/issues/829).

The identifier strategy can just be the standard suffix identifier strategy when using a single pod. It doesn't even matter that much when using a single root pod.
Things that would have to be done depending on the needs:

If the user wants the server to start with a valid WebID: we would have to let the RootInitializer point to the pod template folder instead of the root template folder, and also provide some extra parameters to fill in the templates there.
If the user wants to use their own server as IDP for their WebID: this can be done by having a single extra call in the root initializer (or other initializer for this, might make more sense), that puts the WebID in the IDP together with email and password. The one issue for this might be that this requires the password to be passed to the server through either config or CLI. If neither of those is acceptable we would need a new solution that shows a sort-of registration form to the user when they first start the server.

* a new solution that shows a sort-of registration form to the user when they first start the server.

I'd like that really; so same registration, but it's instead the root pod that's created.

We can probably do something there with the root index.html file since we would probably need a different root template folder for that setup anyway.

Have been working on this a bit. The idea would be to have a SetupHttpHandler that can do about the same as the RegistrationHandler except it can also create a pod at root level (and overwrite the files there). After it does that it would disable itself (its canHandle function would always fail from then on), causing setup to only be possible once. As of writing I just realized that variable would reset when restarting the server so we would need to store it in a storage though.
Some issues I'm having is that its behaviour is quite similar to the RegistrationHandler, but the small differences make it a bit annoying to generalize or re-use. It also requires some similar behaviour to the LDP/IDP handler regarding requests and responses. I'll see if finishing up the main IDP architecture issues first might help with some of the above.

The comment was made on gitter that the fact that the server writes an .acl file to the root with full access might cause for some unsafe situations: users randomly starting the server in some different folders might not be aware that this causes those folders to be fully accessible if they start the server again in a parent folder in the future (or with another server).
One thing I was thinking of that might help would be to reject all requests to the server until the setup operation was executed once. This way we could make clearer that this setup is important for safety. We could even go to more extreme measures and catch the process exit event and remove those files again in case setup was not finished.
Another option might be to only trigger the RootInitializer after the setup process was finished but would have to investiagate how to handle the behavioural changes to support that.

The comment was made on gitter that the fact that the server writes an .acl file to the root with full access might cause for some unsafe situations: users randomly starting the server in some different folders might not be aware that this causes those folders to be fully accessible if they start the server again in a parent folder in the future

For that problem, I wonder if we want a (CSS-custom) triple that states: this MUST be a root ACL file. If not, an error should be flagged.
Or we should have a safer default root ACL.

Done in #951


lockfile is incompatible with netlify version of pnpm

I have got the error below when deploying to netlify via the readme button.
10:19:30 AM: build-image version: 73f16e520fdddf409be6f578c8c2c8941bdf32d3 (focal)
10:19:30 AM: buildbot version: aae36d37bb0728ee6dc52d97c5e1b5b83720773e
10:19:30 AM: Building without cache
10:19:30 AM: Starting to prepare the repo for build
10:19:30 AM: No cached dependencies found. Cloning fresh repo
10:19:30 AM: git clone --filter=blob:none https://github.com/DemoMacro/hexo-boilerplate-netlify-cms
10:19:31 AM: Preparing Git Reference refs/heads/main
10:19:33 AM: Starting to install dependencies
10:19:33 AM: Python version set to 3.8
10:19:33 AM: Attempting Ruby version 2.7.2, read from environment
10:19:34 AM: Using Ruby version 2.7.2
10:19:34 AM: Started restoring cached go cache
10:19:34 AM: Finished restoring cached go cache
10:19:35 AM: Installing Go version 1.19.13 (requested 1.19.x)
10:19:40 AM: go version go1.19.13 linux/amd64
10:19:40 AM: Using PHP version 8.0
10:19:41 AM: Downloading and installing node v18.18.0...
10:19:41 AM: Downloading https://nodejs.org/dist/v18.18.0/node-v18.18.0-linux-x64.tar.xz...
10:19:41 AM: Computing checksum with sha256sum
10:19:42 AM: Checksums matched!
10:19:44 AM: Now using node v18.18.0 (npm v9.8.1)
10:19:44 AM: Enabling Node.js Corepack
10:19:44 AM: Started restoring cached build plugins
10:19:44 AM: Finished restoring cached build plugins
10:19:44 AM: Started restoring cached corepack dependencies
10:19:44 AM: Finished restoring cached corepack dependencies
10:19:44 AM: No pnpm workspaces detected
10:19:44 AM: Started restoring cached node modules
10:19:44 AM: Finished restoring cached node modules
10:19:44 AM: Installing npm packages using pnpm version 8.6.7
10:19:45 AM: Failed during stage 'Install dependencies': dependency_installation script returned non-zero exit code: 1
10:19:45 AM:  ERR_PNPM_FROZEN_LOCKFILE_WITH_OUTDATED_LOCKFILE  Cannot perform a frozen installation because the version of the lockfile is incompatible with this version of pnpm
10:19:45 AM: Try either:
10:19:45 AM: 1. Aligning the version of pnpm that generated the lockfile with the version that installs from it, or
10:19:45 AM: 2. Migrating the lockfile so that it is compatible with the newer version of pnpm, or
10:19:45 AM: 3. Using pnpm install --no-frozen-lockfile.
10:19:45 AM: Note that in CI environments, this setting is enabled by default.
10:19:45 AM: Error during pnpm install
10:19:45 AM: Build was terminated: dependency_installation script returned non-zero exit code: 1
10:19:45 AM: Failing build: Failed to install dependencies

#35 fixes this


Snappi support : config.flows (packet, header, configure_pattern)

Fixed as per snappi[ixnetwork]==0.3.13


fix: update layout for new drawers

I assume this requires the new version of the design system package?

Looks great!


question about how to control expression?

Thank you for your work, but I have some inquiries regarding facial expression control about instantID.
In your work, it seems that the face embedding only provides identity information of the face in the reference image, lacking facial expression details. On the other hand, landmarks only consist of five key points, offering limited spatial control.
I wonder: How does InstantID manage to maintain consistency between the facial expressions in the output image and the reference image?
Thanks.

Actually, we don't want to constrain and fix the facial expression as the same as the given image. That is why we only use 5 kpts. As for your concern, you can adopt another facial pose controlnet to control the detail of face.

Actually, we don't want to constrain and fix the facial expression as the same as the given image. That is why we only use 5 kpts. As for your concern, you can adopt another facial pose controlnet to control the detail of face.

Thanks much for your answer.


featureprofile viewer

Proposed changes
It is a painful work to read profile, especially there are multi-parallel instances.
This tool helps us to grasp the main information of profile in a graphical view.
The profile is represented by a tree.
Sql operation nodes contains operation type(join, scan...), its node id, its fragment id. The number on the arrow edge means how many rows output by child node. This tool will sum the output rows of the same node in multi-parallel instances, that is if there are 4 parallel instance, and each ScanNode on lineitem table output 10 rows, the label on the arrow beginning with ScanNode(lineitem) is 40.
Here is a demo for tpch Q2
tpch q2 profile viewer
Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[ ] No
[ ] I don't know

Has unit tests been added:

[ ] Yes
[ ] No
[ ] No Need

Has document been added or modified:

[ ] Yes
[ ] No
[ ] No Need

Does it need to update dependencies:

[ ] Yes
[ ] No

Are there any changes that cannot be rolled back:

[ ] Yes (If Yes, please explain WHY)
[ ] No

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 37.33 seconds
load time: 436 seconds
storage size:<PHONE_NUMBER>7 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221120160203_clickbench_pr_49408.html


Add MSD to the Platform and Genetics footers (remove BMS)

Tasks

[ ] Add MSD to the Platform and Genetics footers, link to https://www.msd.com/
[ ] Remove BMS from Platform and Genetics footers

Acceptance tests
How do we know the task is complete?

When I go to Platform of Genetics I see MSD in the footer, I don't see BMS

Partner names in the footer are in alphabetical order, so the order should now be:

EMBL-EBI
Genentech
GSK
MSD
Pfizer
Sanofi
Wellcome Sanger Institute


Conflict with Materialize

Hi guyz,
I currently work on an Angular2 project, for which I need some material styles but also components. I want to use directives when I can, but also write raw HTML when needed.
When I first installed the material2 packages, I couldn't find the base css anywhere, even in the core package. So I thought I had to include it manually.
I included materializeCSS
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/materialize/0.97.7/css/materialize.min.css">
But it seems that styles are broken this way, example MdInput

The underline appears twice. When I remove materializeCSS, it appears ok.
Am I doing something wrong ? what should I do to use material2 components but also have accesss to grid, helpers classes, etc...
thx a lot

@supertino7 material2 and materialize are two different projects. You should not mix them together. Check getting started o how to use material2.

What @m0t0r said

Hi guyz,
Yes, I know that Material2 and Materialize are two different projects.
Can you just tell me if Material2 provides a CSS file with all the base styles of the Material Design (grid, tables, etc...), just like Materialize or Bootstrap do ? Or only Materialize provides that ...
thx


⚠ Cioos Pacific CKAN has degraded performance

In 292a635, Cioos Pacific CKAN (https://catalogue.cioospacific.ca) experienced degraded performance:

HTTP code: 200
Response time: 9982 ms

Resolved: Cioos Pacific CKAN performance has improved in 05c05ab after 1 hour, 19 minutes.


Can't connect Raspiblitz to Phone

Discussed in https://github.com/raspiblitz/raspiblitz/discussions/4556

Originally posted by Marus06 April 26, 2024
Hi,
the raspiblitz that im using ran well for about 4 Weeks, but after that i cannot connect to the Node anymore.
In the ZEUS app the folowing error message is prompted:

Just for troubleshooting, it also doesnt work on another phone. Am I missing any settings?

I just found out by Disabling and Enabling Tor on Raspiblitz again I was able to Reduce it to "kCFErrorDomainCFNetwork" Does anybody now what to do?


✏ Fix typo in Spanish tutorial index

Hi. I fix a little mistake on the index tutorial page for spanish docs.

Good catch! Thanks @aviloncho ! :detective:
And thanks @AdrianDeAnda , @Frans06 , @Kludex , @mariacamilagl for the reviews :coffee:


tiker code

how to create a ticker code to apply to exchange ?

what kind of ticker?

Closing for inactivity.


[Wait for #1104] [Layer/props] Convert name to props.

[Layer/props] Convert name to props.

**Changes proposed in this PR:**
- Convert name to props
- Add std::string specialization for property

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.4.20191203. Thank you for submitting PR #1106. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnsuite.mooo.com/.

:octocat: cibot: @zhoonit, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/1106-202104121832420.64719891548157-d391f1557a23e1d87365bbe4a4156e3089ad6951/.


Fix dup/idup documentation

The documentation didn't clearly specify that dup removes immutable/const, and that if the operation would violate const/immutable guarantees, it doesn't compile.
There was also a needless note that idup is D2.0 only.

See forum thread for reference<EMAIL_ADDRESS>

See forum thread for reference<EMAIL_ADDRESS>

@CyberShadow I can't tell what's wrong with this, why is the build failing?

Sorry, I don't know anything beyond that the PDF build is failing with the error messages indicated in the log.

TBH, I don't even know where the error is. I see a lot of warnings and a lot of things. And then "No PDF file produced". Why not? 😠

Who knows how the PDF build works?

I think I know what it is -- the commas. TROW is probably adding extra TDs...

OK, building now.


Consider how to deal with strings literals using different delimiters

We currently use `` delimiters by default, but single and double quotes are valid too. Think about formula bar -> AST, and also AST -> formula bar.

Update: think about how we'll auto-escape " characters


Correction for raw affiliation AgroParisTech, UMR TETIS, F-34035, Montpellier, France; Cemagref, UMR TETIS, F-34035, Montpellier, France

Correction needed for raw affiliation AgroParisTech, UMR TETIS, F-34035, Montpellier, France; Cemagref, UMR TETIS, F-34035, Montpellier, France
raw_affiliation_name: AgroParisTech, UMR TETIS, F-34035, Montpellier, France; Cemagref, UMR TETIS, F-34035, Montpellier, France
new_rors: 02kbmgc12;0458hw939
previous_rors: 02kbmgc12
works_examples: W2004062196
contact: d3b27ecd432f79b1c3d3f3b18f8f6f9d:7a477735ebeb6f626706ca4f3038f98b6d51 @ inrae.fr

This issue was accepted and ingested by the OpenAlex team on 2024-12-19. The new affiliations should be visible within the next 7 days.


Rewrite to remove refills from parsers

Right now there's a pattern we use like the following (simplified from the FASTA parser):
    let (start, end) = loop {
        if let Some(p) = memchr(...) {
            break ...;
        } else if rb.eof() {
            return Err(...);
        }
        rb.refill()?;
    };

The advantage of this is that we don't have to reparse the record out if we need to refill (because we don't exit out and then reenter once the buffer is refilled), but it also means that the lifetime of a record returned from the parser is fixed to right after refill is called (every time we parse, we could potentially wipe the underlying buffer and invalidate all previous records). Theoretically, this means we save time whenever we refill, but at the cost of code clarity and some lifetime constraints. The amount of time saved may not be that great for larger buffers or for mmaped files though.
If we moved the refill logic out of the parser into the caller and returned a sentinel value instead, we could simplify to:
    if let Some(p) = memchr(...) {
        (start, end) = ...
    } else if rb.eof() {
        return Err(...);
    } else {
        return Ok(Incomplete)
    }

This is much closer to how parser libraries like nom handle these situations so it probably won't have a noticeable performance impact? It would require a large rewrite of the parser code and all of the parsers, but I don't think many people are using entab right now that would object to the breakage.

So I wasted a week or so futzing with this and I think it's basically impossible with Rust's lifetime model.
I rewrote some parsers to implement the following (note, buffer is immutable now and we're only modifying buffer_state which tracks things like how many bytes are consumed):
pub trait FromSlice<'b>: Sized + Default {
    type State;

    fn get(
        &mut self,
        buf: &'b [u8],
        buffer_state: &mut BufferState,
        parser_state: &mut Self::State,
    ) -> Result<Option<usize>, EtError>;
}

And then tried to rewrite the buffer next function like:
    fn next<'r, T>(&'r mut self, buffer_state: &mut BufferState, mut state: <T as FromSlice<'r>>::State) -> Result<Option<T>, EtError>
    where
        T: FromSlice<'r> + Default,
    {
        let mut record = T::default();
        loop {
            let buffer = &self.as_ref()[buffer_state.consumed..];
            match T::get(&mut record, &buffer, buffer_state, &mut state) {
                Ok(Some(consumed)) => {
                    buffer_state.consumed_record += consumed;
                    buffer_state.consumed = buffer_state.consumed_record;
                    buffer_state.record_pos += 1;
                    break;
                },
                Ok(None) => return Ok(None),
                Err(e) => {
                    // I also added an "incomplete" flag to EtError to allow the parsers to signal that they needed the buffer to be refilled.
                    if !e.incomplete || buffer_state.eof {
                        return Err(e);
                    }
                },
            }
            if !self.refill(buffer_state)? {
                return Ok(None);
            }
        }
        Ok(Some(record))
    }

This doesn't work at all though because Rust can't tell that the immutable slice borrow that T::get takes isn't returned by the time we get to the mutable borrow to update the buffer. Basically, the associated lifetime on FromSlice is capturing the lifetime from self so we can't do anything further, even though that lifetime should be invalid if we haven't parsed out a value.
I'm not really sure there's a good way around this either? An alternative could be for each parser to first try to parse out a record and return a true/false, if false then refill the buffer and try again, and then finally pull out a full record. This would require either some amount of double parsing or some kind of "parsing guide" that the try step can save for the final step. Either would probably create some kind of speed hit.


Fix: wrong text-md class

rename class from text-md to text-base

Closes #
📝 Description

Add a brief description

⛳ Current behavior (updates)

Please describe the current behavior that you are modifying

🚀 New behavior

Please describe the behavior or changes this PR adds

💣 Is this a breaking change (Yes/No):

📝 Additional Information

Thank you! @camcam2605


[Codespaces] machine-overridable, experimental setting changes its value when the focus changes

Does this issue occur when all extensions are disabled?: No

Version: 1.66.0-insider
Commit: 3e5c7e2c570a729e664253baceaf443b69e82da6
Date: 2022-03-14T05:14:23.074Z (16 hrs ago)
Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.87 Safari/537.36

Steps to Reproduce:
This reproduces with the "Azure: Authentication Library" setting from the Azure Account Extension. Definition in package.json: https://github.com/microsoft/vscode-azure-account/blob/1268dab932c324a63f9875b96e3cf6dca742cfa2/package.json#L224

In Codespaces, install the Azure Account Extension
Open user settings with the "Preferences: Open Settings (UI)" command
Select user settings and use the drop down on the left to navigate to "Extensions" -> "Azure configuration".
Modify the "Azure: Authentication Library" setting and then click away from the dropdown box to change focus as shown below. The setting reverts back to the default value.

https://user-images.githubusercontent.com/22795803/158266976-6c613eff-4c9e-4dff-84ab-38ad330e6692.mov
More info:
This setting does not appear when searching the user settings even though it can be navigated to in the user settings tree. It does appear in a search of the workspace settings as expected.

Related: https://github.com/microsoft/vscode-azure-account/issues/482

Interesting. The value seems to save into the JSON file, but with a warning: This setting cannot be applied in this window. It will be applied when you open a local window.

Machine-overridable setting (machine-overridable scope) is not supported in user settings. It shall not be shown in the UI.

One issue in this case is that the user might be confused if there are no matches, so I'm also thinking of changing the text displayed when no matches are found.


New web-components version (v2.1.0) doesn't load

Description
When update from v2.0.1 to v2.1.0, importing components gives an error:
[vite-node] [ERR_LOAD_URL] @carbon/web-components/es/components/button/button.js

at @carbon/web-components/es/components/button/button.js

✅ Application with<EMAIL_ADDRESS>https://stackblitz.com/edit/github-gkn31f-tzrvfj?file=package.json

❌ Application with<EMAIL_ADDRESS>https://stackblitz.com/edit/github-gkn31f-gn6xrr?file=package.json

Component(s) impacted
All.
Browser
Chrome, Safari, Firefox, Microsoft Edge
Severity
Severity 1 = The design is broken in a critical way that blocks users from completing tasks or damages the brand. Affects major functionality, no workaround.
Application/website
IBM Quantum
Package
@carbon/web-components
Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I checked the current issues for duplicate issues

Reviewing changes in the package.json, why private?
https://github.com/carbon-design-system/carbon-for-ibm-dotcom/blob/72142f06ea627484a439203d3895491cd59594a9/packages/carbon-web-components/package.json#L3

This is the diffing I found:

Why this?
https://github.com/carbon-design-system/carbon-for-ibm-dotcom/blob/72142f06ea627484a439203d3895491cd59594a9/packages/carbon-web-components/package.json#L20
lib is only have .js.map files:

It works if I change from:
    "./es/components/*": {
      "node": "./lib/components/*",
      "default": "./es/components/*"
    },

To:
    "./es/components/*": {
      "default": "./es/components/*"
    },

@kennylam @ariellalgilmore @annawen1 this may have happened after we merged in.

Reviewing changes in the package.json, why private?
https://github.com/carbon-design-system/carbon-for-ibm-dotcom/blob/72142f06ea627484a439203d3895491cd59594a9/packages/carbon-web-components/package.json#L3

@kennylam Not sure if we need this anymore now that everything is merged in?

Fixed by https://github.com/carbon-design-system/carbon-for-ibm-dotcom/pull/11283
And released here: https://github.com/carbon-design-system/carbon-for-ibm-dotcom/releases/tag/%40carbon/web-components%402.1.1


Return err in case of error from findMob function

Returned error in case of error from findMob function
@eftychis, @variadico

LGTM

LGTM

LGTM


Feature/1176/269 jwt audience claim

What this PR changes/adds
Adapted common crypto code to enforce token claim check:

Audience must match IDS Webhook address
Subject must be "verifiable-credential"
Issuer must be set (to any string)
Expiration time must be set, and in the future

This code is used by the DID authentication module for IDS, but can be used in other contexts as well.
Client generates JWT for IDS requests containing destination web hook HTTP address as audience claim. Server rejects requests with wrong audience.
Why it does that
Without this change, the server can reuse the token to impersonate the client at a third party
Further notes
Removed TokenRepresentation.expiredIn, as it was not used, redundant with the token value, and incorrectly set.
Adapted OAuth2 module only to follow interface changes, without implementing additional logic.
Linked Issue(s)
#269
Checklist

[ ] added appropriate tests?
[ ] performed checkstyle check locally?
[ ] added/updated copyright headers?
[ ] documented public classes/methods?
[ ] added/updated relevant documentation?
[ ] added relevant details to the changelog? (skip with label no-changelog)
[ ] formatted title correctly? (take a look at the CONTRIBUTING and styleguide for details)

There are several changed code files which would need to add Microsoft to the contributors header.
thanks, updated


Fix installation workflow

Install command in current master doesn't work.
It happens because of new cache.init function in the installation waterfall. Init function appends an extra argument to the installAll function and breaks it. We should pass only error argument.

Good catch! Thanks!

What's currently installed on master hasn't been published to npm though, so it shouldn't have broken anything at least.


Adding a visitor counter to the site

I want to add a visitor counter to your website
Below are the samples of my previous work

@tushargupta1504  Kindly assign me the issue.

Sorry Sir but the GSSOC-24 EXTD is over


ModuleNotFoundError: No module named 'azure'

Describe the bug
When running the quick-start Python example in the Docker container the following error occurs:
ModuleNotFoundError: No module named 'azure'

Steps To Reproduce
Any of these will have the same issue:
### Option 1 - Locally from source code:
cd examples/msal-python
docker build . --tag=msal-python
docker run --rm msal-python

### Option 2 - Pulling the image from the registry:
docker run ghcr.io/azure/azure-workload-identity/msal-python

### Option 3 - Kubernetes
kubectl logs quick-start

Expected behavior
All dependency modules should be available in the Python environment during execution.
Logs
$ kubectl logs quick-start
Traceback (most recent call last):
  File "/app/main.py", line 4, in <module>
    from azure.keyvault.secrets import SecretClient
ModuleNotFoundError: No module named 'azure'

Environment

Kubernetes version (use kubectl version): Client: v1.22.5, Server: v1.22.6
Cloud provider or hardware configuration: Azure
OS (e.g: cat /etc/os-release): WSL Ubuntu 22.04

Merged #437


Remove provider from add profile view and allow only non-manuals

Coverage increased (+0.04%) to 74.04% when pulling b098bd2525b48873926de0210d69327b11513f58 on nitin099:develop into 243d35fad3121b9bf3639f767c73494a39df0cea on fulfilio:develop.


connection.end() callback issue

Hi,
I have a Node JS program where one js file takes care of creating a connection, querying and closing the connection by calling connection.end(). The functionality is exposed with a function named query().
This function is being called repeatedly from another file in a loop. The problem I have is, it reaches max connection. Ideally I am expecting the connection to be closed after each query. The code seems correct. I noticed that the connection.end() is called on time but the callback which is passed into the end() function is called almost at the end. Can I get some help on why it is delaying ending connection?

All of Node.js networking operations are async. Node.js doesn't provide an async network interface, so you have to wait for the callbacks to complete.


Clarify default value for packageType build flag

Platforms affected
Android
Motivation and Context
Documentation clarification that will save developers some time.
Description
Document that the default value for the packageType build flag is bundle for release builds.
Testing
No testing done.
Checklist

[ ] I've run the tests to see all new and existing tests pass
[x] I added automated test coverage as appropriate for this change
[x] Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
[x] If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
[x] I've updated the documentation if necessary

@breautek

I think this is an improvement. I noticed that you backported the change to 10.x version of the docs. I think we can also apply the change to the 11.x version of the docs as well since it would apply there too.
The dev version will cover the next time we do a snapshot for the new version.

I meant to do that but somehow missed it. Done now. Thanks!

The changes will appear live usually within an hour.


OAS 3.1 : summary should not be mentionned as a warning , display and rendering rendered as x-summary

Describe the bug
Redoc is supposed to fully supprt OAS 3.1
the fied summary is standard and should be fully supported not reporting a warning
it should works as same as x-summary
To Reproduce
add a summary on a responses as below

401-Unauthorized:
  summary: Unauthorized
  description: Unauthorized - Authentication is required.

Expected behavior
No warning on linter , same representation for summary and x-summary
Redocly Version(s)
Redocly 1.6.0

Hi @LasneF! According to the spec, there's no such filed in the Response Object as summary: https://spec.openapis.org/oas/v3.1.0#fixed-fields-14. Am I missing something?

@tatomyr , i apologize sorry for the noise , and thanks for the clarification


Option to only show nil nodes

Not sure if this is still maintained or not, but it would be nice to have an option to show only calls to non existing nodes. Maybe there is a way to do it with overriding the default handler, but I am not sure how to do it based on the instructions in the README (i.e., where should I put the overriding code)

I18n::Debug.on_lookup = lambda do |key, value|
  logger.debug("Missing key: #{key}") unless value
end

Put that somewhere after requiring this gem. In Rails that would be in an initializer. Does that answer your question? :blush:

Yes, thanks.
Added a new initialized with this code
if defined? I18n::Debug
  I18n::Debug.on_lookup = lambda do |key, result|
    Rails.logger.debug("[i18n-debug] #{key}") unless result
  end
end


Update JSON column types to JSONB

After we update to Rails 4.2 we need to change JSON columns to JSONB columns
Rails 4.2: https://github.com/projectjellyfish/api/issues/310

@OBCENEIKON Am I going to be stepping on your toes if I knock this out?

Resolved in 7c5922cf5d1d8b27aa2ee2414d5dda9c77d4a2c0


Strange line behaviour

Hi there, I'm getting this strange behaviour with all the graphs that I try to use, am I missing some setting? I can't find anything on it. This is straight from the line chart sample code :)

WTF,
:))))
May I ask you your fl_chart version?

@imaNNeoFighT thanks for the quick response, I'm using 0.6.0, installed it today so should be latest? 👍

You're welcome,
yes, it is the latest.
I will check it out ASAP.
Thanks for report.

I couldn't reproduce it.
It's working correctly on my machine :)

Hmmm, thanks for checking, I'll start from scratch and see what happens :)

Ok, waiting for you

@imaNNeoFighT I should have probably mentioned I'm building a web app 🤦♂ it's working for mobile, do you know why it's doing that on Flutter web? Is it something I can try to fix for you? :)

I see there is already a ticket open for flutter web, you can close this one if it's bothering you, thanks for the help 😄

Oh man, I didn't know that you are testing it on the web :)
We have an already opened issue,
BTW, I think there is a workaround here but I didn't test it.
It will be great if you help me to fix it with a temporary hack.
Feel free to ask me anything and make the Pull Request.
Thanks!

I will close it, and we can follow it up in #95.


Feature/remove licence restriction on edit contact

Removes the restriction on editing contact details of an issued licence.
Issue https://github.com/Scottish-Natural-Heritage/Licensing/issues/1311.

LGTM :)


Table border color was specified in hex instead of a variable

Hey there,
I was messing around with the code and found that the table border color was specified in hexadecimal instead of having a variable. At first, I thought I could change it by $color-quaternary, but decided to make one more variable for it.
If it should be $color-quaternary, just let me know and I'll update my pull request.
As for the code samples, I didn't submit any, the change is very subtle in fact.

Good call!
Thanks for the PR


Wrong .lib file name for packaging

It seems that UBT expect XXXX.lib, but I have XXXXd.lib in the same directory instead of XXXX.lib.
For example UBT want async++.lib, while in the certain folder it's not there. But async++d.lib is.
I've built cesium-native with VS 2019, no errors or warnings. Tried to build with other ways but failed.
Is this problem happening just because of not using VS 2017?
Here is the log file of UBT.

By the way, is there any way to test whether cesium-native is built correctly without packaging?

The d suffix indicates you've built a debug build of cesium-native. If you're building a Development or Shipping version of Cesium for Unreal, you'll need to build the Release or RelWithDebInfo configuration instead. This is already mentioned in the Developer setup instructions on this page:
https://github.com/CesiumGS/cesium-unreal/blob/ue5-main/Documentation/developer-setup.md
See the second-to-last bullet under "Principles". And the Windows build instructions state how to build the Release configuration:
https://github.com/CesiumGS/cesium-unreal/blob/ue5-main/Documentation/developer-setup-windows.md#cmake-command-line
So I'm closing this. Please open community forum threads instead of GitHub issues in the future, unless you're fairly confident you're seeing a bug.

Thank you SO MUCH. That really helps. I'll check the forum.


Migrate multi-select tests over to Cypress

Summary
Migrate multi-select tests over to Cypress
💬 Description
As per #1166, tests for the multi-select component need to be migrated over to Cypress. This involves migrating over every e2e test we currently have, as well as accessibility and visual testing. This also involves removing the old framework's code, so two versions of the same tests do not run.
💰 Use value
Improve our testing toolset to allow us to test in more states and deliver a higher quality product.

Following work done in #1277, tests were added for the functionality added under that ticket. There were issues with the popup menu closing when items were clicked. This appeared to be a Cypress only issue as did not happen anywhere else the component was used (eg storybook\stackblitz).
May want to consider re-looking into trying to resolve this issue when implementing these tests

complete before #2108


Add creativenull/diagnosticls-configs-nvim

Repo URL:
https://github.com/creativenull/diagnosticls-configs-nvim
Checklist:

[X] The plugin is specifically built for Neovim, or if it's a colorscheme, it supports treesitter syntax.
[X] The lines end with a .. This is to conform to awesome-list linting and requirements.
[X] The title of the pull request is Add/Update/Remove `username/repo`  (notice the backticks around `username/repo`) when adding a new plugin.
[X] The description doesn't mention that it's a Neovim plugin, it's obvious from the rest of the document. No mentions of the word plugin unless it's related to something else.
[X] The description doesn't contain emojis.
[X] Neovim is spelled as Neovim (not nvim, NeoVim or neovim), Vim is spelled as Vim (capitalized), Lua is spelled as Lua (capitalized), Tree-sitter is spelled as Tree-sitter.
[X] Acronyms should be fully capitalized, for example LSP, TS, YAML, etc.

Thanks for the PR!

nice 1!
I could see more like this popping in the near future following the deprecation of null-ls


Using Faiss-node in aws having problem working fine locally

Describe the bug
A clear and concise description of what the bug is.
Environment:

Operating system: [e.g. Windows x64]
Nodejs Version: [e.g. v18.15.0]
Package Version: [e.g. v0.1.0]

To Reproduce
Steps to reproduce the behavior:
1.
2.
Expected behavior
should be run succesfully and store text in form of vector store
Screenshots
"errorType": "Error",
"errorMessage": "Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node and make sure you have libomp installed in your path.\n\nError: /opt/nodejs/node18/node_modules/faiss-node/build/Release/faiss-node.node: cannot open shared object file: No such file or directory",
"trace": [
"Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node and make sure you have libomp installed in your path.",
"",
"Error: /opt/nodejs/node18/node_modules/faiss-node/build/Release/faiss-node.node: cannot open shared object file: No such file or directory",
"    at FaissStore.importFaiss (file:///opt/nodejs/node18/node_modules/langchain/dist/vectorstores/faiss.js:207:19)",
"    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
"    at async FaissStore.addVectors (file:///opt/nodejs/node18/node_modules/langchain/dist/vectorstores/faiss.js:60:37)",
"    at async FaissStore.fromDocuments (file:///opt/nodejs/node18/node_modules/langchain/dist/vectorstores/faiss.js:197:9)",
"    at async Runtime.handler (file:///var/task/index.mjs:57:19)"
]
}
.

@abgupta99  Thank you for the issue, I will try to reproduce and fix it

This seems identical to the issue I have with HNSWlib-node.
It might be coincidental, but in case this is a langchainjs issue, not faiss, see:
https://github.com/hwchase17/langchainjs/issues/943
https://github.com/hwchase17/langchainjs/issues/1764

Spent the day building test cases for this, and it's the same langchainjs error.
Most approaches end up with:
"Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory"
Currently I see that there is any route to using langchain with either faiss or hnswnlib on AWS Lambda.

Having similar problem deploying on Vercel, while on local dev is working fine

Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory","stack":"

I have create a build for AWS Lambda in https://github.com/ewfian/faiss-node/pull/30
And is was works on my side:

It has not been released yet, but you can try the version I built manually below:
faiss-node-v0.2.2-aws-x64.zip
For other projects, please replace with the files under the path below:
faiss-node-v0.2.2-aws-x64.zip\node_modules\faiss-node\build\Release

Hi,
No luck my side I’m afraid: still missing libgomp.so.1:
{
  "errorType": "Error",
  "errorMessage": "Could not import faiss-node. Please install faiss-node as a dependency with, e.g. `npm install -S faiss-node`.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory",
  "trace": [
    "Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. `npm install -S faiss-node`.",
    "",
    "Error: libgomp.so.1: cannot open shared object file: No such file or directory",
    "    at FaissStore.importFaiss (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:237:19)",
    "    at async readIndex (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:141:37)",
    "    at async Promise.all (index 1)",
    "    at async FaissStore.load (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:144:51)",
    "    at async query (/var/task/index.js:112:20)",
    "    at async exports.handler (/var/task/index.js:20:9)"
  ]
}

David

@dmb0058 I want to know whether it works properly if you directly upload the zip(faiss-node-v0.2.2-aws-x64.zip) I provided above to AWS.
Please confirm that you have replaced the files(node_modules/faiss-node/build/Release) in my zip above to the zip you finally want to upload to AWS.
I also made a demo for langchain here: https://github.com/hwchase17/langchainjs/issues/1930#issuecomment-1646500643
You can try to upload this demo directly to AWS.

Yes, the demo zip (faiss-node-v0.2.2-aws-x64.zip) runs when I upload it as a lambda and gives the same results as you show above.
I can't create a layer though, so can't use this in practice:
 aws lambda publish-layer-version --layer-name my-faiss-layer \
        --description "My Faiss layer" \
        --license-info "MIT" \
        --zip-file fileb://my-faiss-node.zip \
        --compatible-runtimes nodejs18.x \
        --compatible-architectures "x86_64"
An error occurred (RequestEntityTooLargeException) when calling the PublishLayerVersion operation: Request must be smaller than 70167211 bytes for the PublishLayerVersion operation

@dmb0058 Can you reduce the size of the zip to make it meet the limit first.
The prebuilt binaries is too large under Linux was also mentioned in your previous issue(https://github.com/ewfian/faiss-node/issues/26).
The binaries for AWS have shrunk a lot. The zip containing faiss-node is only 7.5MB. And the zip containing langchain and faiss-node has 15.5MB. Which is far less than the limit of AWS. Please identify what content takes up space and try to reduce it.

That sounds really good. Let me see if I can strip some of the other
modules in the application.
On Sat, 22 Jul 2023, 16:10 Fan, @.***> wrote:

@dmb0058 https://github.com/dmb0058 Can you reduce the size of the zip
to make it meet the limit first.
The prebuilt binaries is too large under Linux was also mentioned in your
previous issue(#26 https://github.com/ewfian/faiss-node/issues/26).
The binaries for AWS have shrunk a lot. The zip containing faiss-node is
only 7.5MB. And the zip containing langchain and faiss-node has 15.5MB.
Which is far less than the limit of AWS. Please identify what content takes
up space and try to reduce it.
—
Reply to this email directly, view it on GitHub
https://github.com/ewfian/faiss-node/issues/25#issuecomment-1646603955,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDXCQKCC5FK5EWNK5PSMC3XRPUN7ANCNFSM6AAAAAA2B4CLBY
.
You are receiving this because you were mentioned.Message ID:
@.***>

@dmb0058 You can also try this demo https://github.com/hwchase17/langchainjs/issues/1930#issuecomment-1646994780

The new version was released, To use faiss-node in AWS, you can  download faiss-node-vX.X.X-aws-x64.zip from release. and upload to AWS as a layer.
BTW. if you are using the faiss-node as layer, do NOT install faiss-node as npm package in your AWS Function, see the demo here: https://github.com/hwchase17/langchainjs/issues/1930#issuecomment-1646994780

This now works flawlessly with Fly.io, I had made a cool demo before and now I don't need to copy the zip folder over.
Thank you for maintaining and improving this 🙏🏻

@ewfian
I'm happy to pay you one time 30$ as the best I can for now. Can you share a buymeacoffee/paypal/stripe link? Thanks!

The newer version is still cannot get through on the lambda layers, still having the same error which is
{
"errorType": "Error",
"errorMessage": "Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory",
"trace": [
"Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.",
"",
"Error: libgomp.so.1: cannot open shared object file: No such file or directory",
"    at FaissStore.importFaiss (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:237:19)",
"    at async readIndex (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:141:37)",
"    at async Promise.all (index 1)",
"    at async FaissStore.load (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:144:51)",
"    at async query (/var/task/index.js:112:20)",
"    at async exports.handler (/var/task/index.js:20:9)"
]
}


Updates to latest brave and removes all static thread local use

This moves to the pending brave release. This passes the same tests
locally as before this commit.
feign has some problems on 2.2.x so I skipped like so:
./mvnw clean install -pl -:spring-cloud-sleuth-instrumentation-feign-tests

2.2.x is broken so merging without CI


No such module 'LDSwiftEventSource'

Hi everyone! I am getting a No such module 'LDSwiftEventSource' when updating to version 1.5.0. I am using Xcode's Package dependencies, any chance this is an issue on the SDK side when using the Xcode's Package dependencies? Thanks!

Hi @lrodriguezcds,
Were you able to resolve this issue by updating to the latest version of the SDK 1.6.0? If not, we can investigate this further!

Hi @kaushalkapasi! No, I was not able to solve the issue yet.
Answering your question, I am seeing the error in the version 1.6.0 as well. It seems it can't find the module .. I tried to add the dependency with the issue (LDSwiftEventSource) manually (via Xcode's Package Dependencies)  but when I do that I get the error mentioned here  https://github.com/DevCycleHQ/ios-client-sdk/issues/114

Tried to run it on a real device and without having selected any simulator just in case but get the same error.
Thank you so much for your time!

Hi @lrodriguezcds, we've published a new version v1.6.1 with a fix for this issue.
Let us know once you have verified it and we can close this issue!

I've updated to vertion 1.6.1 and once cleaned and built I am seeing this error in Xcode

@lrodriguezcds taking a look! will update you shortly!

@lrodriguezcds published v1.6.3 with the fix! thanks for your patience on this!

No problem @kaushalkapasi! Thank you for taking the time to review and fix this!
Now it is compiling when upgrading to version 1.6.3 but I tried the feature flags and it seems  the onUpdate callback is not being fired (not even at initialization time..) We are aiming to update to a higher version to be able to make use of the real time update! 🙌  but now it seems it is not going through the onUpdate not even when we first run the app (that is working in the previous version (1.3.1) )

Thank you!

@lrodriguezcds the onUpdate handler will get triggered every time the value changes when compared to the previous value for a specific User based on the variation they are receiving. You'll also need to hit save on the Feature page after updating the Variable / Variation values in order to trigger a realtime update!
Hopefully our iOS Realtime Updates Docs can clarify things!

Hi @kaushalkapasi! Thank you for the link! I am doing exactly what you mentioned i.e changing the value on the console and hitting "save" however the realtime update is not being triggered and neither when the app first loads, what I mean by this is that it is not bringing the flag value from DevCycle when I first run the app.
As I mentioned, the same code is working fine in version 1.3.1 but with the version (1.6.3) the onUpdate is not being triggered. I am looking at the docs and it seems we are doing what the doc says we have to do, so this seems to be a problem with this version since version 1.3.1 works ok!
Just in case, we are sending the user as anonymous (.isAnonymous(true)) however if we remove the anonymous property and send a userId instead, the problem remains (the flag is not updated on the app because the onUpdate is not being triggered).
Thanks you!

Hi @lrodriguezcds,
Thanks for confirming! Is there a way you could share a minimum reproduction of your app? Additionally, join us on the DevCycle Discord and we can get some more eyes on this issue!
Note: the DevCycle Discord link is limited to 5 uses & expires in 24 hours!

Hi @kaushalkapasi, yes sure.
In the didFinishLaunchingWithOptions method we configure DevCycle as it follows:
let user = try? DVCUser.builder()
            .isAnonymous(true)
            .build()

        if let user = user {
            dvcClient = try? DVCClient.builder()
                .environmentKey(Config.shared.devCyclenvironmentKey)
                .user(user)
                .build(onInitialized: nil)
        }

_ = dvcClient?.variable(
            key: key,
            defaultValue: defaultValue
        ).onUpdate(handler: { newValue in
            self.isFlagEnabled = newValue
        })

The line self.isFlagEnabled = newValue is not being executed.

Hey @lrodriguezcds, found a few things that I wanted to share with you that should fix your problem! Let me know if this works for you!

When using a DevCycle Variable (as you have here), its using a weak-reference which means the onUpdate reference is lost once that code is executed. The way to solve this, will be to create a variable in scope of the class where you want to use the variable so that a strong reference exist throughout the lifecycle of your app and updates can be triggered!

For the issue where the onUpdate is not being triggered initially, this is due to the caching that has been introduced in newer versions of the SDK. I've added a line for how the Variable can be read initially, before an onUpdate is triggered in the example below.

var myBooleanDevCycleVariable: DVCVariable<Bool>?         // fix for point 1

self.myBooleanDevCycleVariable = dvcClient?.variable(     // fix for point 1
    key: key,
    defaultValue: defaultValue
).onUpdate(handler: { newValue in
    self.isFlagEnabled = newValue
})

self.isFlagEnabled = self.myBooleanDevCycleVariable.value // fix for point 2

Will try it and let you know!

Hi @kaushalkapasi!
Awesome! I have updated the code with the suggested changes and the realtime updates and initialization are working now!
Thank you your help!! 🙏


Debugging session expires shortly after using the vpn trick

When ever I debug DolphiniOS with the vpn self debugging in 1.1.0, it works but shortly after (about 30 seconds) the session expires and the game crashes. I’ve re installed dolphin and jitterbug multiple time and it still doesn’t work. Using a normal debugging session with Libimobiledevice works fine without crashes.

You need to switch back to JitterBug immediately after starting your game so that JitterBug can detach from the game. Then you can switch back and play as long as you want. This is necessary because JitterBug, like all iOS apps, cannot run in the background, so as soon as the game opens, JitterBug stops executing, and it cannot perform the detach.

Might be worth to implement location-based background activity for Jitterbug, as this will be a major annoyance and source of support tickets. Apps like iSH successfully use location tracking to stay active in the background.


[FR]Use multiplexing over HTTP2 in sendEachForMulticast method for batch push.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is.
When I upgrade fcm to Http V1, the speed will become very slow when using sendEachForMulticast in firebase-admin:9.2.0.
Isn't HTTP/2 used in the SDK?
Describe the solution you'd like
Convert bulk send in SDK to HTTP/2.
Describe alternatives you've considered
Keep the original sendMulticast method.
Additional context
According to the database conditions, users (hundreds of thousands) who meet the sending conditions are selected for message push.
If the current sendEachForMulticast is used, it will be a nightmare.
The following code took 15 seconds to execute.
        List<String> registrationTokens = new ArrayList<>();
        for (int i = 1; i <= 500; i++) {
            registrationTokens.add(TOKEN + i);
        }

        MulticastMessage message = MulticastMessage.builder()
            .setNotification(Notification.builder().setTitle("test title").setBody("test body").build())
            .addAllTokens(registrationTokens).build();
        BatchResponse response = FirebaseMessaging.getInstance().sendEachForMulticast(message);
        System.out.println(response.getSuccessCount() + " messages were sent successfully");

@lahirumaramba, @jonathanedey Instead of just using HTTP/2 multiplexing I strongly advocate restoring the sendMulticast() method.
There is not much in computer science that can be said accross the board. But one of the few exceptions is, that IO operations, especially network requests, are slow compared to CPU bound operations. That's a fact, every software developer should know.
While it might be acceptable for small hobby projects to use sendEachForMulticast() to send messages in separate network requests, is is surely not for a company that is sending approx. 16 million messages per month.
It's a standard use case for us to send notifications to thousands of devices that are subscribed to some topics. While this was no problem with the legacy API, it will make thousands (!) of HTTP requests with the new API taking up seconds to minutes instead of milliseconds.
That doesn't only mean that user experience get's worse, due to delayed notifications, it also means, that our costs rise by a factor of 10, 15 or even more, as we are using serverless infrastructure that is billed by the millisecond.
Even if you don't care about performance and costs, this is a massive waste of resources. I honestly believe it is the task of every developer and software architect to be as efficient as possible with resources. Especially in the times, we live in.
=> It is therefore safe to say that this is a severe design error in the HTTP API v1.
And just in case you want to point out, that we should send to topics instead devices: that doesn't work either. Because it doesn't support the use case where we need to address users, that are interested in either one of multiple topics without sending duplicate notifications. (I don't even want to open a feature request suggesting this, as it is pretty much pointless, considering the other issues here.)
Also, the option mentioned in the docs, to use OR conditions, does also not work. It was obviously not built to scale. It works only for a very small number of topics, but with millions of topics, it takes many minutes for a single message to be delivered.
I really can't understand why you make these decisions and not listen to your users. It's very frustrating.

Hey @MightySepp666 the support for batch send APIs including sendMulticast have been deprecated in the backend services, which is beyond scope for this SDK. Firebase Admin Node SDK (this repo) is essentially a wrapper for the REST API. We are actively looking into implementing http/2 to efficiently use the new FCM send API for batch requests. In the meantime, I'd recommend filing a Firebase support ticket from https://firebase.google.com/support/troubleshooter/contact to share your feedback on the above to reach the correct internal teams.

Hope this gets prioritized soon. Critical issue IMO.

I am not a serious node programmer, just want to simply replace the "sendMulticast" method to another one if deprecated.
I am sure 95% server maintainers have no idea that 1 month later their server will be crashed, after changing sendMulticast => sendEachForMulticast.
node firebase admin has been downloaded over 1 million per week. Do you guys think majority of those maintainers are smart enough to write their own batch codes?
Where are you spending my $1🧐☺ paying for the firebase services?
I am just waiting for the crash going to be happening. As I don't really know how to resolve this situation actually.

@MightySepp666 Like you, I don't understand this step backwards. The sendAll method was introduced in the SDK because the use of individual send calls wasn't performing well enough. I understand that the maintainers of this repo can't do anything about it because they're not the ones in charge of developing the Firabse Messaging infrasctructure, but I think the problem should be brought to their attention.
Also, the topics system isn't usable because it involves sending exactly the same notification to users, but in most cases, the payload of each notification needs to be customized according to the person for whom the notification is intended, so it doesn't work.
I opened a ticket as suggested by @lahirumaramba to contact support. If you find a solution that improves performance I'm interested...

@LebonNic @MightySepp666
Using this package may be an option if you cannot afford to wait for longer and need a workaround. Have not tested it myself so cannot vouch for how well it works. Ideally, the Firebase team should fix this as soon as possible.

@jimnor0xF thanks for the tip. But I looked at the code and it would be rather cumbersome for me to integrate. It's not a drop-in replacement and doesn't provide the other API methods of the Firebase SDK. Also I would require to patch it, like the regular Firebase SDK, due to the still open bug of duplicate push notifications caused by the retry functionality (see https://github.com/firebase/firebase-admin-node/issues/1900#issuecomment-2014853972). It doesn't export types as well, so it's harder to integrate with TypeScript.
At the moment I am rather considering replacing Firebase completely by a different push notification provider backend.

@jimnor0xF thanks for the tip. But I looked at the code and it would be rather cumbersome for me to integrate. It's not a drop-in replacement and doesn't provide the other API methods of the Firebase SDK. Also I would require to patch it, like the regular Firebase SDK, due to the still open bug of duplicate push notifications caused by the retry functionality (see #1900 (comment)). It doesn't export types as well, so it's harder to integrate with TypeScript.
At the moment I am rather considering replacing Firebase completely by a different push notification provider backend.

I faced this issue too. Can you suggest another Firebase alternative service?

All users and businesses using the Firebase Admin SDK are experiencing the same issue.
I don't understand why they are pushing to end support for the legacy API (batch API) without showing any interest in user feedback.
https://github.com/firebase/firebase-admin-java/issues/941

Any progress regarding this topic?

Thank you for your patience! With the release of v12.3.0, sendEach() and sendEachForMulitcast() now use a HTTP/2 connection by default when sending messages. We'd like for you to give it a try and give us your feedback on these changes.
While we work through this transition, if you are facing any issues with the new HTTP/2 transport, we have provided access to the legacy HTTP/1.1 versions of these methods. This can be enabled by using the enableLegacyTransport() method. This method is already marked as deprecated and will be removed once the HTTP/2 transport is considered fully stable.

There is no support for HTTP/2 multiplexing yet, so notifications are sent over a single HTTP/2 connection, which is limited to 100 concurrent streams (requests).
This means that only 100 notifications can be sent at any given time over this single connection, which makes firebase-admin unsuitable for use cases that require high throughput (>100 notifications per second).
Any estimate for when this will be available?

Hey @victorHugoCarvalho,
The current implementation does use HTTP/2 multiplexing. Each call to sendEach() or sendEachForMulitcast() creates a new connection with 100 concurrent streams. Meaning 2 sendEach() calls would result in 2 HTTP/2 connections with 200 concurrent streams and 200 messages in flight at a time.
If the issue here is having a 1 to 1 ratio of streams to messages this can be done by batching your sendEach() calls with 100 messages each rather than the allowed maximum of 500.
If this isn't the behaviour you are experiencing please open a new issue and we can take a look there.

hey @jonathanedey,
The problem we are facing is precisely the delay in sending these notifications. Overall we are making 450 calls to sendEachForMulticast() where each batch has 500 tokens.


feat: add GetFocused to field_input

My use case is that I have a field that I'd like to update suggestions for based on what the user has input. When the user starts typing, a search endpoint is hit with the current value of the field as the query parameter and the suggestions is updated based off the returned search.
So basically, in my Update I want to check if the field is focused and then if a new key was pressed to hit the endpoint and update the suggestions.

Hey @SpyMachine, dynamic inputs are now merged. Let me know if this example is what you were trying to accomplish:
https://github.com/charmbracelet/huh/blob/main/examples/dynamic/dynamic-suggestions/main.go


Update REST API to contain Various Activity Transaction Time

Calculate and Store Activity Transaction Time
As a user of Whisk I would like to be able to be able to understand how much time is spent performing various Whisk Activities.  I need to know more than just the overall execution time.  Here are the sorts of details I would like to know:

user: billable time in the action

system:  controller-to-controller time - user

network: time to communicate from user’s browser to CF Node.js web application that implements the REST Client **

REST Client:  user-measured client-to-server response time - user - system - network **

** I would like to know the ui and network times for the following two reasons:

performance testing
to avoid confusion between the amount of time I witness the browser (or other REST client) take and the amount of time portrayed by the user and system times.

As a result, the backend REST API should be updated to include 4 new parameters:

user (as defined as above)
system (as defined as above)
network - which is the time it takes to communicate from the REST Client to the the API Server.
REST Client - (as defined above)

Note: For now we could avoid displaying 3 and 4 on the Whisk UI.

Going to close this as a duplicate of https://github.com/openwhisk/openwhisk/issues/373 (since by way of creating transient records, we can record the various checkpoints).


NGINX example config is broken

The /socket.io location section is missing, and the braces do not match up. I recommend to only publish the site configuration, not a complete nginx configuration with ssl and whatnot, as many people do not terminate ssl on their backends.
My working config for reference:
root@lxt-prod-invnt-consolere01:~# cat  /etc/nginx/sites-enabled/tty.my.domain
upstream consoleServerBackend {
    ip_hash;
    server <IP_ADDRESS>:8081;
}

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}

server {

    server_name  tty.my.domain;
    listen       8080;
    underscores_in_headers on;
    error_log /var/log/nginx/consolere.error.log;
    keepalive_timeout    60;
    large_client_header_buffers 8 32k;
    charset UTF-8;
    root /srv/app/console-remote-server/build;

    location ~* \.(jpg|jpeg|gif|png|ico|css|zip|tgz|pdf|txt|js|flv|swf|ttf|otf|woff|eot)$ {
        access_log off;
        error_log /var/log/nginx/consolere.static.error.log;
        expires 30d;
    }

    location /socket.io {
        proxy_pass http://consoleServerBackend/socket.io;
        proxy_set_header Proxy '';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect off;

        set $CORS_CREDS true;
        set $CORS_ORIGIN $http_origin;
        set $CORS_METHODS 'GET, POST, PUT, DELETE, OPTIONS';
        set $CORS_HEADERS 'Authentication-Token, Cache-Control, Cookie, If-Modified-Since, Range, User-Agent, X-Requested-With';
        # FYI: Always allowed headers: Accept, Accept-Language, Content-Language, Content-Type
        set $CORS_EXPOSE_HEADERS 'Content-Disposition, Content-Length, Content-Range, Set-Cookie';
        # FYI: Always exposed headers: Cache-Control, Content-Language, Content-Type, Expires, Last-Modified, Pragma
        set $CORS_PREFLIGHT_CACHE_AGE 600;
        set $X_FRAME_OPTIONS '';
        # set $X_FRAME_OPTIONS "ALLOW FROM $http_origin";

        if ($request_method = 'OPTIONS') {
            add_header Access-Control-Allow-Origin $CORS_ORIGIN;
            add_header Access-Control-Allow-Methods $CORS_METHODS;
            add_header Access-Control-Allow-Headers $CORS_HEADERS;
            add_header Access-Control-Allow-Credentials $CORS_CREDS;

            add_header Access-Control-Max-Age $CORS_PREFLIGHT_CACHE_AGE;
            add_header Content-Type 'text/plain; charset=utf-8';
            add_header Content-Length 0;
            return 204;
        }

        if ($request_method != 'OPTIONS') {
              add_header Access-Control-Allow-Origin $CORS_ORIGIN;
              add_header Access-Control-Allow-Methods $CORS_METHODS;
              add_header Access-Control-Allow-Headers $CORS_HEADERS;
              add_header Access-Control-Allow-Credentials $CORS_CREDS;

              add_header Access-Control-Expose-Headers $CORS_EXPOSE_HEADERS;
              add_header X-Frame-Options $X_FRAME_OPTIONS;
        }

        # supposedly prevents 502 bad gateway error;
        proxy_buffers 8 32k;
        proxy_buffer_size 64k;
        proxy_read_timeout 120;
        proxy_connect_timeout 120;

        # proxy_next_upstream error timeout;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        access_log /var/log/nginx/consolere.websockets.log;
        error_log /var/log/nginx/consolere.websockets.error.log;

    }
    location / {
        access_log /var/log/nginx/consolere.access.log;
        expires 30d;
        index /www/index.html;
        error_log /var/log/nginx/consolere.error.log;
        try_files $uri $uri/ /app/consoleapp.html;
    }
}

@elpollodiablo Thank you for sharing ! You right, I think I lost one line location /socket.io { in the configuration.
I'll update my example.
I would not recommend to use port 8080 on public domain, because some client could have problem to connect, using SSL (with let's encrypt)  is better.

I would not recommend to use port 8080 on public domain, because some client could have problem to connect, using SSL (with let's encrypt) is better.

That's why I suggested leaving the https config out, most serious users will not have something like a console.re server public facing, but rather put it behind a reverse proxy (in addition to the nginx). I terminate my SSL on haproxy, so I had to remove all the ssl references :)
If a user wants SSL/le for their nginx, there are better documentation resources than an nginx example in an unrelated project.
I like the software very much, so please do not take this negatively. Thanks for your work!

@elpollodiablo
Yeah, all good and thanks for the feedback and thank you for using it. I think more or less advance users will know that do it with configuration, what I have is just an example, it is not like copy and paste code (in my view). Less experienced users will install and use it for local development.
I have clients with private servers and they all pretty much use the nginx only for console with subdomains like remoteconsole.domain.com. I wondering why do you need haproxy in front of nginx ?

I wondering why do you need haproxy in front of nginx?

Because it's the central loadbalancer for all my projects, websites and apis alike.

Security (because everything that's public facing is a potential vector), ease of maintenance (only need to update SSL in one place, only need to configure letsencrypt in one place etc) and high reliablity/failover (...that I only need to maintain in one place instead of each project seperately).
I could leave out the local nginxen, but then I'd have to poke around in the haproxy, and I don't want that, because it serves more than one project/client. Each tool has its place :)

Well, you can load balance with nginx just fine and security is pretty good too. Pretty much everything that haproxy can, nginx could do as well.

I could leave out the local nginxen, but then I'd have to poke around in the haproxy, and I don't want that, because it serves more than one project/client. Each tool has its place :)

This is true, but each additional running tool on server eats resources, that is why I asked, I think if possible to cut down to minimal number of tools on server to serve the same content, you could save some money on your servers. But yeah, it is all depend on many aspects and cases, sometime it is just a very convenient to separate similar functionality between tools.

Nevermind, I was just trying to point out that the example config is unexpectedly opinionated :) I have my reasons for my setup (all domains are on one ip, seperation of concerns, security, maintainability etc, I've been refining this for 20+ years), but indeed my setup is saving time instead of the last 10% of hardware :)
I found two issues relating to CORS:
The X-consolere header that the client sends needs to be included in the cors headers:
    set $CORS_HEADERS 'Authentication-Token, Cache-Control, Cookie, If-Modified-Since, Range, User-Agent, X-Requested-With, X-consolere';

Also, since both nginx and the node software set the cors origin header, we need to unset it first in nginx, otherwise there'll be two of them, and browsers don't like that:
 proxy_hide_header 'access-control-allow-origin';

Maybe also add a note that this is an "every domain can log to your server" setup and that this would be the place to hard-code the allowed referer:
// this will set the header to the incoming referer,
// there could be a regex condition to validate if one wants to support a wildcard:
// add_header Access-Control-Allow-Origin $CORS_ORIGIN;
// this will set the allowed referers to only https://only.my.domain
add_header Access-Control-Allow-Origin https://only.my.domain;

It would be a little easier for some to configure it in the software, but since nginx is necessary for the deployment, this should be fine too.

Not to be a nag, but https://github.com/kurdin/console-remote-server/commit/bcc1730acf3fab436e89631883095a4c58e27cfd#diff-b7a041c6d181d81dfa72eda8d8c80e89625d0df3270437ef4f82f6462dcf34b4R43 will also need a /socket.io I think? At least it does in my setup.

Not to be a nag, but bcc1730 will also need a /socket.io I think?

I think I added location /socket.io {  on line 42
https://github.com/kurdin/console-remote-server/blob/main/config/console.nginx.example.conf#L42
Where did you add that on your setup ?

The X-consolere header that the client sends needs to be included in the cors headers:

location /socket.io {
    proxy_pass http://consoleServerBackend/socket.io;

otherwise the backend receives only the urlpart after /socket.io

I don't have x-consolere header on my console.re setup, need to double check that on other server.

I had to add it because Chrome balked without it in a cross sub domain setup (Version 90.0.4430.51 (Official Build) beta (x86_64))

otherwise the backend receives only the urlpart after /socket.io

ok, yeah, you right
I have rewrite /socket.io/(.*) /socket.io/$1 break; but using proxy_pass http://consoleServerBackend/socket.io; might be a better solution.

@elpollodiablo I am not Nginx config expert for sure, but strangely if I remove rewrite /socket.io/(.*) /socket.io/$1 break; and add http://consoleServerBackend/socket.io to configuration, Google Chrome starts complaining about x-consolere header needs to be in CORS, with rewrite, it works just fine.

Maybe also add a note that this is an "every domain can log to your server" setup and that this would be the place to hard-code the allowed referer.

Good idea, I'll add that, thanks


fix(compiler): Remove the waring for valid v-slot value

What kind of change does this PR introduce? (check at least one)

[x] Bugfix
[ ] Feature
[ ] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change? (check one)

[ ] Yes
[x] No

The Vue doc says that one can use destructuring props with fallback value as v-slot's value.

Actually, this will cause a warning.

The PR fulfills these requirements:

[x] It's submitted to the dev branch for v2.x (or to a previous version branch), not the master branch
[ ] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[x] All tests are passing: https://github.com/vuejs/vue/blob/dev/.github/CONTRIBUTING.md#development-setup
[x] New/updated tests are included

If adding a new feature, the PR's description includes:

[ ] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

Nice catch, although I think the proper fix would be introducing an extra branch for v-slot and check whether the expression is valid in a function argument position.

@yyx990803 I'm glad to implement this.

I just fixed CI - could you  rebase against dev? Thanks.

@yyx990803 Done.


XP support

Hi
Any chance of adding support for Windows XP? Some games written for this system cannot be run on Vista/7/10/11 systems, and at the same time they are too new to run on 95/98/Me systems and in practice they only run on XP or only there they work reasonably well. If the softgpu supported XP it would be perfect

Don't you know Windows XP has guest additions?

@ilikecoding-197 3D support for Windows XP is little to none in current virtualizers. It only makes sense a request for Windows XP SoftGPU driver would appear as a result.

I can't add XP support to this project – NT and Window 95 display driver architecture is completely different (the 9x is backward compatible even with Windows 3.x). But there isn’t much difference between XP and Windows 10 and 11 – there shouldn’t be much work at last for OpenGL, DirectX is bit problematic – you must implement DDI 9 (in newer Windows is emulated by D3D12 which is part of Mesa), or use WineD3D same way as in VirtualBox 5.1 or in SoftGPU :-)
But I’m currently focus on Windows 9x, XP may come later…

for XP support, something like this I assume would be out of scope but for anyone stumbling on this it might not be a bad idea to look towards reactOS for inspiration, IIRC they use their own wined3d implementation internally, it won't be a 1:1 by any means, but might be something that could get hobbled together.

the better target
is drivers for new videocards for XP %)

@MrPepka, do you have a list of those games that "cannot be run on Vista/7/10/11 systems, and at the same time they are too new to run on 95/98/Me"? I'm mentioning the same topic at exoscoriae/eXoWin3x/issues/680 - listing potential reasons for support to break, etc.
@CE1CECL, you mention you work on Vista/7 driver, maybe you are interested in XP version as well?

There is a related patch from http://lonecrusader.x10host.com/rloew/rloew.html

WDMEX: Allows Windows 9x to load drivers written for Windows 2000/XP.

Maybe a game, which doesn't work on Vista+ can be made to work on Win9x via XP driver... although somebody has to develop the XP driver first

@MrPepka, do you have a list of those games that "cannot be run on Vista/7/10/11 systems, and at the same time they are too new to run on 95/98/Me"? I'm mentioning the same topic at exoscoriae/eXoWin3x/issues/680 - listing potential reasons for support to break, etc.
@CE1CECL, you mention you work on Vista/7 driver, maybe you are interested in XP version as well?

Actually, its only a backend, I chose the VMware SVGA because lots of OS's support A driver, even if its not good, macOS, Linux, Windows 95-11, etc... Its not a driver, for the os, its only to bring 3D to QEMU (at the moment, software).

In other words, to bring Aero to vista and 7 on QEMU, at the moment it works only with VMware tools (windows, linux is working, without OpenGL using kernel) https://packages.vmware.com/tools/esx/4.0u1/windows/x86_64/ through https://packages.vmware.com/tools/esx/4.1latest/windows/x64

@MrPepka, do you have a list of those games that "cannot be run on Vista/7/10/11 systems, and at the same time they are too new to run on 95/98/Me"? I'm mentioning the same topic at exoscoriae/eXoWin3x/issues/680 - listing potential reasons for support to break, etc.
@CE1CECL, you mention you work on Vista/7 driver, maybe you are interested in XP version as well?

I think that there are fewer and fewer such games, but there will still be some exceptions, once Uka from the old-games.ru forum even made a list of such games
https://www.old-games.ru/forum/threads/igry-kotorye-idut-na-windows-xp-no-ne-na-bolee-pozdnix-versijax-os.89881/

Virtualbox 6.1-7 would need a display driver.
However, VMWare still supports 3D acceleration under Windows XP. Its D3D9 support seems to be fine; the main issues are with older ones. Basically, what it needs is a "D3D2-7 to 9" wrapper to make it work.
For D3D8 you can already use d3d8to9.

I compared with box9x and boxnt display driver inf files,and tried to add glide and wined3d files to it.However the different between two files are so big.So it's quite a hard work.

Mesa9x works with XP and later, there was only small bug (also in the lasted release, correct in source now) with system version detection causing softpipe was default renderer instead of much faster llvmpipe.
Only really missing piece is ICD support in XP driver (without it you have to replace opengl32.dll, which leads to unstability).
But after all it is still software acceleration only and a bit slow. I still hope that someone[^1] correct XP driver in VirtualBox to have OpenGL acceleration and after it is relative easy to use WineD3D/Wine9x to do rest of the work.
[^1]: I also hoped, that Oracle development around Windows 7/10/11 driver will be faster, results you can see here https://www.virtualbox.org/ticket/21515#comment:43

Doesn't older vmware builds support opengl in windows xp? That might work with wined3d/wine9x.
https://www.vogons.org/viewtopic.php?t=88678

Virtualbox 6.1-7 would need a display driver.
However, VMWare still supports 3D acceleration under Windows XP. Its D3D9 support seems to be fine; the main issues are with older ones. Basically, what is needed is a "D3D2-7 to 9" wrapper to make it work.

how did you find out?

I thought VirtualBox dropped 3D acceleration for XP/Vista:
* XP since VirtualBox 6.0.24

3d acceleration crashing on 5.x.x (at least on XP host)
i found moar stable version 4.2.0


Fix JavaDoc and Run Brotli tests on Windows

Motivation:
JavaDoc of StandardCompressionOptions should point towards public methods. Also, Brotli tests were failing on Windows.
Modification:
Fixed JavaDoc and enabled Brotli tests on Windows.
Result:
Better JavaDoc and Brotli tests will run on Windows

/cc @normanmaurer @trustin

@hyperxpro this fails on linux now:
2021-07-08T15:40:57.2241892Z [INFO] Running io.netty.handler.codec.http.DefaultHttpResponseTest
2021-07-08T15:40:57.2277075Z [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in io.netty.handler.codec.http.DefaultHttpResponseTest
2021-07-08T15:40:57.5675937Z [INFO]
2021-07-08T15:40:57.5682300Z [INFO] Results:
2021-07-08T15:40:57.5682843Z [INFO]
2021-07-08T15:40:57.5683370Z [ERROR] Failures:
2021-07-08T15:40:57.5685806Z [ERROR]   HttpContentDecoderTest.testResponseBrotliDecompression:216 Content-Length header should match uncompressed string's length ==> expected: <92212> but was: <97715>
2021-07-08T15:40:57.5730751Z [ERROR]   HttpContentDecoderTest.testResponseChunksBrotliDecompression:260 Content-Length header should match uncompressed string's length ==> expected: <92212> but was: <97715>

It is due to Charset?

Current status: Passing on Windows but failing on Linux due to charset encoding.
If this method is used: SAMPLE_STRING = IOUtils.toString(uncompressed, CharsetUtil.UTF_8); then we get something like this:
2021-07-08T16:44:58.4475064Z [WARNING] ForkStarter IOException: io.netty.handler.codec.http.HttpContentDecoderTest,testResponseChunksBrotliDecompression,null,Response body should match uncompressed string ==> expected: <{\000A"items": [\000A{\000A"index": 1\002C\000A"index_start_at": 56\002C\000A"integer": 2\002C\000A"float": 11.1118\002C\000A"name": "Stephanie"\002C\000A"surname": "Puckett"\002C\000A"fullname": "Lauren Quinn"\002C\000A"email"<EMAIL_ADDRESS>true\000A}\002C\000A{\000A"index": 2\002C\000A"index_start_at": 57\002C\000A"integer":

I changed that method with: SAMPLE_STRING = new String(IOUtils.readFully(uncompressed, uncompressed.available()), CharsetUtil.UTF_8);. Now encoding issue is somewhat fixed but still it's not equals. ;/

I have no idea and not time atm... maybe later today

It's because sample.json is a text file and it will have different line ending in Windows when checked out. We could replace \r\n with \n to get the consistent result.

@hyperxpro can you do what @trustin suggested ?

If I use CRLF, tests will pass on Windows but fail on Linux.
;/

I was just thinking lately that maybe we should replace this test with something simple and subtle.
What do you guys think? @normanmaurer @trustin @slandelle

@trustin thoughts ?

Thanks a lot!


UIWebView deprecated in OS 12.0 (WKWebView)

Hi!
Is there any plans to update to WKWebView?
Thanks!

The App Store will no longer accept new apps using UIWebView as of April 2020 and app updates using UIWebView as of December 2020.”
Can you please update WKWebView instead of UIWebView

Please update on this. yesterday I update my app and got reject just because of this issue (UIWebView). Kindly update.

@bluelabeldeveloper1  if you have any substitute, kindly provide me.
Thanks.

Any update on this? I really need this updated so I can push out an update to my app.


install.sh calls gcc with the wrong path

I've tested install.sh again and let it install all prequisites. In the end, it tries to compile opencoarrays, but it is using the wrong path for the just compiled gcc, /batch/test/opencoarrays/prerequisites/installations/gcc instead of /batch/test/opencoarrays/prerequisites/installations/gcc/6.1.0/bin/gcc .

install-opencoarrays.log.txt

Ah, I see.... this is because of the greedy %% parameter expansion and the fact that gcc appears twice once in the path and once as the name of the binary. We need to handle/parse the output of mpicc -show much more carefully. I also don't think we can cavalierly just discard the binary name either since some systems are gcc-6 etc.


Version bump for retriggering release

GEMD Python PR
Description
This is just a version bump, to retrigger the PyPI deploy action.
PR Type:

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Maintenance (non-breaking change to assist developers)

Adherence to team decisions

[ ] I have added tests for 100% coverage
[ ] I have written Numpy-style docstrings for every method and class.
[ ] I have communicated the downstream consequences of the PR to others.
[x] I have bumped the version in __version__.py

Resolved via delete & reissue release


Add background papers to homework

E.g. in homework 2, some references discussing Kernel Ridge Regression would be useful for interested students.

Closing as stale.