Expose S3 Gateway to web based clients

What is the goal / desired outcome?
Allow web based s3 clients to connect to the pachyderm s3 server implementation. For this to work in browser, we should need to handle CORS pre-flight OPTIONS requests. This would unlock additional S3 compatible tooling
In general, i think the s3 gateway could eventually replace our (aging) http api and allow us to consolidate support on one option.
Proposal
Add a S3_ALLOWED_DOMAINS flag for CORS domain support.

@ysimonson I get the sense you have some reservations about this approach? Can we capture them here?

Chatted with @Connor-Wallace about this offline. I don't think we're going to be able to use s3 gateway, because it's not feasible to browse non-head commits on PFS.

I have some reservations about this too. It sounds like there's a decent amount that would have to be added to s3g to make it viable for this, which expands its scope from a shim layer that does just what's necessary to allow other software that's designed around s3 apis to interface with data in pfs, to something that's got to expose everything in pfs that we might want to expose in the UI. Moreover I'm worried there will be things that just can't be exposed this way because our filesystem model doesn't totally map to an object store model, at the very least I think we'll have to shoehorn a bunch of things in.
To me it makes more sense to base the filesystem browser on an API that we have full control that unfortunately probably means we can't grab something off the shelf to base the filebrowser on.

Thanks all! Was hoping this approach would be a quick win, but it's looking like we'll need to go back to the drawing board.


cant run the app from within the Applications folder on my Mac.

cant run the app from within the Applications folder on my Mac.

Can you provide more info? Mac OS version, which version of Code you have...I'm sure there's more...

Please refer to https://github.com/Microsoft/vscode/wiki/Submitting-Bugs-and-Suggestions for how to submit issues.
Closing the issue until more steps are provided.

Osx El Capitan version 10.11.2  vscode Version 0.10.3 (0.10.3)


Prelude to clojurescript upgrade

Unlike previous attempts, this PR only address upgrade todos for #1973 that are compatible with the current version of ClojureScript. By doing this upfront work, we make the upgrade branch more manageable and closer within reach :smile: . This PR does the following:

Fixes all clojurescript warnings that future versions of lein-cljsbuild will pick up. These are warnings that we shouldn't be causing anyway so these are uncontroversial.
This introduces a second build target for generating cljsDeps.js as done by @BenjaminVanRyseghem in #1952. Having a second build target can introduce unreliability in lein cljsbuild once. I'll double check to see if I need to update script/build.sh in some way. This build target only needs to be run when we upgrade cljs.
Remove our dependence on js->clj and clj->js. This is necessary for an upgrade as we can't continue to depend on home-rolled versions of such a core fn. This change has some small risk. I"ll run through the QA checklist but I don't expect to find any issues. If you're curious how I determined what plugins use our js->clj, it was with this search.

@kenny-evitt @rundis I'll be using this branch exclusively until we merge. I plan to merge Monday or Tuesday unless you have any concerns

I'm running this branch now. I'm not doing any work with Clojure or ClojureScript tho.

I ran through most of the QA checklist and haven't seen any issues. I'll double check cljsbuild reliability and if that's good merge tomorrow

I've been running a build of this branch on a Mac and a Windows computer without issue.

@kenny-evitt @rundis Fyi. Now that this branch is merged you should run lein cljsbuild [once|auto] app when recompiling LT cljs


Computation Loss

Hello,
It looks like the coupling layer loss is first summed over all batches, time steps, and channels (https://github.com/NVIDIA/waveglow/blob/master/glow.py#L55). Note that the number of channels for all coupling layers is not fixed due to early outputs reducing the channel count. But then the loss is averaged over batches, time steps, and n_group (https://github.com/NVIDIA/waveglow/blob/master/glow.py#L59). This means that regardless of the number of channels actually present in a given coupling layer, we're averaging over n_group (8).
Is this intentional? It's unclear to me why the coupling layer loss is computed this way.

We're always averaging over the dimensionality of the input audio vector, which is equal to batch size x segment length.
The audio vector is folded and becomes batch size x n_group x (segment length / n_group).


Update Atlassian product information

Update link to point to Atlassian's formal security advisory
Update products to 'Not Affected' that are not impacted by CVE-2021-44228
Add notes to indicate that products in certain non-default
configurations may be vulnerable to a related but lower severity
vulnerability involving Log4j.

Atlassian has also admitted for Bitbucket to be affected and has published an advisory. Bitbucket isn't even on this list yet, nor is it in this pull request.
https://confluence.atlassian.com/security/multiple-products-security-advisory-log4j-vulnerable-to-remote-code-execution-cve-2021-44228-1103069934.html

That’s correct. We made that determination in the last 24 hours and will update this PR shortly.
On Thu, Dec 16, 2021, at 11:24, Maik Musall wrote:

Atlassian has also admitted for Bitbucket to be affected and has published an advisory. Bitbucket isn't even on this list yet, nor is it in this pull request.
https://confluence.atlassian.com/security/multiple-products-security-advisory-log4j-vulnerable-to-remote-code-execution-cve-2021-44228-1103069934.html
—
Reply to this email directly, view it on GitHub https://github.com/cisagov/log4j-affected-db/pull/117#issuecomment-996023951, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADTKHQ7K6QWG73XEMHDZ43URIODPANCNFSM5KEUYIPQ.
You are receiving this because you authored the thread.Message ID: @.***>

We updated the advisory today to include new information about Bitbucket Server. I'm updated the PR to reflect that addition.


Contribute and develop

So far it's been easy to create a new app and add content to it, just follow the instructions in https://facebook.github.io/react-native/docs/getting-started.html and eveything works great.
Now, I couple of questions:

How can update the sources for react? As far as I know when I use react-native init [MY_PROJECT] they are copied and never updated again.
I would also like to contribute. i.e. I've exposed a property in the webview to automatically adjust the zoom to fit the content.

I've tried to accomplish 1 and 2 by forking the repo in ./react-native and under ./my_project have my project. After that I tried to link the libraries to my project following https://facebook.github.io/react-native/docs/linking-libraries.html. I copied the same structure used by react-native init [...] (drag&drop libraries, link them and added the headers). But when I try to compile I get:
module.js:338
    throw err;
          ^
Error: Cannot find module 'chalk'
    at Function.Module._resolveFilename (module.js:336:15)
    at Function.Module._load (module.js:278:25)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/Users/jmrodriguez/dev/test/react-native/packager/getFlowTypeCheckMiddleware.js:11:13)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
Process terminated. Press <enter> to close the window

How can I solve this issue? Is this the correct configuration for contribute and develop?
Thanks in advance!
JM

To update React, update the version number of react-native in your your package.json file. To help with this you could run npm install -g npm-check-updates to install a helper script, and then run npm-check-updates -u inside of your project directory. Then run npm install to get the latest copy of React.

You might need to run npm install in your project directory since it looks like you're missing dependencies. For testing my development copy of React, here's what I recommend:

a) Fork this git repo and clone the fork to your computer
b) Create a new git branch and make your changes
c) In your iOS project, change the value of the "react-native" dependency from a version number to the path of your local git repo. For example "react-native": "~/Code/react-native".
d) Run npm install in your iOS project
e) You should be able to run your project using your own copy of React
f) Push your branch when you're done and submit a PR

Pretty comprehensive answer there from @ide, nice one :smile:


Add autofix for at-if/else-closing-brace-newline-after

Looks like you did some indentation changes in a0c2011 that broke the tests.

@kristerkari Fixed it 👍
I think we're good to go on all auto-fix PRs, right?

Hmm... I'm thinking that maybe this rule should also have disableFix as it adds a newline without indentation. What do you think @OriR ?

Yeah, on second look it looks odd.
Will add it now.


Expose admin ports for agent, collector, and query Deployments via the equivalent Service

Which problem is this PR solving?
Resolves #2203
Short description of the changes

Expose the admin port listed in the agent, collector, and query Deployments via the equivalent Service. This allows Prometheus to scrape the /metrics endpoint on that port and therefore monitoring of Jaeger itself.

oh it seems some basic tests are still failing

@thomaspaulin could you take a look to the failing tests?

@iblancasa Sure thing, I'll take a look at the tests.

seems there is still something wrong here: https://github.com/jaegertracing/jaeger-operator/actions/runs/5410048296/jobs/9830973296#step:6:1541

mh...  @thomaspaulin I pushed to your branch and now then it was closed?

@frzifus how odd, thanks for creating a new PR with the fixes!


Ambassador application - Rose Sakach

Thank you for your interest in becoming an Open Mainframe Ambassador. To be considered, please answer the following questions. Please also reference the Ambassador Program overview for any questions.

Your name ( also put in the title replacing NAME )

Rose Sakach

Email

<EMAIL_ADDRESS>

Bio

Rose is a Global Product Manager at Broadcom responsible for Zowe CLI, Zowe Support and solutions that integrate mainframe developers with modern tooling and enable DevOps for the Enterprise. She has lead sessions at SHARE, CA World, DevOps.com, and various worldwide Broadcom Product User Groups. She has over 30 years of experience in technology roles including application development, systems programming, operations, product administration, consulting services and product management.

Projects within OMP you would want to be an ambassador for

Zowe

List any other topics you would be able to speak about
_* Zowe Conformance

Zowe LTS

Zowe Components

Zowe Onboarding

COBOL, Modern Mainframe Development_

Why do you want to be an ambassador?
I've spent my entire career (more than 3 decades) as an I.T. professional working with mainframes and remain very passionate about this platform despite the rumors or its impending demise.  The introduction of Zowe - the first EVER Open Mainframe Project for z/OS gave all Mainframers an opportunity to become a part of a greater community while making history.  As a Zowe Ambassador I'd love the opportunity to help all I.T. professionals understand the importance of the platform, how it has continually transformed to meet the needs of customers, demystify any concerns associated with "Open Source" and welcome them to the project.

Attach a headshot

Your application has been approved - next steps for the announcement at https://github.com/openmainframeproject/foundation/issues/167

Your application has been approved - next steps for the announcement at https://github.com/openmainframeproject/foundation/issues/167


[FEATURE] Add entries() and values() to adapters going forward.

Is your feature request related to a problem? Please describe.
SELECT * from 'table';
Fetch all keys and values, remove encoding, and return as an indexed list of [ ['key', 'value' ] pairs, and [ 'value1', 'value2' ]
Add option for limiting number of results for keys() entries() and value()
Describe the solution you'd like
Update api.ts and all current adapters to implement entries() and values()
Describe alternatives you've considered
The current solution is keys() and get(keys) to fetch all keys and values.
Additional context
N/A

Landed in 849e134. [Milestone 2.0.0 Development] and set for release next publish period.
Breaking changes are pending release with this milestone. This is not yet complete and will be revisited, so this issue will remain open for the time being.

Full integration landed in 5487aa4. These are tentative final implementations and can be considered the final API.


ja: Make docs/concepts/workloads/pods/pod-lifecycle.md follow v1.18 of the original text

This is a Feature Request

What would you like to be added
Update docs/concepts/workloads/pods/pod-lifecycle.md to follow v1.18 of the original (English) text.
Why is this needed
content/ja/docs/concepts/workloads/pods/pod-lifecycle.md is outdated.
Comments
/language ja
/good-first-issue
File to update
https://github.com/kubernetes/website/blob/dev-1.18-ja.2/content/ja/docs/concepts/workloads/pods/pod-lifecycle.md
Original
https://github.com/kubernetes/website/blob/fb6364d/content/en/docs/concepts/workloads/pods/pod-lifecycle.md
diff between translated and v1.18
https://gist.github.com/64d53513239e268f5b4988ad7af83ecd
Note
Currently, we use dev-1.18-ja.2 branch. Please open a PR targeting the branch.
ref
https://kubernetes.io/ja/docs/contribute/localization/
If you have a question, feel free to ask us at slack.k8s.io #kubernetes-docs-ja channel.

/triage accepted

/assign

/close
duplicate https://github.com/kubernetes/website/pull/23786


Client Linux 32

Hello! I have been developing and using it successfully on linux64, windows32 and macos64. Now testing on linux32 which I need working as well. The connection is not established, checking, the expected key is always different from the calculated one (in the handshake).
Any suggestion?
Thank you

I currently don't have the capabilities to test this under Linux 32 bit, but what you could do is check the function GenerateAcceptingKey from wsstream.pas.
You create a simple project that includes wsstream.pas and calls GenerateAcceptingKey on a few strings fro 32 and 64 bit linux and see if they are different. I can take a look into that later this week, but it would be great if you could do this little test if you already have such a testing environment available

Hi!, i do that and isolate the problem, and I post in Lazarus forum:
https://forum.lazarus.freepascal.org/index.php?board=66.0


goog.asserts.AssertionError is not compatible in ExpectedFailures.js

This issue was imported from Closure Library's previous home at http://closure-library.googlecode.com
The link to the original issue is:
https://code.google.com/p/closure-library/issues/detail?id=575

Incorrect assumption about AssertionError's purpose. assertThrows should be used instead.


New pythons not supported

Because of this:
https://github.com/mcurlej/joft/blob/a52567643ffde9db5b5a6c3e6eeffd722be94c15/pyproject.toml#L13
I cannot use an environment with 3.12 or 3.13 even the documentation says:
https://github.com/mcurlej/joft/blob/a52567643ffde9db5b5a6c3e6eeffd722be94c15/README.md?plain=1#L17

Fixed.


Bind parameter(s) popup panel always shows

Hello,
As soon as I use any host variable in my query, the Bind parameter(s) popup panel is show even if @set is used to define a host variable value.
Any chance to avoid it?
I am using latest Dbeaver UDM 5.1.0 on MSWindows
Regards,

If all parameters are set then this dialog won't be shown.


Created chatting view

Added onClickListener to ChatGroupRecyclerAdapter
Created ChatHistoryRecyclerAdapter
Created business objects hierarchy for getting chat messages and users who sent them
Added to mock data simple chat history

+1

+1


ldap-checker: fix for Python 3.12 compatibility

The ldap-checker module is currently not compatible with Python 3.12+ because ssl.wrap_socket() was deprecated and has been removed. Fixed this by using SSLContext.wrap_socket() instead.
While doing so, my code linter was complaining so I added a cleanup and code quality commit first.

Thanks for the fix! I'll check it out soon. I would let the elif statements though, there is no reason to check the error multiple times if one matches.

I would let the elif statements though, there is no reason to check the error multiple times if one matches.

The thing is, there is always a return right before the next if, so it will stop anyway if it evaluates to True once. That's why the linter was complaining.
However, if any stylistic change is not at yours discretion, please let me know. I can change that of course. :)

I would let the elif statements though, there is no reason to check the error multiple times if one matches.

The thing is, there is always a return right before the next if, so it will stop anyway if it evaluates to True once. That's why the linter was complaining.
However, if any stylistic change is not at yours discretion, please let me know. I can change that of course. :)

Makes sense, i would leave it anyway though, because it is clearer, that we can enter only one if statement.
Btw which linter do you use?

Okay, just stripped the cleanup commit.
I'm currently using pylint. I know you are using ruff, and I use it on a project at work, too. But haven't switched to it entirely, yet.

@exploide thanks for the report and fix!


Version 31 and above requires that one of FLAG_IMMUTABLE or FLAG_MUTABLE

Bug Report
I think there is bug with PendingIntent, because i do not receive background notifications.
Or am I missing some parameter while sending push notification?
Logs taken while reproducing problem
FATAL EXCEPTION: Firebase-FCMService
Process: com.myapp.myapp, PID: 32530
java.lang.IllegalArgumentException: com.myapp.myapp: Targeting S+ (version 31 and above) requires that one of FLAG_IMMUTABLE or FLAG_MUTABLE be specified when creating a PendingIntent.
Strongly consider using FLAG_IMMUTABLE, only use FLAG_MUTABLE if some functionality depends on the PendingIntent being mutable, e.g. if it needs to be used with inline replies or bubbles.
at android.app.PendingIntent.checkFlags(PendingIntent.java:408)
at android.app.PendingIntent.getActivityAsUser(PendingIntent.java:491)
at android.app.PendingIntent.getActivity(PendingIntent.java:477)
at android.app.PendingIntent.getActivity(PendingIntent.java:441)
at com.google.firebase.messaging.zzb.zzf(Unknown Source:68)
at com.google.firebase.messaging.zzc.zzas(Unknown Source:34)
at com.google.firebase.messaging.FirebaseMessagingService.zzd(Unknown Source:59)
at com.google.firebase.iid.zzf.run(Unknown Source:4)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at com.google.android.gms.common.util.concurrent.zza.run(Unknown Source:6)
at java.lang.Thread.run(Thread.java:1012)

I ran into the same under Android.  All is good under iOS.
I spent a few hours googling it. Nothing works thus far. Below is what I have:
cordova -v:
12.0.0<EMAIL_ADDRESS>
Installed platforms:
  android 12.0.1
  browser 7.0.0
  ios 6.3.0

Error message:
Process: com.myapp, PID: 12463
java.lang.IllegalArgumentException: com.myapp: Targeting S+ (version 31 and above) requires that one of FLAG_IMMUTABLE or FLAG_MUTABLE be specified when creating a PendingIntent.
Strongly consider using FLAG_IMMUTABLE, only use FLAG_MUTABLE if some functionality depends on the PendingIntent being mutable, e.g. if it needs to be used with inline replies or bubbles.
    at android.app.PendingIntent.checkPendingIntent(PendingIntent.java:428)
    at android.app.PendingIntent.getActivityAsUser(PendingIntent.java:544)
    at android.app.PendingIntent.getActivity(PendingIntent.java:530)
    at android.app.PendingIntent.getActivity(PendingIntent.java:494)
    at com.google.firebase.messaging.zzb.zzf(Unknown Source:68)
    at com.google.firebase.messaging.zzc.zzas(Unknown Source:34)
    at com.google.firebase.messaging.FirebaseMessagingService.zzd(Unknown Source:59)
    at com.google.firebase.iid.zzf.run(Unknown Source:4)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
    at com.google.android.gms.common.util.concurrent.zza.run(Unknown Source:6)
    at java.lang.Thread.run(Thread.java:1012)

Version @dev4.0.0 version has that handled. I needed to update package.json firebase version to 13+

Is the latest firebase version 10.6.0? Can you post your package.json?
On Sat, Nov 25, 2023 at 07:15 mkdevel @.***> wrote:

Version @dev4.0.0 version has that handled. I needed to update
package.json firebase version to 13+
—
Reply to this email directly, view it on GitHub
https://github.com/havesource/cordova-plugin-push/issues/256#issuecomment-1826294606,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACOLXZDVTQFSSRMUCYZOUYDYGHOOXAVCNFSM6AAAAAA7XXBQWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRWGI4TINRQGY
.
You are receiving this because you commented.Message ID:
@.***>

Removing and adding the plugin to Cordova did the trick. All is working. Please close the issue.

Closing this issues as it was already resolved since version 4.0.0.
I would recommend:

Reinstall the plugin. If defining FCM_VERSION, must be at least 23.+. The default is 23.+ since v4.0.0.
Update to the latest cordova-android platform.


Add ability to control colorized output

cargo-near provides colorized output which might be undesirable if you are piping stderr to a file.
We could do what cargo does: --color auto,always,never
Originally suggested here.

Resolved in #86.


Thinkpad X1 Yoga 4th Gen

Which device did you used ?

The default device.
Ouput of linux-enable-ir-emitter --verbose configuration
configure command output
[gctrindade@goose-yoga ~]$ sudo linux-enable-ir-emitter --verbose configure
INFO: Warning to do not kill the process !
[<EMAIL_ADDRESS>global /build/opencv/src/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (2401) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module source reported: Could not read from resource.
[<EMAIL_ADDRESS>global /build/opencv/src/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1356) open OpenCV | GStreamer warning: unable to start pipeline
[<EMAIL_ADDRESS>global /build/opencv/src/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (862) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Did you see the ir emitter flashing (not just turn on) ? Yes/No ? Yes
ERROR: Your infrared camera is already working, skipping the configuration.
ERROR: The configuration has failed.
INFO: Do not hesitate to open an issue on GitHub ! https://github.com/EmixamPP/linux-enable-ir-emitter/wiki
[gctrindade@goose-yoga ~]$

Output of v4l2-ctl --list-devices
list devices output
```
[gctrindade@goose-yoga ~]$ v4l2-ctl --list-devices
Video Capture 3 (usb-0000:00:14.0-8):
	/dev/video0
	/dev/video1
	/dev/video2
	/dev/video3
	/dev/media0
	/dev/media1
```

Ouput of v4l2-ctl -d /dev/video2 --list-formats-ext
v4l2 device output
[gctrindade@goose-yoga ~]$ v4l2-ctl -d /dev/video2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'GREY' (8-bit Greyscale)
		Size: Discrete 640x360
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)

Additional info

OS: Arch Linux x86_64
Host: 20SA0004US ThinkPad X1 Yoga 4th
Kernel: 5.15.13-arch1-1
CPU: Intel i5-10210U (8) @ 4.200GHz
GPU: Intel CometLake-U GT2 [UHD Graphics]
Version of linux-enable-ir-emitter: 3.2.5
How did you install linux-enable-ir-emitter: AUR

I don't see the problem if you answered "yes" to the first "Did you see the ir emitter flashing (not just turn on) ? Yes/No ? Yes" question. This means that your infrared camera works because you saw the ir emitter blinking.
I'm not sure that I understand your problem ?
Or conversely, I'm not sure that you understand the aim of this software ?

Sorry for the confusion!
When I run the configure command, the IR emitter does flash successfully.
However, if I run the test command, or the run command, I get a message saying "ERROR: No driver is currently saved."
Something similar happens with the fix driver command: "ERROR: No driver files to delete."
Is this normal?

When I run the configure command, the IR emitter > does flash successfully.

So why have you answer yes to the question "Did you see the ir emitter flashing (not just turn on) ? Yes/No ?"
You have to answer no until the emitter works.
And yes the other command don't work because you haven't yet configured the camera.

When I run the configure command, the IR emitter does flash successfully.

So, as the error message said, your infrared camera works natively. You don't to use this software.
For which reason did you install linux-enable-ir-emitter    ? Let me know if I can help you

I installed this originally because the command howdy test initially showed me a series of black-and-white lines, instead of the proper IR image. I then installed this package to see if that would fix the problem.
Anyway, I just ran howdy test again and it to works now.  :smile: Not sure if your package helped or if it was something else; but that doesn't matter anymore.
Thank you for your support. :+1:


Slack integration - works from command line but not directly in python

Rasa Core version: 0.9.6
Python version:  3.6.5
Operating system: Win7
Issue:
The moodbot runs as expected if I use the command prompt:
python -m rasa_core.run -d models/current/dialogue -u models/current/nlu/ --port 5004 --connector slack --credentials slack_credentials.yml
But if I use the code for "directly using python" I get odd behaviour.
The code I am using is as follows. I am testing locally with NGROK.
Can anyone point out what I am missing?
from rasa_core.channels import HttpInputChannel
from rasa_core.channels.slack import SlackInput
from rasa_core.agent import Agent
from rasa_core.interpreter import RegexInterpreter

agent = Agent.load("./models/current/dialogue", interpreter=RegexInterpreter())

input_channel = SlackInput(
   slack_token="xoxb-***************",
   slack_channel="@mychannelname"
)
agent.handle_channel(HttpInputChannel(5004, "/", input_channel))```

you need to load your NLU model as well, by replacing interpreter=RegexInterpreter() with the directory of your NLU model


add docker compose with postgres, mariadb & db2

Hey, thanks for taking a look at this. I don't think that this kind of stuff really belongs in this project (or in zdb) - these are zig libs and this stuff is a bit out of scope. This would belong in a project that depends on one of these libs.
I'd be open to having another repo for project scaffold stuff like this, but I'm not sure if it's something that I want to personally maintain.

No worries. I can add this to my arrow odbc library

I'm trying to update zdb to use the latest zig-odbc and I'm running into a bunch of errors that aren't caught in the zig-odbc test suite.
Revisiting this original PR would you be open to taking a modified PR with integration tests against these databases?


Create test for project reset

since this is rarely used we might break it without realizing it, so we should have a test for it.

This should also let us verify that #515 is fixed.

bumping up the priority of this as we had an issue where it was broken and an admin couldn't recover a user issue due to this. So while this is not frequently used it is an escape hatch so we want to be sure it's working properly.

@hahn-kev - I currently have one working test in #550 (which I'd welcome feedback on if you have any), but I'm still figuring out how I'm going to integrate the Playwright Node tests with a dotnet Send/Receive test in order to test both sides of the equation (i.e., do a Send/Receive after a project reset, all as part of the test). I think I'll end up writing a small command-line tool in C# that can trigger a Send/Receive of the Sena-3 project, and call that command-line tool from Node. Then the Playwright tests are, conceptually, the E2E integration tests, and the C# code acts as a service.

Rethinking that approach. It might be better to create a new .NET test in SendReceiveServiceTests, which drives the API via ApiTestBase (similar to how the SendNewProject test works). The main thing we're concerned about here is that the reset project ends up in a good state (file permissions, etc) after resetting, and driving it via the UI in a Playwright test adds no benefit over driving it via calling the API.
So I'll leave #550 alone for now and start a new PR to create a .NET test or two in SendReceiveServiceTests.

Yeah I think that would be a better approach than creating a tool. It might be difficult to drive the test due to using tus for the upload, but that could be covered in another tests.
I'm thinking we have a playwright test that makes sure we can do a project reset, maybe query /hg/project-code/browse?style=json-lex to determine that an uploaded file in the zip made it where we expect.
Then a S&R based reset test that ensures we can S&R to projects after a reset, but this doesn't need to go through the UI, we could even make a dedicated endpoint for this test to simplify the reset process (as long as it's consistent with the real reset process).

A quick record of my findings so far:
If the /var/vcs/public/sena-3 repo is empty in hgresumable when I try to do a Send/Receive to it, it goes into a nasty looping failure mode where NGINX returns a 499 (client closed connection early), then Chorus Resumable keeps retrying the connection because it doesn't know how to handle a 499. (Since the LexSyncReverseProxy is involved, the "client" that closed the connection might be the other end, i.e. hgweb; I haven't managed to track this down yet).
Logging into hgresumable and doing ps auxw to see what's going on reveals a bunch of processes, one per worker thread, trying to run sh -c hg tip --template "{rev}:{branches}\n" or sh -c hg log --template "{node|short}:{branches}\n" and each using 100% CPU. That looks like something that the resumable PHP code is trying to run, but I haven't yet looked into the PHP code to see what code path tries to run those commands, so it might be something else.
I don't know yet why NGINX is seeing a closed connection on one end, while hgresumable is seeing hg processes (which should complete very quickly) stuck at 100% CPU. I'll continue investigating, but by the nature of the problem this is going to take a little time as every time I trigger this behavior my computer slows down quite a lot.
Here's a copy of the lexbox-api logs for one such cycle:
[lexbox-api] info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
[lexbox-api]       => TraceId:abb237ebcb66279da54f3fdbed543e9f => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000001
[lexbox-api]       Request starting HTTP/1.1 GET http://resumable.localhost/api/v03/pullBundleChunk?offset=0&chunkSize=5000&transId=f375d994-03db-4661-abee-b8c1081101bb&baseHashes[]=fb853b6ed66f&baseHashes[]=1af18777e23e&repoId=sena-3 - - -
[lexbox-api] info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[1]
[lexbox-api]       => TraceId:abb237ebcb66279da54f3fdbed543e9f => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000001
[lexbox-api]       Request:
[lexbox-api]       Protocol: HTTP/1.1
[lexbox-api]       Method: GET
[lexbox-api]       Scheme: http
[lexbox-api]       PathBase:
[lexbox-api]       Path: /api/v03/pullBundleChunk
[lexbox-api]       Host: resumable.localhost
[lexbox-api]       User-Agent: HgResume v03
[lexbox-api]       X-Request-ID: [Redacted]
[lexbox-api]       X-Real-IP: [Redacted]
[lexbox-api]       X-Original-Proto: [Redacted]
[lexbox-api]       X-Forwarded-Host: resumable.localhost
[lexbox-api]       X-Forwarded-Port: [Redacted]
[lexbox-api]       X-Forwarded-Scheme: [Redacted]
[lexbox-api]       X-Scheme: [Redacted]
[lexbox-api]       X-Original-For: [Redacted]
[lexbox-api] info: LexSyncReverseProxy.Auth.BasicAuthHandler[7]
[lexbox-api]       => TraceId:abb237ebcb66279da54f3fdbed543e9f => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000001
[lexbox-api]       HgAuthScheme was not authenticated. Failure message: No authorization header
[lexbox-api] info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
[lexbox-api]       => TraceId:abb237ebcb66279da54f3fdbed543e9f => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000001
[lexbox-api]       Authorization failed. These requirements were not met:
[lexbox-api]       DenyAnonymousAuthorizationRequirement: Requires an authenticated user.
[lexbox-api]       LexSyncReverseProxy.Auth.UserHasAccessToProjectRequirement
[lexbox-api] info: LexSyncReverseProxy.Auth.BasicAuthHandler[12]
[lexbox-api]       => TraceId:abb237ebcb66279da54f3fdbed543e9f => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000001
[lexbox-api]       AuthenticationScheme: HgAuthScheme was challenged.
[lexbox-api] info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler[12]
[lexbox-api]       => TraceId:abb237ebcb66279da54f3fdbed543e9f => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000001
[lexbox-api]       AuthenticationScheme: Bearer was challenged.
[lexbox-api] info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[2]
[lexbox-api]       => TraceId:abb237ebcb66279da54f3fdbed543e9f => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000001
[lexbox-api]       Response:
[lexbox-api]       StatusCode: 401
[lexbox-api]       WWW-Authenticate: Basic realm="SyncProxy",Bearer
[lexbox-api]       lexbox-version: dockerDev
[lexbox-api] info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
[lexbox-api]       => TraceId:abb237ebcb66279da54f3fdbed543e9f => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000001
[lexbox-api]       Request finished HTTP/1.1 GET http://resumable.localhost/api/v03/pullBundleChunk?offset=0&chunkSize=5000&transId=f375d994-03db-4661-abee-b8c1081101bb&baseHashes[]=fb853b6ed66f&baseHashes[]=1af18777e23e&repoId=sena-3 - 401 - application/problem+json 1.1041ms
[lexbox-api] info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Request starting HTTP/1.1 GET http://resumable.localhost/api/v03/pullBundleChunk?offset=0&chunkSize=5000&transId=f375d994-03db-4661-abee-b8c1081101bb&baseHashes[]=fb853b6ed66f&baseHashes[]=1af18777e23e&repoId=sena-3 - - -
[lexbox-api] info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[1]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Request:
[lexbox-api]       Protocol: HTTP/1.1
[lexbox-api]       Method: GET
[lexbox-api]       Scheme: http
[lexbox-api]       PathBase:
[lexbox-api]       Path: /api/v03/pullBundleChunk
[lexbox-api]       Host: resumable.localhost
[lexbox-api]       User-Agent: HgResume v03
[lexbox-api]       Authorization: [Redacted]
[lexbox-api]       X-Request-ID: [Redacted]
[lexbox-api]       X-Real-IP: [Redacted]
[lexbox-api]       X-Original-Proto: [Redacted]
[lexbox-api]       X-Forwarded-Host: resumable.localhost
[lexbox-api]       X-Forwarded-Port: [Redacted]
[lexbox-api]       X-Forwarded-Scheme: [Redacted]
[lexbox-api]       X-Scheme: [Redacted]
[lexbox-api]       X-Original-For: [Redacted]
[lexbox-api] info: Microsoft.EntityFrameworkCore.Database.Command[20101]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Executed DbCommand (15ms) [Parameters=[@__email_0='?'], CommandType='Text', CommandTimeout='30']
[lexbox-api]       SELECT t."Id", t."CanCreateProjects", t."CreatedDate", t."Email", t."EmailVerified", t."IsAdmin", t."LastActive", t."LocalizationCode", t."Locked", t."Name", t."PasswordHash", t."Salt", t."UpdatedDate", t."Username", t0."Id", t0."CreatedDate", t0."ProjectId", t0."Role", t0."UpdatedDate", t0."UserId", t0."Id0", t0."Code", t0."CreatedDate0", t0."DeletedDate", t0."Description", t0."LastCommit", t0."MigratedDate", t0."MigrationStatus", t0."Name", t0."ParentId", t0."ProjectOrigin", t0."ResetStatus", t0."RetentionPolicy", t0."Type", t0."UpdatedDate0"
[lexbox-api]       FROM (
[lexbox-api]           SELECT u."Id", u."CanCreateProjects", u."CreatedDate", u."Email", u."EmailVerified", u."IsAdmin", u."LastActive", u."LocalizationCode", u."Locked", u."Name", u."PasswordHash", u."Salt", u."UpdatedDate", u."Username"
[lexbox-api]           FROM "Users" AS u
[lexbox-api]           WHERE u."Email" = @__email_0 OR u."Username" = @__email_0
[lexbox-api]           LIMIT 1
[lexbox-api]       ) AS t
[lexbox-api]       LEFT JOIN (
[lexbox-api]           SELECT p."Id", p."CreatedDate", p."ProjectId", p."Role", p."UpdatedDate", p."UserId", t1."Id" AS "Id0", t1."Code", t1."CreatedDate" AS "CreatedDate0", t1."DeletedDate", t1."Description", t1."LastCommit", t1."MigratedDate", t1."MigrationStatus", t1."Name", t1."ParentId", t1."ProjectOrigin", t1."ResetStatus", t1."RetentionPolicy", t1."Type", t1."UpdatedDate" AS "UpdatedDate0"
[lexbox-api]           FROM "ProjectUsers" AS p
[lexbox-api]           INNER JOIN (
[lexbox-api]               SELECT p0."Id", p0."Code", p0."CreatedDate", p0."DeletedDate", p0."Description", p0."LastCommit", p0."MigratedDate", p0."MigrationStatus", p0."Name", p0."ParentId", p0."ProjectOrigin", p0."ResetStatus", p0."RetentionPolicy", p0."Type", p0."UpdatedDate"
[lexbox-api]               FROM "Projects" AS p0
[lexbox-api]               WHERE p0."DeletedDate" IS NULL
[lexbox-api]           ) AS t1 ON p."ProjectId" = t1."Id"
[lexbox-api]           WHERE t1."DeletedDate" IS NULL
[lexbox-api]       ) AS t0 ON t."Id" = t0."UserId"
[lexbox-api]       ORDER BY t."Id", t0."Id"
[lexbox-api] info: Microsoft.EntityFrameworkCore.Database.Command[20101]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Executed DbCommand (11ms) [Parameters=[@__id_0='?' (DbType = Guid)], CommandType='Text', CommandTimeout='30']
[lexbox-api]       UPDATE "Users" AS u
[lexbox-api]       SET "LastActive" = now()
[lexbox-api]       WHERE u."Id" = @__id_0
[lexbox-api] info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Executing endpoint '/api/v03/{**catch-all}'
[lexbox-api] info: Yarp.ReverseProxy.Forwarder.HttpForwarder[9]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Proxying to http://hg/api/v03/pullBundleChunk?offset=0&chunkSize=5000&transId=f375d994-03db-4661-abee-b8c1081101bb&baseHashes[]=fb853b6ed66f&baseHashes[]=1af18777e23e&repoId=sena-3 HTTP/2 RequestVersionOrLower
[otel-collector] 2024-02-07T07:07:02.184Z	error	exporterhelper/retry_sender.go:145	Exporting failed. The error is not retryable. Dropping data.	{"kind": "exporter", "data_type": "traces", "name": "otlp", "error": "Permanent error: rpc error: code = Unauthenticated desc = attempted to use disabled API key", "dropped_items": 5}
[otel-collector] go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send
[otel-collector] 	go.opentelemetry.io/collector/exporter@v0.87.0/exporterhelper/retry_sender.go:145
[otel-collector] go.opentelemetry.io/collector/exporter/exporterhelper.(*tracesExporterWithObservability).send
[otel-collector] 	go.opentelemetry.io/collector/exporter@v0.87.0/exporterhelper/traces.go:177
[otel-collector] go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).start.func1
[otel-collector] 	go.opentelemetry.io/collector/exporter@v0.87.0/exporterhelper/queue_sender.go:126
[otel-collector] go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).Start.func1
[otel-collector] 	go.opentelemetry.io/collector/exporter@v0.87.0/exporterhelper/internal/bounded_memory_queue.go:52
[otel-collector] 2024-02-07T07:07:06.178Z	error	exporterhelper/retry_sender.go:145	Exporting failed. The error is not retryable. Dropping data.	{"kind": "exporter", "data_type": "metrics", "name": "otlp/metrics", "error": "Permanent error: rpc error: code = Unauthenticated desc = attempted to use disabled API key", "dropped_items": 53}
[otel-collector] go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send
[otel-collector] 	go.opentelemetry.io/collector/exporter@v0.87.0/exporterhelper/retry_sender.go:145
[otel-collector] go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send
[otel-collector] 	go.opentelemetry.io/collector/exporter@v0.87.0/exporterhelper/metrics.go:176
[otel-collector] go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).start.func1
[otel-collector] 	go.opentelemetry.io/collector/exporter@v0.87.0/exporterhelper/queue_sender.go:126
[otel-collector] go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).Start.func1
[otel-collector] 	go.opentelemetry.io/collector/exporter@v0.87.0/exporterhelper/internal/bounded_memory_queue.go:52
[lexbox-api] warn: Yarp.ReverseProxy.Forwarder.HttpForwarder[48]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       RequestCanceled: The request was canceled before receiving a response.
[lexbox-api]       System.Threading.Tasks.TaskCanceledException: The operation was canceled.
[lexbox-api]        ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
[lexbox-api]        ---> System.Net.Sockets.SocketException (125): Operation canceled
[lexbox-api]          --- End of inner exception stack trace ---
[lexbox-api]          at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
[lexbox-api]          at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
[lexbox-api]          at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)
[lexbox-api]          at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[lexbox-api]          --- End of inner exception stack trace ---
[lexbox-api]          at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[lexbox-api]          at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
[lexbox-api]          at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[lexbox-api]          at System.Net.Http.Metrics.MetricsHandler.SendAsyncWithMetrics(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[lexbox-api]          at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[lexbox-api]          at Yarp.ReverseProxy.Forwarder.HttpForwarder.SendAsync(HttpContext context, String destinationPrefix, HttpMessageInvoker httpClient, ForwarderRequestConfig requestConfig, HttpTransformer transformer, CancellationToken cancellationToken)
[lexbox-api] fail: LexSyncReverseProxy.ForwarderTelemetryConsumer[0]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Forwarder Failed: RequestCanceled
[lexbox-api] info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Executed endpoint '/api/v03/{**catch-all}'
[lexbox-api] info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[2]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Response:
[lexbox-api]       StatusCode: 400
[lexbox-api]       lexbox-version: dockerDev
[lexbox-api] info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
[lexbox-api]       => TraceId:1f2d1affbb391aaf54836fbffe8e837b => ConnectionId:0HN17LBEE0752 => RequestPath:/api/v03/pullBundleChunk RequestId:0HN17LBEE0752:00000002
[lexbox-api]       Request finished HTTP/1.1 GET http://resumable.localhost/api/v03/pullBundleChunk?offset=0&chunkSize=5000&transId=f375d994-03db-4661-abee-b8c1081101bb&baseHashes[]=fb853b6ed66f&baseHashes[]=1af18777e23e&repoId=sena-3 - 499 - application/problem+json 29973.6265ms

The root cause (well, ONE of the root causes) appears to be that Mercurial's behavior on an empty project is not what the PHP code expects to receive. When a project is empty, hg log --template "{node|short}:{branches}\n" returns an empty string, while hg tip --template "{rev}:{branches}\n" returns the string -1:\n, i.e. there's nothing after the colon. This triggers a bug in the isValidBase PHP code, which is fixed by the following patch to hgresumable:
diff --git a/api/src/HgRunner.php b/api/src/HgRunner.php
index 4b186d7..ac9d6bb 100644
--- a/api/src/HgRunner.php
+++ b/api/src/HgRunner.php
@@ -204,7 +204,7 @@ class HgRunner {
                $i = 0;
                while($foundHash < count($hashes)) {
                        $revisions = $this->getRevisions($i, $q);
-                       if (count($revisions) == 0) {
+                       if (count($revisions) == 0 || count($revisions) == 1 && $revisions[0] == "0:") {
                                return false;
                        }
                        foreach($revisions as $hashandbranch) {

That's not enough to get us out of the woods, though. It solves the issue of hgresumable getting stuck in a loop, but Chorus is still stuck in a loop. Because once that patch is applied to the PHP code, Chorus now goes into a loop of receiving a 400 error and retrying it, even though the 400 error is a message from Mercurial saying "Hey, your request is invalid". Specifically, inside hgresume's pullBundleChunkInternal function, there's the following snippet:
			// TODO This might be bogus, the given baseHash may well be a baseHash that exists in a future push, and we don't have it right now. CP 2012-08
			if (!$hg->isValidBase($baseHashes)) {
				return new HgResumeResponse(HgResumeResponse::FAIL, array('Error' => 'invalid baseHash'));
			}

And Chorus is supposed to see the "invalid baseHash" message in the hgresume error and give up. However, it doesn't.
We can't patch Chorus. But we can patch our deployed version of hgresume. I suspect that if you have an empty repo, ALL base hashes are valid, because they are a baseHash that exists in a future push, as Cambell's comment from 2012 mentions. I'll try that and see what happens.

I added a check for an empty repo and had pullBundleChunkInternal return HgResumeResponse::NOCHANGE if the repo is empty. That got Chorus to proceed with pushing the bundle. However, that then made Chorus think that the resumable server had revision 2, so it didn't push a bundle with all the changes, it just pushed a bundle with the changes since revision 2. This resulted in Mercurial erroring out with:
abort: 00changelog@6f010b976aaaab1561a29893bf9820c68af47794: unknown parent
Command exited with non-zero status 255

And that, in turn, produced a nasty retry cycle between Resumable and Chorus where Resumable was repeatedly throwing a PHP exception from AsyncRunner: "PHP Fatal error:  Uncaught AsyncRunnerException: Lock file '/var/cache/hgresume/8ac501c2-225b-4d6c-b365-5df157d30502.bundle.async_run' not found, process is not running". But Chorus just kept retrying that, even though it was never going to get better on the server end.
I have one more thing to try. So far I've been using our Send/Receive tests (the ModifyProjectData test, specifically) to do this testing. That uses the Send/Receive process in LfMergeBridge. But there's one more Chorus code path: the "Send this project for the first time" option in FLEx. I will have to try out that code path on a project that's been reset.

Manual testing on FLEx shows that the "Send this project for the first time" option only appears when there is no .hg folder in the project storage, which is not what we want.
However, I did find a way to work around the issue. If you're using resumable to send/receive the project in FLEx, the project folder (default location: C:\ProgramData\SIL\FieldWorks\Projects\(project name)) will contain a "Chorus" folder. Delete the "Chorus" folder and then everything will work. I'll do one more experiment where the only thing I delete from the Chorus folder is the "revisioncache.db" file (named "revisioncache.json" on Linux), and see if that works.

That gives us a working procedure for a .NET test of sending projects via resumable after a project reset. First create a new project (with a tiny .fwdata file, see the SendNewProject test). Then do the following steps in the test:

Clone project with resumable
Reset project via API calls (and leave it empty)
Delete the "Chorus" folder in the project clone
Send/Receive project with resumable
Clone project again into new folder
Verify that old folder and new folder have same hg tip SHA

For the Playwright reset-project test, create a new project (with a random project code so the test can run in parallel with itself) and commit a single file into it. Verify that we can see that file via /hg/browse in hgweb. Reset project to empty. Verify the file is gone. Reset project again, but this time upload the .zip file. Verify the file is visible again in hgweb.


Add influx tests

This PR adds influx handler tests

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 63.97%. Comparing base (bebe565) to head (7e56d35).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #18      +/-   ##
==========================================
+ Coverage   56.93%   63.97%   +7.03%
==========================================
  Files          13       13
  Lines         483      483
==========================================
+ Hits          275      309      +34
+ Misses        181      145      -36
- Partials       27       29       +2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Test for Taowei

First make sure you read the test guidelines here: https://github.com/Wiredcraft/test-pm
This task will be followed up asynchronously over a few days and should not take more than two hours in total.
Project Introduction
On behalf of Wiredcraft (WCL), you have been assigned as project manager for a project for the client Kuno Corp (your client stakeholder is Leo). The outcome of this project will be a Wechat miniapp which allows a user to sign up and pay for membership to a coupon system. The system as a whole is expected in 9 weeks.
Project Details
The work has been divided into two parts:
• the backend and frontend for wechat miniapp signup (which requires the user to pay a yearly fee of 100 rmb) as well as the frontend for coupon information - This will be handled by Wiredcraft completely
• the backend for coupon information which will be provided by a separate company Jhonny Labs, you will need to interface with Frank, their project manager, to coordinate as they are building this system from scratch for Kuno Corp.
The specific requirements for the registration have been decided as the default Wechat authentication and WCL will need to store Username and location fields, at the last stage of signup the user needs to pay 100 rmb from their Wechat wallet for 1 year of membership. The login for existing users will be Wechat authentication as usual. And after sign up the coupin information system expects to receive the Username and location so they can customise the coupon information in their backend for WCL to access and display in the front-end.
In your team you have:
• Two frontend developeres who can make mockups and flow diagrams and once confirmed, implement the frontend parts.
• A developer who can implement the WCL backend and help out integrating the coupons frontend with the coupons backend
• A dev-ops engineer who can set up the servers and take care of deployment.
Your Task
The first step of this task is for you to provide the following:
• Clarify the requirements and provide your understanding of the final scope for Wiredcraft's work.
• An initial project plan including:

A work breakdown for Wiredcraft's integration with Jhonny Labs up to the final delivery
a rough timeline of what you think needs to happen in this project and deliverables. You can use the methodology of your choice for this plan.
Outline the project phases you would have and what should be delivered at each phase.
The plan can be a few paragraphs of text and the timeline can also be made using Github markdown by outlining what needs to be done during each week.
A rough outline of the process you would use for the day-to-day organisation of the development team and a short overview of how you would manage the communication flows internally and externally.

You can also kick off with any questions to multiple people in one GitHub comment, I will answer likewise and bring up more cases to test your communication skills as we go along:
(Assume you have already been introduced to the client (@leo111) and the coupons service vendor (@flfrank222) ) and you can @myteam333 for questions to your development team)
e.g.
@flfrank222 - Hi there, can I confirm with you the requirements for ...
@leo111 - I heard from FLFrank222 that they need to have xxx, does that mean we need to change ....
@myteam333 - I'm not clear on how xxx could work, can you tell me more about it?
@fuzhuang  - For the test, I don't understand what you mean by xxx ...

Prerequisite:

Assume prototype design has finished by product manager.
The introduction did not mention about second payment logic which is still part of prototype design scope.

Hi Team @myteam333,
There would be Kuno Corp project kick-off meeting to tomorrow at 10:00 am.
Please refer to project overview below in preparation for the meeting.
Let me know if you have any questions.
Kick-off meeting details:

Project overview / Q&A
Setup design workshop meeting
Setup sprint planning meeting
Update project start and plan info to Leo and Frank after meeting

Project Overview: Kuno Corp
Client: Kuno Corp
Timeframe: 9 weeks
API provider: Johnny Labs responsible for the coupon information.
Outcome: a wechat miniapp which allows a user to sign up and pay for membership to a coupon system
Wiredcraft project scope: Wechat authentication for sign up/login; store username, location fields and then pay 100 rmb from wechat wallet; send username and location fields to the coupon system; display customized coupon info
Specific requirements
Register page: (fields)

Wechat authentication
Store Username and location fields
Pay 100 rmb from Wechat wallet for 1 year of membership

Login page: (fields)

Wechat authentication
Display customized coupon info

Information required by API provider: (access service)

Username
Location

Sprint plan overview:
Sprint planning:

Sprint goal definition
Backlog items prioritisation
Sprint backlog (item + sub - tasks)

Sprint execution:

Perform task-level work

Daily scrum:

Yesterday ‘done’ review
Next task assignment
Might discuss task obstacles

Sprint review:

Review ‘Done’ user stories
Gather feedbacks to adapt the product

Sprint retrospective:

Discuss what is not working
Adapt the process (improvement actions)

QA Plan:
Preparation:

Goals identification
Critical success factors identification
Write down test cases for each user scenario

Implementation:

Continuous integration
Acceptance testing
Test units
Functional testing

Reporting
UAT Plan:
Preparation:

Validate user journeys
Consolidate test cases for each scenario
Setup environments
Test account creations
Issues tracking (device type; OS update…)

Execution
Post-execution:

List all issues, document them
Fixed identified issues
Re-assess

Project Plan
Project introduction
Client: Kuno Corp
Project background:
A wechat miniapp which allows a user to sign up and pay for membership to a coupon system
Success criteria:

QA
UAT
Client’s feedback

Project definition
Methodologies & tools:

Scrum
Tools:
Calendar
Slack - internal communication
Email - external communication
Github - backlogs & user stories board / host and review code
Dependency:
Kuno Corp
Johnny Labs
Constraints:
Timeframe: 9 weeks
Technical: API service provider
Resources: 1 project manager / 2 frontend developers / 1 backend developers / 1 dev-ops engineer

Deliverables:
Wechat miniapp which allows a user to sign up and pay for membership to a coupon system. Johnny labs provide API for coupon system.

Backend (payment, API integration, signup/login logic)
Frontend (mockups, flow diagrams, signup/login)
Dev-ops (server setup, deployment)
QA
UAT

Requirements:

Signup: Wechat authentication -> Store Username and location fields -> Pay 100 rmb from wechat wallet for 1 year of membership
Login: Wechat authentication
Fields send to coupon system: username / location
Display customized coupon info

WBS baseline
week1

Define, clarify, assess project outcome & requirements
Prepare risk & cost control document.
Update members attendance information for next 9 weeks.
User journeys draft

week 2

Product mapping and break down
Personas - User stories
Clarify project requirements - Product backlog review - Milestones definition
Discuss & schedule milestones - Refine strategy - Outline team roles
User research - User flows - Information architecture

week 3

Mockup and workflows discussion
Back-end data structure design
Assess & consolidate product design
Prepare miniapp account and environment
Sprint preparation: Prepare repository, backlog, prioritization, authorization, labels and update tasks on github

week 4 and 5
Sprint 1

sprint 1 kick-off meeting: Assign tasks - Weight user stories - Refine sprint backlog
Dev: Server setup, signup/login, API, web page structure, payment
PM: Keep track of project advancements & solve facing issues
Sprint 2 preparation - Client and Johnny labs update
Sprint 1 Review meeting: Review ‘Done’ user stories - Adapt next sprint backlog
Sprint 1 Retrospective meeting: Gather feedbacks from the team - improve process

week 6 and 7
Sprint 2

sprint 2 kick-off meeting: Assign tasks - Weight user stories - Refine sprint backlog
Dev: Payment, Integration part with Johnny labs; Display front-end info after integration; Deployment
PM: Keep track of project advancements & solve facing issues
Sprint 3 preparation - Client and Johnny labs update
Sprint 2 Review meeting: Review ‘Done’ user stories - Adapt next sprint backlog
Sprint 2 Retrospective meeting: Gather feedbacks from the team - improve process

week 8 and 9
Sprint 3

sprint 3 kick-off meeting: Assign tasks - Weight user stories - Refine sprint backlog
Dev: Unit test; buffer time for disapproved issue; Bug fixing; Optimization
PM: Keep track of project advancements & solve facing issues
Sprint 3 Review meeting: Review ‘Done’ user stories - Adapt next sprint backlog
Sprint 3 Retrospective meeting: Gather feedbacks from the team - improve process
Testing
Achievements review

Project Plan
Project introduction
Client: Kuno Corp
Project background:
A wechat miniapp which allows a user to sign up and pay for membership to a coupon system
Success criteria:

QA
UAT
Client’s feedback

Project definition
Methodologies & tools:

Scrum
Tools:
Calendar
Slack - internal communication
Email - external communication
Github - backlogs & user stories board / host and review code
Dependency:
Kuno Corp
Johnny Labs
Constraints:
Timeframe: 9 weeks
Technical: API service provider
Resources: 1 project manager / 2 frontend developers / 1 backend developers / 1 dev-ops engineer

Deliverables:
Wechat miniapp which allows a user to sign up and pay for membership to a coupon system. Johnny labs provide API for coupon system.

Backend (payment, API integration, signup/login logic)
Frontend (mockups, flow diagrams, signup/login)
Dev-ops (server setup, deployment)
QA
UAT

Requirements:

Signup: Wechat authentication -> Store Username and location fields -> Pay 100 rmb from wechat wallet for 1 year of membership
Login: Wechat authentication
Fields send to coupon system: username / location
Display customized coupon info

WBS baseline
week1

Define, clarify, assess project outcome & requirements
Prepare risk & cost control document.
Update members attendance information for next 9 weeks.
User journeys draft

week 2

Product mapping and break down
Personas - User stories
Clarify project requirements - Product backlog review - Milestones definition
Discuss & schedule milestones - Refine strategy - Outline team roles
User research - User flows - Information architecture

week 3

Mockup and workflows discussion
Back-end data structure design
Assess & consolidate product design
Prepare miniapp account and environment
Sprint preparation: Prepare repository, backlog, prioritization, authorization, labels and update tasks on github

week 4 and 5
Sprint 1

sprint 1 kick-off meeting: Assign tasks - Weight user stories - Refine sprint backlog
Dev: Server setup, signup/login, API, web page structure, payment
PM: Keep track of project advancements & solve facing issues
Sprint 2 preparation - Client and Johnny labs update
Sprint 1 Review meeting: Review ‘Done’ user stories - Adapt next sprint backlog
Sprint 1 Retrospective meeting: Gather feedbacks from the team - improve process

week 6 and 7
Sprint 2

sprint 2 kick-off meeting: Assign tasks - Weight user stories - Refine sprint backlog
Dev: Payment, Integration part with Johnny labs; Display front-end info after integration; Deployment
PM: Keep track of project advancements & solve facing issues
Sprint 3 preparation - Client and Johnny labs update
Sprint 2 Review meeting: Review ‘Done’ user stories - Adapt next sprint backlog
Sprint 2 Retrospective meeting: Gather feedbacks from the team - improve process

week 8 and 9
Sprint 3

sprint 3 kick-off meeting: Assign tasks - Weight user stories - Refine sprint backlog
Dev: Unit test; buffer time for disapproved issue; Bug fixing; Optimization
PM: Keep track of project advancements & solve facing issues
Sprint 3 Review meeting: Review ‘Done’ user stories - Adapt next sprint backlog
Sprint 3 Retrospective meeting: Gather feedbacks from the team - improve process
Testing
Achievements review

Hi @fuzhuang, just want to make sure if there's 3 comment besides this one, because there were 2 identical comments of project plan while if I deleted one of them, both would be disappeared which seemed really weird.

Hi @mix33 Thanks for the fast response!
Could you update the project plan/wbs baseline section with the below:

elaborate on the dependencies? You current answer only listed two parties that we will have dependency with, could you list what their dependency items will be and pls incorporate that into your project plan;
Looks like QA and UAT are not part of the plan, pls assume 9 weeks are for us to launch, so pls include all needed items to reflect this;
From all the project requirements, could you pick up 3 features to write up a high quality user story with acceptance criteria?

Thanks.

Hi @fuzhuang Thank you very much for your reply. I've updated project plan as suggested and below you can find 3 features user story with acceptance critera.

3 User story with acceptance criteria
As a new user I want to sign up for the coupon system.

See a sign up button page for new user after QR scan or miniapp name search
Trigger authentication page after click the sign up button with allow and deny button
See location and others authentication info on the page
Show main page after choose allow
Back to sign up button page after choose deny

As a user I want to pay member fee

Trigger payment page from membership page or main page if not a member
See payment amout
Verify payment by our payment processor
Show confirmation after finalizing

As a user I want to view a list of coupons so I can select some

See a thumbnail image and name for each coupon
Click to view details for coupon
Add to cart from detail page
Search for a coupon
View coupon by category


Enable to start exemple

Thanks for the initiative!
It seems to look for a specific patched version of onnx
Could not find com.microsoft.onnxruntime:onnxruntime:1.13.1.patched
Specifying an explicit version will find it but I'm not sure it's compatible.
implementation("com.microsoft.onnxruntime:onnxruntime:1.13.1")
I run through a lot of errors then
java.lang.ClassNotFoundException: javaslang.λ

I'm sorry for long answer, I just didn't get notification about your issue. implementation("com.microsoft.onnxruntime:onnxruntime:1.13.1") is not compatible, you mustn't override it.
Please add this to your build.gradle.kts file:
repositories {
        maven(url = "https://packages.jetbrains.team/maven/p/ki/maven")
        maven(url = "https://packages.jetbrains.team/maven/p/grazi/grazie-platform-public")
}

and remove implementation("com.microsoft.onnxruntime:onnxruntime:1.13.1")
I'll add it to our README


Batched CRF in pytorch

This is a batched version of a CRF in pytorch. When I was testing the CRF masks in pytorch each epoch was way too slow so I batched it.
Old Speed:
Train: 112.7518 ± 1.064
Dev:     4.4706 ± 0.023651

Batched Speed:
Train:   37.0712 ± 0.143
Dev:      2.1887 ± 0.009753

Tested by running examples through the old and the new CRF and making sure the values match. Also I tested changes to the tagger model by training a tagger with CRF and mask, with CRF and no mask, and no CRF. All models successfully trained.

Testing this

I tested this with 10 runs each of old and new. Verified no regression of performance!


Leverage rust marginalization function for marginal_counts()

Summary
In #8026 we added a new marginalization function marginal_distribution()
which was a standalone marginalization function for counts and
distribution dictionaries. The core of that new function was written in
rust for performance. In that PR we didn't use the rust core because
it's behavior was slightly different especially around the order of
indices. This commit reworks the internals of the marginal_counts()
function to leverage the same rust core function to ensure the ordering
is consistent after the migration the index sort is changed.
Details and comments

I've removed the on hold from this, I was misremembering #8051 and it doesn't have an efficient rust bin to hex converter (it has a hex to bin converter). Let me quickly look at adding a hex return option to the rust marginal distribution functions quickly right now but we shouldn't necessarily block on it since I doubt the extra iteration will be a real bottlneck.

This is too stale, I'm just going to close this. It was never critical I'll open up a different PR to update the documentation around the function to just point people towards marginal distribution instead of marginal counts.


Compatibility issues with EnderIO +another mod?

Compatibility issues with EnderIO. causes Server crash when teleporting to overworld (nether, Aroma, ect).
My tracking this bug  info at EnderIO.
https://github.com/SleepyTrousers/EnderIO/issues/3626
Sha

Please update Deep Resonance and ElecCore. You're using an outdated version

I have that log was from the start of the issue. Read the entire report please.
read test #3
""
Test 1: IO/core only. No crash (10 portals in new chunks with 150+ teleports).
Test 2: testing update IO/Core with full pack (and updating the rabbit hole of mods):
Updated IO/Core, Forge and JEI: same results on crash.
Test 3: Updated All mods; same crash
Test 4: starting to add/remove mods leaving IO/core to see if i can track down the mod conflict.
""

If it crashes with the lastes version of both DR & ElecCore, please upload the entire game log.

Is it intentional that there is consistant liquid loss when using the enderio fluid conduits i am really confused(eg i had 10B in a tank and transfered it to recieve 6B)

No but are you sure the liquid isn't lost due to the laser? Do you have a laser firing on the tank?

No, it simply tranfers the 100q 85p 10s 10e liquid from one tank to another abot 20blocks worth of pipe via pressurized fluid conduit.it does this even when broken replaced, other tanks.used other liq etc.

Basicaly in any confoguration or form. I cant give you the exact version now but it did update on aug 30th, ill send the versions tomorrow when i have access to the computer

By the way i have noticed thw issue coming and going in previous versions as well, but i just asumed it was intentional because it is always abot 40% of the liquid that is lost(i didnt have many cases to examine but i am pretty sure that is the case

Also i abologize for awful spelling and grammar, but the screen of my phone is barely functional

Also correction not aug the 30th but aug 29
stupid date autocorrect...

Alright i think i have found ehy that is happening so here goes. As we all know the EIO conduits have  4 modes: Force push, force pull, both , disabled. And the DR tanks have 3 modes: Force pull, force push, disabled. Neither the tank nor the conduit have what would be the equivalent yo TE4 default mode, where the conduit is connected, but it doesnt push or pull, but insthead relies on the tank "telling" it what to do. I have tested and the cause of losing part of the liquid is because the EIO pressurised fluid conduit tries to pull 100mb(default extract rate) and the tank tries to push 100mb in the same tick(doesnt occour every tick but still jappens a lot). So while i think it would be impossible to change the code of either of the mods, there is a simple solution: Simply tell the DR tank to push into a EIO tank and then pull with a conduit, and do vice versa on the recieving end. The versions of theods are irrelevant since i tested on a LOT of versions of either mod, yielding the same result. I really hope i was clear in trying to ecplain what is causing the issue, english isn't my native language. Thank you for the attention anyway MCJty

So what makes the liquid getting lost? The conduit shouldn't pull nor accept if it has no place to put it. With our tanks combining conduit pull and tank push works fine.
Sent from my iPhone

On 30 Aug 2016, at 15:40, thwafflemymx<EMAIL_ADDRESS>wrote:
Warning: This message has had one or more attachments removed
Warning: (msg-20989-15.html).
Warning: Please read the "j-e-b-Attachment-Warning.txt" attachment(s) for more information.
Alright i think i have found ehy that is happening so here goes. As we all know the EIO conduits have  4 modes: Force push, force pull, both , disabled. And the DR tanks have 3 modes: Force pull, force push, disabled. Neither the tank nor the conduit have what would be the equivalent yo TE4 default mode, where the conduit is connected, but it doesnt push or pull, but insthead relies on the tank "telling" it what to do. I have tested and the cause of losing part of the liquid is because the EIO pressurised fluid conduit tries to pull 100mb(default extract rate) and the tank tries to push 100mb in the same tick(doesnt occour every tick but still jappens a lot). So while i think it would be impossible to change the code of either of the mods, there is a simple solution: Simply tell the DR tank to push into a EIO tank and then pull with a conduit, and do vice versa on the recieving end. The versions of theods are irrelevant since i tested on a LOT of versions of either mod, yielding the same result. I really hope i was clear in trying to ecplain what is causing the issue, english isn't my native language. Thank you for the attention anyway MCJty
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/McJty/DeepResonance/issues/111#issuecomment-243441768
This is a message from the MailScanner E-Mail Virus Protection Service
The original e-mail message contained potentially dangerous content,
which has been removed for your safety.
The content is dangerous as it is often used to spread viruses or to gain
personal or confidential information from you, such as passwords or credit
card numbers.
Due to limitations placed on us by the Regulation of Investigatory Powers
Act 2000, we were unable to keep a copy of the original attachment.
The content filters found this:
MailScanner: Found a script in HTML message
--
Postmaster
MailScanner thanks transtec Computers for their support

yes I'm also curious about how this could happen. And even if it occurs in the same tick then still there should normally not be a loss as the conduit would simply get double the amount of liquid then. Right?

Exactly. The conduit would either take it (if there was space), do a "push through" (if there's space in an adjacent target) or reject the amount it cannot take.
Sent from my iPhone

On 30 Aug 2016, at 15:48, McJty<EMAIL_ADDRESS>wrote:
Warning: This message has had one or more attachments removed
Warning: (msg-20989-22.html).
Warning: Please read the "j-e-b-Attachment-Warning.txt" attachment(s) for more information.
yes I'm also curious about how this could happen. And even if it occurs in the same tick then still there should normally not be a loss as the conduit would simply get double the amount of liquid then. Right?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/McJty/DeepResonance/issues/111#issuecomment-243444229
This is a message from the MailScanner E-Mail Virus Protection Service
The original e-mail message contained potentially dangerous content,
which has been removed for your safety.
The content is dangerous as it is often used to spread viruses or to gain
personal or confidential information from you, such as passwords or credit
card numbers.
Due to limitations placed on us by the Regulation of Investigatory Powers
Act 2000, we were unable to keep a copy of the original attachment.
The content filters found this:
MailScanner: Found a script in HTML message
--
Postmaster
MailScanner thanks transtec Computers for their support

... Put a tank and a purifier next to itm put a valve below said tank with the 85p config. Put a tank below the valve. Set one side of the tank to push. Connect it to conduit with extract- always active. Connect to other tank which will accept the liquid. Enjoy the result

I really cant explain it more than i did in my long post. Now you can rescreate the same bug. I really am not that good w programming


Remove unused /nsmd/client.go

Signed-off-by: Viktoria Dlugopolskaya<EMAIL_ADDRESS>
Description

Motivation and Context

How Has This Been Tested?

[ ] Covered by existing integration testing
[ ] Added integration testing to cover
[ ] Tested locally
[ ] Have not tested

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

Interesting how this file persisted through time and was never used.

The last usage of this method was removed in https://github.com/networkservicemesh/networkservicemesh/pull/604:
https://github.com/networkservicemesh/networkservicemesh/pull/604/files#diff-568b6744efbc8d485e84ab7fa3770440L87


Undefined Ratios and Fairness Metric Testing

Overview
AIF360's sklearn.metric.ratio function returns 0.0 where the denominator of a ratio is 0, rather than undefined. This can be misleading, although the other functionality is still useful. This PR adds code to return NaN values in those cases.
Also in this PR are related updates to test__fairnessmetrics to increase robustness. Some cleanup was done in the associated scripts __testing_utilites and _preprocessing, and a ceiling was placed on the version number for AIF360 in setup.py.
Resolves: bugs detected during documentation update
How was this update tested?
pytest and active testing in ToolUsage notebooks
Checklist

[x] I have read the Contributing guidelines
[ ] I performed a self-review of my own code
[x] The CHANGELOG and version.json have been updated
[x] Documentation has been updated to reflect the changes

[x] The code has clear and helpful comments, particularly in hard-to-understand areas
[x] READMEs, Tutorials/Examples, and links have been updated as appropriate

[x] Tests have been added to prove that the fix is effective or that the feature works
[x] Pytest was run; new and existing unit tests pass locally with my changes
[x] I have made note of any breakages or dependencies (if applicable)
[x] Any dependent changes have been merged and published in downstream modules
[x] These changes generate no new warnings

Dropping this PR in favor of newer PR (#66)


Rework of app design

Description
App should be reworked following this prototype
Acceptance criteria

[ ] Code is commented
[ ] Code uses typescript
[ ] Result presents the prototyped design
[ ] Legend can be a placeholder
[ ] Page should stay responsive (For bigger screen mostly)

is this prototype final?
here's some feedback:
the left sidebar looks nice but it will permanently take up space, which was the original reason of making it floating on the right
Putting the event legend on the top left feels awkward, feels unfinished
Stacking the graph options onto the filter panel makes it very crowded with information

is this prototype final?
here's some feedback:
the left sidebar looks nice but it will permanently take up space, which was the original reason of making it floating on the right
Putting the event legend on the top left feels awkward, feels unfinished
Stacking the graph options onto the filter panel makes it very crowded with information

We are currently working on it so it is not the final prototype with all the features implemented :) (Luckily)
The main idea is to have that bar on the left side and be able to toggle off every panel.
The panels would stack within the space they are allocated using a combination of cards and scroll when overflow.
The graph is currently not taking all the space it can on the left side so it would not be very different, just clearer. (Panels would float on the graph space when they are activated)
Ideally users will not need to have all panels open (we will add more panels with the features that are coming, like for shortcuts for instance).
We will also rework the filtering view to make it a bit more compact and clearer (without changing the overall functionning).
We think that by allowing to hide the filter view (requirement from customer), users would be able to manage how that space is crowded. And if they want to open several panels at the same time, there will be a possibility to collapse them as well as scroll in that panel column space.
We also considered the possibility to have the slide bar sliding in and out on hover to make space for the graph but we want to talk to the customer first to see if it is something they would feel is useful or annoying (we can also make it an option eventually).
A bit difficult to explain this through words haha! We have planned on reworking a prototype that shows in a clearer way our full intention. This one used screenshots to be able to have a base to go from which limits the ability to make an actual prototype.
If I was unclear by any mean, we can take a moment to chat over zoom, or I can make a quick prototype just with areas to show you the intention =)

Alright ping this issue when the prototype has evolved

@Kevinayad you can post the link here when you are done with the prototype :)


When clicking on tags in editor, it should show the preview and not redirect you to the tag page

This is pretty tricky to do, so as a compromise, they just open in a new tab instead of changing the current page that you're on.


boost::asio::io_service -> boost::asio::io_context

io_service is a deprecated name, I think asio renamed this as part of aligning with the networking TS (possibly coming in C++23? who knows). This likely isn't the only thing which changed, but it stands out when reading the codebase. https://www.boost.org/doc/libs/1_79_0/doc/html/boost_asio/net_ts.html is a reference if anyone's interested

I guess, we can change the variable name from ioService to ioContext. But that will not prevent us for merging this PR.


Calculation for "most likely" maximum turnip price

Currently, the price chart shows a wide range in which the highest price in a week could be. I propose the addition of a median highest price.

Below is my price chart for this week:

Now, I've never taken a statistics class in my life, so I'm not sure if there is a better way of doing this,  if I'm doing it all wrong, or even if this methodology is statistically sound. What I did was first make three matrices in GNU Octave, each a range between a Guaranteed Minimum and it's corresponding Potential Maximum. Next, I concatenated each of them into a fourth matrix a number of times equal to the percent likeness multiplied by 100. Finally, I took the median, and also produced the following box plot.

(I know a violin plot would be significantly more appropriate for this, but I could not figure out how to produce one.)
Anyway, the "most likely" maximum turnip price for my week was 412. This would make sense considering that the likeliness of the pattern for this week being a big spike is 59%. I now know roughly at what price I can expect to sell my turnips for, excusing huge margin of error.
I would like to see this information somehow presented.

This is being discussed under #217. The current proposal there is to have an alternative graph display, that shows a violin plot for each day of the week, showing the probabilities for that day.
Having some sort of summary for the week might be useful too.. Though it's complicated by various decisions on when you might sell.
It's not actually uniform in some of the ranges either, in the small spike pattern there's a double selection of random variables going on, which makes it a weird distribution

I managed to figure out how to produce a violin plot in case you or anyone is curious.


Using FFmpeg to record a test that goes through multiple browsers causes test to hang when running multiple tests

What is your Test Scenario?
I want to run multiple tests, some of which open multiple windows using the t.openWindow() function, and be able to record them using FFMpeg.
What is the Current behavior?
Say we have a test A that opens multiple windows using t.openWindow(). Then we have a test B that may or may not open multiple windows. Then we run testcafe chrome A.js B.js --video artifacts/videos. Test A will complete as expected but when it moves on to test B it will get stuck and run until there's a FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory error. For some reason this doesn't happen every time I run it, but it happens most of the time. Removing either the --video option or the t.openWindow() calls prevents this from occurring. My guess is that FFmpeg has some problem dealing with multiple windows opening in a test.
What is the Expected behavior?
The tests all run and are recorded regardless of whether they open multiple windows or not.
What is your web application and your TestCafe test code?
Code is proprietary. Website needs authentication. I made a github repo that reproduces the issue.
Steps to Reproduce:
Check out this repo I made to easily reproduce it.
Your Environment details:

testcafe version:                   1.14.2
node.js version:                   v14.16.1
command-line arguments:  testcafe chrome mptest1.js mptest2.js --video artifacts/videos
browser name and version:  chrome 89
platform and version:          macOS 10.15.7
other:                                   Sometimes it works! Please run it a few times in a row until it fails.

Thank you for your excellent report. I reproduced the bug. Please watch for our updates on this issue to keep abreast of our progress.
For team:
This issue looks similar to https://github.com/DevExpress/testcafe/issues/6037.

Hi TestCafe Team,
Is there any update on this issue?
We are facing it and is very easy to reproduce even with the Getting Started example.
Environment details:
TestCafé version: 1.16.1
Node.js version: v16.13.0
Command-line: testcafe chrome tests/getting_started.js
Browser name and version: Chrome 95.0.4638.69
Platform and version: MacOS 10.15.7
Other: .testcaferc.json
{
    "skipJsErrors": true,
    "stopOnFirstFail": false,
    "disableMultipleWindows": true,
    "videoPath":"./videos",
    "videoOptions": {
      "pathPattern": "${DATE}_${TIME}/${TEST}/${USERAGENT}/${TEST_INDEX}.mp4"
      },
    "reporter": [
      {
        "name": "spec"
      },
      {
        "name": "json",
        "output": "./reports/report.json"
      },
      {
        "name": "xunit",
        "output": "./reports/report.xml"
      },
      {
        "name": "html",
        "output": "./reports/report.html"
      }
    ]
  }

We are disabling video recording by removing those options from the testcafe config file to be able to run our tests in chrome.
Thank you!

Thank you for your input. We reproduced the issue and need some time to investigate it. Please stay tuned.

This is currently affecting our test runs as well. We'd really like to be able to reliably record our test runs to debug failures in our GitHub Workflows but as of right now, the process often hangs until the process throws an out of memory error.

Hi TestCafe Team,
Is there any update on this issue? Or, is there any workaround?
Thank you!

I think I've bumped into same issue, going over debugger I think I see a loop here
    async _capture () {
        while (this.active) {
            try {
                const frame = await this.connection.provider.getVideoFrameData(this.connection.id);

                if (frame) {
                    await this.emit('frame');
                    await this._addFrame(frame);
                }
            }
            catch (error) {
                this.debugLogger(error);
            }
        }
    }

caused by this part here

throwing an exception
Error: WebSocket is not open: readyState 3 (CLOSED)
    at sendAfterClose (/xxxx/node_modules/ws/lib/websocket.js:967:17)
    at WebSocket.send (/xxxx/node_modules/ws/lib/websocket.js:405:7)
    at Chrome._enqueueCommand (/xxxx/node_modules/chrome-remote-interface/lib/chrome.js:286:18)
    at /xxxx/node_modules/chrome-remote-interface/lib/chrome.js:88:22
    at new Promise (<anonymous>:null:null)
    at Chrome.send (/xxxx/node_modules/chrome-remote-interface/lib/chrome.js:87:20)
    at Object.handler [as captureScreenshot] (/xxxx/node_modules/chrome-remote-interface/lib/api.js:32:23)
    at BrowserClient.getScreenshotData (/xxxx/node_modules/testcafe/lib/browser/provider/built-in/dedicated/chrome/cdp-client/index.js:239:50)

and then going through the loop again and again
Not sure what's the root cause, but if such error happens maybe worth changing browser active state?

Thank you for your research. We will take your results into account when fixing this issue.

Hi folks,
I tried reproducing the issue with<EMAIL_ADDRESS>and everything works fine. Could you please run your tests with this version and let us know your results?


Req. #01 - Submit FPGA Request

Requirement Description
Description: Request Chosen FPGA.
Artifact: We will show that we have submitted a request for the FPGA we chose in Sprint 1 (Zynq-7000).
Story Points: 1

This is completed and the request has been approved.


Eliminate MD rule suppression for headers being surround by blank lines

The linter rule is 'MD022' and the failing pages currently include: /publications, /about, and /faq.
This should be able to be fixed once the rest of the CSS/SASS is optimized so we don't have to add specific classes to headers.

This issue was resolved.


fix: check for activeElement before calling blur() on it

Description

Currently the tests throw errors in IE11 on Windows 10 because there might be some race conditions or differences in the browser behaviors as document.activeElement is not always defined.

Closes https://github.com/zurb/foundation-sites/issues/11213

Motivation and Context

Fixes the unit tests in IE11 on Windows 10.
Types of changes

[ ] Documentation
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing
functionality to change)

Checklist (all required):

[x] I have read and follow the CONTRIBUTING document.
[x] There are no other pull request similar to this one.
[x] The pull request title is descriptive.
[x] The template is fully and correctly filled.
[x] The pull request targets the right branch (develop or support/*).
[x] My commits are correctly titled and contain all relevant information.
[x] My code follows the code style of this project.
[ ] I have updated the documentation accordingly to my changes (if relevant).
[ ] I have added tests to cover my changes (if relevant).
[x] All new and existing tests passed.

I still guess that the tests on Browserstack run slower than on real devices.

chore: add missing space between condition and statement

Prefix depends on the type of change, not where it is. Here, it should be style

So this line is not the issue.

The testsuite tells differently but ok. How would you solve this here? Not sure if we can solve these issues related to timing functions when the browser / VM is too slow.

I'll take a look at this later.

Will do this tomorrow

So the bug with the previous Interchange tests came from the browser not able to respect the setTimeout delays, which lead to big differences between several timeout running in parrallel. If the difference is bigger than the utils.trigger debounce time (10ms), several reflow may be triggered.

Well, in general setTimeout is never exact when it comes to timing as it is often longer than what is set as timeout.
So what is the correct solution in your opinion? Keep in mind that the VM might be slower and timeouts may trigger later.

See https://github.com/zurb/foundation-sites/pull/11259

If you're okay with these PRs

Which PRs? You linked just one? I thought these replace mine?

Sorry, see https://github.com/zurb/foundation-sites/pull/11259 and its required fix https://github.com/zurb/foundation-sites/pull/11258

Closed in favor of #11260


Reprojection planes

Since Exokit can do a composite of multiple sites/environments at once we need to handle the case where the sites are not in sync, or are misbehaving in the render loop. The current behavior is to just best-effort all of the sites serially but that results in the MR systems thinking there is a hang and suspending to the tracking environment when something misbehaves.
I think we can do this much better.
Basically I see an implementation where Exokit manages the render loop for sites in their own threads, and if they are too slow then reproject the old view (depth-aware) while still drawing the rest of the sites.
I don't see any technical blocker with doing this, and it might improve perceived performance on every site that misses the frame deadline, while still having no negative effect if the site keeps up.
This is especially important for something like Emukit where the timing of the code can be wildly variable.

I think the most important part of this is to get the happy case perfect.
That is, we want zero desync in the render loop for behaving sites, so the MR system tracks perfectly until it can't. Notably we shouldn't just go to pure async rendering for sites, because that would be a regression from the perfect sync we currently have.


[Popover] Add option to disable Menu auto positioning

Closes #10968

@nicoffee Are you sure it's enough to get it working?
https://codesandbox.io/s/rj409287mq

Oh, sorry, i will dig some more into it.


ImageFolder add option to ignore some folders like '.ipynb_checkpoints'

Is your feature request related to a problem? Please describe.
I sometimes like to peek at the dataset images from jupyterlab. thus '.ipynb_checkpoints' folder appears where my dataset is and (just realized) leads to accidental duplicate image additions. I think this is an easy enough thing to miss especially if the dataset is very large.
Describe the solution you'd like
maybe have an option ignore or something .gitignore style
dataset = load_dataset("imagefolder", data_dir="./data/original", ignore="regex?")
Describe alternatives you've considered
Could filter out manually

Maybe it would be nice to ignore private dirs like this one (ones starting with .) by default.
CC @mariosasko

Maybe we can add a ignore_hidden_files flag to the builder configs of our packaged loaders (to be consistent across all of them), wdyt @lhoestq @albertvillanova?

I think they should always ignore them actually ! Not sure if adding a flag would be helpful

@lhoestq But what if the user explicitly requests those files via regex?

Usually hidden files are meant to be ignored. If they are data files, they must be placed outside a hidden directory in the first place right ? I think it's more sensible to explain this than adding a flag.

glob.glob ignores hidden files (files starting with ".") by default unless they are explicitly requested, but fsspec's glob doesn't follow this behavior, which is probably a bug, so maybe we can raise an issue or open a PR in their repo?

After globbing using fsspec, we already ignore files that start with a . in _resolve_single_pattern_locally and _resolve_single_pattern_in_dataset_repository, I guess we can just account for parent directories as well ?
We could open an issue on fsspec but I think they won't change this since it's an important breaking change for them.


fix: better prefixing

closes #3

Pull Request Test Coverage Report for Build<PHONE_NUMBER>8
Details

12 of 12 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.7%) to 92.029%

Totals

Change from base Build<PHONE_NUMBER>8:
-0.7%

Covered Lines:
105

Relevant Lines:
111

💛  - Coveralls

Oops the execSync test messes with coverage 😬


Scroll click (middle mouse click) isn't working correctly both on the website & extension

I make heavy use of scroll click so if it doesn't work correctly anywhere that immediately becomes a huge downer! please fix it!

Hi @Droyk,
What behaviour are expecting when clicking on the middle the scroll button?

apologies forgot to add what was wrong with it! Hmm, this is strange seems like the issue got fixed. the issue was scroll click was interacting with clickable links & other interactive elements in the scroll process. which was immediately stopping the scrolling process. and I had to again click on the scroll click to enable the scroll again though this wasn't happening while using the scroll wheel! but now I can't reproduce it seems like it was from my end.
but since I've already made the issue I have a request wouldn't it be better to scroll the inactive columns with scroll click just like users do with the scroll wheel
currently, scrolling while the scroll click is active doesn't work for the inactive columns only the active one can scroll up & down https://gfycat.com/ifr/madniceduckbillplatypus?controls=0&hd=1
& scroll click also doesn't work if the cursor isn't inside the columns it won't even activate! I would like to see it active so whenever I hit scroll click no matter where the cursor is on the page except clickable links it activates the scrolling function and then starts scrolling whenever the cursor gets over a column.

I've found a way to do it, it should be available in the coming version.
Thank you guys