High values of mouse polling rate cause severe stuttering when moving the camera

When the mouse works with a polling rate of more than 125hz, games show severe stuttering and high frametimes, ONLY when the player move the camera. Everything else seem to run smooth.
Affect the next games

Sekiro: Polling rate 1000, or 500 cause frametimes spikes and stuttering when the player move the camera. Polling rate 125 fixes the issue.
Quake Champions:  Polling rate 1000 cause frametimes spikes and stuttering when the camera move quickly. Polling rate 125 fixes the issue.
The witcher 3: Polling rate 1000 cause frametimes spikes when the camera move too fast, but it doesn't produce stuttering. Polling rate 125 fixes the issue.

It doesn't affect the next games

Nier: Automata: Probably because the camera move extremely slow.

Other people reporting this bug:
Ultimate Doom FPS drops with higher mouse polling rate.
High polling mouse causes stutter in editor.
How to reproduce
In the game 'Sekiro', press the physical DPI button of your mouse, and move the camera.
System information

Distro: Arch antergos XFCE
Kernel: Linux 5.0.5.arch1-1
GPU: Ryzen 1700
Driver: 418.49.4
Wine version: 4.5-tkg
DXVK version: Tested with master and 1.0.2, but probably affects all the other versions.
Mouse: BenQ Zowie EC2-A

This is not necessarily a DXVK issue. Did you try on Windows?

I have a kernel issue, please fix it in your graphics library!

Not necessarily, but I want to hear doitsujin opinion about the issue. Worst case scenario, we'll find out the right place to report it.

Relevant for testers:

I've been able to reproduce it in Doom 2016, which doesn't use DXVK. So most likely is a Wine issue. Any recommendation to make this reach the right eyes?

So most likely is a Wine issue.

Testing a few (true) native games would be a good idea here.

I don't really see the point in reporting this here at all, especially if Doom is affected. It's not like there's if (mousePollingRate > 125) makeEverythingSlow in my code.
Please report this on the proton bug tracker.

Issue moved to Proton:
https://github.com/ValveSoftware/Proton/issues/2455
And Wine:
https://forum.winehq.org/viewtopic.php?f=8&t=32235


CmdRes.Unmarshal silent fail for non-pointer fix

If the reference was passed to CmdRes.Unmarshal instead of a pointer,
json.Unmarshal received a pointer to the interface argument anyway and
failed silently without any marshalling.
This bug did not manifest in any existing tests yet.

This change is

test-me-please


feat: added privcy policy pages

Added Privacy Policy page and banner.
Screenshots
1

2

[ ] The privacy policy could be under the "About Us" section, similar to the disclaimer section.
[ ] Clicking the "Accept Button" does not redirect to the home page


Exposed ports of microservices need to be disabled in docker compose file

Currently microservices in EdgeX expose ports ( e.g, 48080 etc) without protection. With proxy in place for Fuji release, these ports need to be disabled and all the access to microservices need to go through proxy.

Attached the response from Cloud to show that the QA group already implemented methods to access EdgeX resources through KONG proxy. As the discussion thread mentioned above, we prefer to cover this in the deployment guide other than removing the port exposed through docker-compose file.
From: Cloud Tsai<EMAIL_ADDRESS>Sent: Wednesday, February 26, 2020 9:01 AM
To: Jim White
Cc: Zeng, Tingyu; Robin Chatterjee; Andrew Foster; Cherry Wang
Subject: Re: issue #190 of developer-scripts - blocking exposed microservice ports
[EXTERNAL EMAIL]
Hi all,
Currently, the blackbox test with security enabled is testing services via Kong.  That's the improvement mentioned by Tingyu.
However, I don't think we need to test access is actually denied when trying to access the APIs outside of security, because it is basically testing the firewall not provided by us.  Unless we include a firewall into our scope.
The easiest way to disable every access from outside to the individual micro service is to remove the port mapping in the docker-compose file.  Isolating the services in the docker network can achieve this objective.  Also, I still don't think we should make sure the access of each service from outside is denied, because it is testing the docker network.
For example, remove the port section in the service and only keep the port mapping for Kong.
command:
image:
nexus3.edgexfoundry.org:10004/docker-core-command-go:master
ports:

"48082:48082"

container_name:
edgex-core-command
hostname:
edgex-core-command
networks:

edgex-network

volumes:

/tmp/edgex/secrets/ca:/tmp/edgex/secrets/ca:ro,z

/tmp/edgex/secrets/edgex-core-command:/tmp/edgex/secrets/edgex-core-command:ro,z

depends_on:

As discussed during the security WG meeting on March 25th, this will be addressed as part of #236. Leaving this open for reference.

Close this as the related issue of #236 has been closed.

Reopening; pr never merged due to upstream drift.


VOB OCR instability

The popeye_basic_vob test will pass for me locally, but fail in GitHub Actions:
---- test::popeye_basic_vob stdout ----
thread 'test::popeye_basic_vob' panicked at 'assertion failed: `(left == right)`
  left: `Some(("Title T01-2.mkv", "whos the most phenom inal extra ordinary how do you like that stooges on one of my travels i ran into this now there was a thrill id be sorry to miss he spread out his wings and the sunlight grew dim"))`,
 right: `Some(("Title T01-2.mkv", "whos the most phenom inal extra ordinary how do you like that stooges on one of my travels i ran into this now there was a thrill id be sorry to miss spread out his wings and the sunlight grew dim"))`', src\main.rs:592:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The difference is a single word ("he").

If the instability can't be resolved, we may have to use a Levenshtein distance in the test like the main application does.


feat: further refactoring RL models

As discussed here, isolate environment and task by a general gymnasium env + a task that defines block structure.

TODO:

add randomwalk to the task
add reward baiting to the env
put to another isolated repo


Additional conversions

While implementing mint support for gltf crate I faced the following problem:
     /// 4x4 column-major transformation matrix.
-    pub fn matrix(&self) -> [f32; 16] {
-        self.json.matrix
+    pub fn matrix(&self) -> ColumnMatrix4<f32> {
+        let m = self.json.matrix;
+        let vec = [
+            [m[0],  m[1],  m[2],  m[3]],
+            [m[4],  m[5],  m[6],  m[7]],
+            [m[8],  m[9],  m[10], m[11]],
+            [m[12], m[13], m[14], m[15]],
+        ];
+        vec.into()
     }

It's too boilerplate.
I suggest new conversions for mint matrices:

[T; 16] -> ColumnMatrix4<T>, RowMatrix4<T>
[T; 9] -> ColumnMatrix3<T>, RowMatrix3<T>
...

What's your opinion?

If needed badly, we can introduce this without any risk of breaking, if behind a new feature gate.

Is there any reason not to have this in 0.5?

Actually, given our Row/Column distinction right in the type name, there is no ambiguity on how conversion to [T; 9] or [T; 16] is going to be done. That is for, I think having those extra implementations would be useful 👍
Care to make a PR?

Having a go at this in #23.

Fixed in #23


questions about cryopipeline

Hello, I am interested in your research of cryomodel, and would like to learn about your code. But when running the cryopipeline, I encountered some issues, could you please give me any suggestions about them？

The power results reported after running cryopipeline/logic_model.py show the global operating voltage = 1.1V, while the nom_voltage in freepdk_45nm/stdview/stdcell.lib is 1.1V as well. But the voltage of reference_pgen in logic_model.py is 1.25V. In logic_model.py-line 225-227, it seems that the data of reference_pgen and target_pgen is used to offset the differences in voltage and other parameters between the stdcells of lib and target design. Could you please give me any advice on how to comprehend the data processing of design power in logic_model.py-line 225-227 and the different voltages?

In logic_model.py-line 218-219, the static power reduction is calculated by dividing the sum of Isub and Igate. May I ask why the voltage isn’t taken into consideration here, for example, (V*(Isub+Igate))target/(V*(Isub+Igate))ref?

Hello, shlin21.
Thanks for your interest in our model.
First of all, we use 1.25V because it is the peak operating voltage of 45nm Intel CPU under turbo boost mode.
As we compare 45nm Intel CPU with our architecture in our paper, we set the voltage to 1.25V.
Please set the baseline voltage at 185 to the voltage that you want (e.g., 1.1V following freePDK, 1.0V following PTM).
Note that Vth0 value (0.46893V) is based on PTM which follows the ITRS roadmap.
Meanwhile, for the static power reduction, your suggestion is really valid.
I think we omitted them while cleaning up the code before the tool release.
I will fix the line 218-219 as you mentioned.

Hello, thank you for your reply so much, it really helps me.
I’m glad my question is valid.
But I encountered another issue. In logic_model.py-line 225-227, I find the code uses transistor data to calculate target power under 77K only. Since the voltage in 45nm FreePDK is 1.1V, I wonder whether the reference power under 300K should also be calculated like 77K(line 225-227), if I use different reference voltage (e.g.,1.3V, 1.5V) from the FreePDK, and, whether the variable power_total[0] is the power of target design working in the condition of 300K, using transistor of pgen_300K(line 185)?
Could you please give me any suggestions about them?
Thank you!

Hello, thank you for your reply so much, it really helps me.
I’m glad my question is valid.
But I encountered another issue. In logic_model.py-line 225-227, I find the code uses transistor data to calculate target power under 77K only. Since the voltage in 45nm FreePDK is 1.1V, I wonder whether the reference power under 300K should also be calculated like 77K(line 225-227), if I use different reference voltage (e.g.,1.3V, 1.5V) from the FreePDK, and, whether the variable power_total[0] is the power of target design working in the condition of 300K, using transistor of pgen_300K(line 185)?
Could you please give me any suggestions about them?
Thank you!

We appreciate your kindness for sharing the issue.
I added the new code to compensate for the different operating voltage at 300K.
I think it does not much affect the relative numbers (i.e., speedup, power reduction) used in our paper, but it can be a problem if someones plan to use the absolute numbers (e.g., critical-path delay, power) for their research.
Please check our update and contact us if any problem exists.
Thank you again :)

Thanks for your response, it's my pleasure to share the issue.
I rerun the cryopipeline with updated logic_model.py, and everything looks fine.
Thank you very much!


Some emojis don't show properly on Ubuntu 19.04

As you can see from the screenshot, the keycap emojis from 0 to 9 have a weird rendering on Ubuntu 19.04, whereas on Ubuntu 18.04 everything was fine.
I did a fresh install (and update) on two different laptops and the issue seems to affect every component - Nautilus, Firefox, etc.

Another font on the new system might be providing these characters and it got precedence over Noto Color Emoji.

Thank you for the answer.
I found out that some additional software renders correctly the emojis, probably referencing the correct font...I'll report the issue on the Ubuntu community

@moropallo This is partially an issue with Firefox, I discovered a bug in this issue where the linked comment links to the Firefox bug tracker issue I created for it. It's been fixed in nightly afaik, so should be available in the near future.
That bug is specifically about the white-space/layout issue. It's probably why your numbers are offset horizontally for the keycaps. In Chrome they looked better aligned.
The other issue is font support/fallbacks from the looks of it. From what I can see in dev tools, the numbers are being rendered with Bitstream Vera, while the keycaps are Noto Sans Symbols. Assuming the Symbols font doesn't have a composited glyph/ligature for representing the number and keycap together, you would want to get both using the same font or at least compatible metrics? As mentioned, the alignment issue is probably related to the firefox bug.
Your emoji in the screenshot look like they're Twemoji, if you force the CSS font to that font-family("Twemoji Mozilla"), it should replace them with emoji. I assume they're not replaced due to other fallback fonts taking priority? (should be possible to adjust with fontconfig) Likewise, Noto Color Emoji has it's own equivalent emoji for those.
Chrome should be working fine though?

You'll have to wait until glibc is updated to version 2.30 on 19.04's repository.

@Some1NamedNate are you referring to the Unicode 12.1 support? That's unrelated to this, as keycap emoji are from Unicode 3.0 back in 1999. Unicode 12 is from the update this year with new emoji.

@polarathene thank you for your insight! Yes, Chrome works properly. Nevertheless I have to admit I'm more interested in emoji usage in Nautilus than Firefox - that's why I posted a bug report on some Ubuntu forums (with no responses till today)

@moropallo How are you wanting to use emoji in Nautilus? You shouldn't want to use emoji in file/directory names.
Support for that would likely be fontconfig related, if you're seeing black/white for older emoticons and symbols pre-emoji, that'd be because of fonts like DejaVu Sans having a higher priority.

@polarathene I regularly use emojis in filenames almost from 2016, and had no problems until the 19.04 Ubuntu LTS came out. As I said in the opening post, the defective visualization of Noto emojis affects pretty every application.

This is kind of the reason why you shouldn't use emoji for such :P Or if you had to use a terminal/TTY you might have a tough time typing the emoji file names without a nice way to input them? I hear that on Windows or macOS it can be quite problematic and at least in the past, emoji in file names prevented the files from being deleted off the filesystem requiring linux to fix.
Anyway, could you share a screenshot perhaps? And if you know the emoji should look like, share some of those here? I've been working on a custom fontconfig to get emoji working everywhere on my system too, and the stuff out of the browser had been the most stubborn.

Try this.

can confirm similar effects in ubuntu 18.04 http://i.imgur.com/KoKRob0.png
by the way, nothing is wrong on android 11 powered phone

can confirm similar effects in ubuntu 18.04 http://i.imgur.com/KoKRob0.png
by the way, nothing is wrong on android 11 powered phone

I have coded a fix: emoji.conf

I have coded a fix: emoji.conf


Fix/issue 9

Add documentation for keccakf.rs and aux_functions.rs.
Specifically:
keccakf.rs:

rotate_left64() function.
keccakf_1600 function.

aux_functions.rs:

nist_800_185 module
byte_utils module

thanks for the effort on this. The original objective of #9 is to find anything that is undocumented and provide documentation if it makes sense to do so. Much of this PR is modifying existing docs which isnt really needed at this time


questions about Grounding evaluation

In your grounding task, you used 'dets.json' to evaluate your results, how do you get the 'dets.json' file? which object detector did you use?

Hi, following previous work such as UNITER, we use the object proposal from Mask-RCNN, provided by "Mattnet: Modular attention network for referring expression comprehension."

Thank you, your work is amazing, It inspires me a lot, I really appreciate it.


Fix error messages when language changes

Details

Trigger the validation function in Form.js when the language changes in order to also update the error messages.
Fixed Issues

$ https://github.com/Expensify/App/issues/13701
Tests

Go to Workspace - Connect Bank Account - Connect Manually
Without entering any values click Save & continue
Change language in another session
Verify the error messages are updated to the corresponding language
Continue flow till Step 3 of 5 (Personal Information)
Press Save & Continue button to display the error messages in the input fields
Change language in another session
Verify error messages on Personal Information page are also updated to the corresponding language

[x] Verify that no errors appear in the JS console

Offline tests

N/A
QA Steps

Same steps from Test section.

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

https://user-images.githubusercontent.com/6829422/209248940-7ebee649-5e02-4c66-b908-f05dabdd87a8.mov

Mobile Web - Chrome

https://user-images.githubusercontent.com/6829422/209415482-fb137128-3ea7-46c0-851f-af4a1086d4da.mov

Mobile Web - Safari

https://user-images.githubusercontent.com/6829422/209709431-a23af4e7-3e5e-4c3f-adcd-f8edc5093c4b.mov

Desktop

https://user-images.githubusercontent.com/6829422/209720822-598518d5-d584-4dfd-8f47-9ac4c4d3cf4c.mov

iOS

https://user-images.githubusercontent.com/6829422/209721102-098de2f8-feb0-44aa-b665-cab8328aea68.mov

Android

https://user-images.githubusercontent.com/6829422/209710604-ff75d9b8-e474-4204-8b5e-92437876f91f.mov

Hey @rushatgabhane, do you think you will be able to get to this by tomorrow?

Hey @marcochavezf, I'm taking a break from expensify this semester to finish up classes. Do you think you could reassign this to another engineer?

Sure, I'm going to re-assign PullerBear to get another internal reviewer 👍🏽

@marcochavezf I think it didn't change the internal reviewer but changed the C+ reviewer here. I also see @rushatgabhane is already assigned.
I am available in case this requires a C+ review.

Oh interesting, since this PR hasn't been reviewed yet I'm going to remove all assignees and assign PullerBear again.

@marcochavezf It assigned a new C+ here. I think @sobitneupane can take it forward now.

Screenshots/Videos

Web
https://user-images.githubusercontent.com/25876548/210704564-495f6ef9-5e8e-4fdd-b3fb-b4f10e3988a8.mov

Mobile Web - Chrome

Mobile Web - Safari

Desktop
https://user-images.githubusercontent.com/25876548/210708617-b168b610-200b-45c3-88ee-732b3a3b39be.mov

iOS
https://user-images.githubusercontent.com/25876548/210709893-c85ed748-015b-4c39-9d1c-5ffe4ecaa3f7.mov

Android
https://user-images.githubusercontent.com/25876548/210710228-3d3cbccf-24b8-4039-a995-3738af5ecd2f.mov

Would it be worth adding offline tests?

Ah I didn't add them because the problem was more related to the component not updating/validating than the offline UI feedback, which doesn't apply to this case (Pattern A).

@marcochavezf A similar issue is reported here. AddSecondaryLoginPage does not use Form Component. So, won't be fixed by this PR. Do we want to handle the issue in this PR?

@sobitneupane should we get this merged?

@cead22 I agree with @rushatgabhane. The solution of above mentioned issue will require form refactoring. And a github issue is already created for that. So, I think we are good to merge this if @marcochavezf agrees.

Ah interesting, I think if we use the change made here to Form and if we use Form in AddSecondaryLoginPage, the error message on Add Phone number will be fixed. I'm going to merge it.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by @marcochavezf in version: 1.2.53-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by @Julesssss in version: 1.2.53-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Fixed the bump of release in deploy artifacts

Since #173, we have a few more files to bump the version during the releases.
Since #217, we don't have the operator-openshift.yaml file anymore, causing the release.sh to fail.
This PR fixes both scenarios and bumps the necessary versions to the latest ones.
Signed-off-by: Juraci Paixão Kröhling<EMAIL_ADDRESS>

This change is

@awgreene -- I realize that I should use operator-sdk olm-catalog gen-csv --csv-version instead of messing with the files directly, but the Operator SDK version we are using (0.4.0) doesn't support that command yet. Right now, it's too risky for us to use any other version of the SDK.


update coveralls badge

None
Description
Fix coveralls badge

Coverage remained the same at 91.732% when pulling baea5d246445555abf4f31947813793556d8ba7f on coverall-badge into 04415d980d5d1546b11c029089ff089861d1d526 on master.


start/stop command limits

A 127-character limit for start/stop commands was added in https://github.com/bbg-cookbooks/monit-ng/commit/22c6eef0b055f163267055e228e81eb282df717a. For some use cases I'm having to wrap long commands in a shell script because. Is the limit still necessary?
I'm running Monit 5.6 and have not seen any issues with commands being truncated.

well, it's an interesting question that seems to come up from time to time. i'm not 100% sure whether it still applies or not, but the most recent reference to the limit is from just last year.
my gut feeling on it so far is that:

since the cookbook supports monit versions known to suffer the problem
since commands that long are fairly rare and usually belong in a wrapper script that gets called from monit instead
since this has the potential to bite people, and the limit's not well advertised in the docs

then, it's preferable to keep it in the cookbook. That said, i'm open to changing my mind if circumstances change. do you have an example of a type of command that you're running into the limit for?

Trying to execute a jar with a long classpath (easily longer than 127 chars).

since commands that long are fairly rare and usually belong in a wrapper script that gets called from monit instead

I think this is reasonable. It does involves additional steps in Chef like creating a dir for wrapper scripts and creating a file with the long command. I just wish monit had better documentation around the 127-char limit. Thanks!

My pleasure, thanks for the feedback!


Program full sectors, longer read/write timeouts

As diagnosed in https://github.com/timvideos/litex-buildenv/issues/137, there seems to be an issue with programming the TinyFPGA BX SPI flash with less than 256 octets (minor sector size).  In theory the SPI flash supports this (see datasheet pages 14-16), but in practice it always seems to fail.  It's unclear whether this is caused by a state machine issue in the TinyFPGA-Bootloader verilog, or some bug or unclear documentation in the SPI flash.  But since we've already erased the entire 256 octets (and on any subsequent write, we'll also erase the entire 256 octets) the simplest solution is to pad all minor sector writes out to a full 256 octets (with 0xff, which is what is read back after erase, as that hopefully saves one cycle of wear on the "unused" flash cells).
Also double the read timeouts (was 1.0 seconds, now 2.0 seconds), because the only way that pySerial will return a short read is if the readTimeout is hit, and there are multiple reports of users getting short reads and verify errors (and thus programming failures; some bricking their boards).  And substantially increase the writeTimeout (was 1.0 seconds, now 5.0 seconds) because tinyprog gives up immediately if there is a writeTimeout, which could lead to a partially programmed board (possibly one that is bricked), so it seems worth being more patient.  (IIRC there's another pull request around that retries the sector write if it times out/fails, but ISTR it's buried in a gigantic pull request that adds a whole other board, etc, and is against 6-12 month old code, so it is not easily merged directly.)
Seems to work for me, both images that are a multiple of 256 octets, and images that are not a multiple of 256 octets.  (See https://github.com/timvideos/litex-buildenv/issues/137 for the testing to demonstrate there was a problem; I re-ran the tests in https://github.com/timvideos/litex-buildenv/issues/137#issuecomment-493830552 to check this version was working.)
Programing output:
(LX P=tinyfpga_bx.minimal F=none R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$ make clean; export FIRMWARE=none; make image-flash
[...]
tinyprog --program-image build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+none.bin

    TinyProg CLI
    ------------
    Using device id 1d50:6130
    Only one board with active bootloader, using it.

    Programming /dev/ttyACM0 with build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+none.bin
    Programming at addr 028000
    Waking up SPI flash
    180192 bytes to program
    Programming and Verifying: 180kB [00:04, 39.6kB/s]
    Success!

(LX P=tinyfpga_bx.minimal F=stub R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$ make clean; export FIRMWARE=stub; make image-flash
[...]
tinyprog --program-image build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+stub.bin

    TinyProg CLI
    ------------
    Using device id 1d50:6130
    Only one board with active bootloader, using it.

    Programming /dev/ttyACM0 with build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+stub.bin
    Programming at addr 028000
    Waking up SPI flash
    203904 bytes to program
    Programming and Verifying: 205kB [00:05, 39.7kB/s]
    Success!

(LX P=tinyfpga_bx.minimal F=stub R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$ make clean; export FIRMWARE=micropython; make gateware; scripts/build-micropython.sh; make image-flash
[...]
tinyprog --program-image build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+micropython.bin

    TinyProg CLI
    ------------
    Using device id 1d50:6130
    Only one board with active bootloader, using it.

    Programming /dev/ttyACM0 with build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+micropython.bin
    Programming at addr 028000
    Waking up SPI flash
    442884 bytes to program
    Programming and Verifying: 446kB [00:11, 40.3kB/s]
    Success!

(LX P=tinyfpga_bx.minimal F=micropython R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$

And the serial output (note that if you program FIRMWARE=none it just writes to the gateware section of the flash, which pretty much guarantees it'll boot whatever firmware was written previously, thus in my case it booted MicroPython; see further down for writing a different firmware and then re-writing MicroPython back again):
        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
 SoC BIOS / CPU: LM32 /  16MHz
(c) Copyright 2012-2018 Enjoy-Digital
(c) Copyright 2007-2018 M-Labs Limited
Built Jun  8 2019 12:59:13

BIOS CRC passed (494556cb)
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
Booting from flash...
Executing booted program at 0x20058008
MicroPython v1.9.4-534-gd2bd404 on 2019-05-20; litex with lm32
>>>
        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
 SoC BIOS / CPU: LM32 /  16MHz
(c) Copyright 2012-2018 Enjoy-Digital
(c) Copyright 2007-2018 M-Labs Limited
Built Jun  8 2019 13:03:17

BIOS CRC passed (f68ee6ca)
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
Booting from flash...
Executing booted program at 0x20058008
Stub firmware booting...

LiteX-Buildenv Stub Firmware, built Jun  8 2019 13:03:18

Type "help" for help

STUB>
        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
 SoC BIOS / CPU: LM32 /  16MHz
(c) Copyright 2012-2018 Enjoy-Digital
(c) Copyright 2007-2018 M-Labs Limited
Built Jun  8 2019 13:09:46

BIOS CRC passed (71f12f87)
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
Booting from flash...
Executing booted program at 0x20058008
MicroPython v1.9.4-534-gd2bd404 on 2019-06-08; litex with lm32
>>>

@mithro According to the flash chip documentation, with program sectors, it'll "wrap around" within the 256 octet (minor) sector (start of page 15 of the datasheet), ie that method only adjusts certain low address values.
I think the way that program_sectors() is used (called by program_bitstream(), called by various places in perform_bootloader_update() and main()) probably means that it's always called with a starting address on a 256 octet boundary.
But you're right, tinyprog does accept arbitrary addresses, so I suppose it's possible that the first write might be a short one too.  It looks like self.erase tries to handle this case (by read/erase/rewrite).  But the rest of the writing is then done in 256-byte chunks, ignoring any "sector boundaries", so I think it would then hit the SPI flash "wrap around" behaviour with the SPI flash write method that program_sectors uses, and the result would be garbage in the SPI flash.  So AFAICT it'll only actually work if the start address is on a 256-byte sector boundary.
We should probably (a) check that the start address is actually a 256-byte sector boundary (and fail out at the beginning if it is not), and (b) adjust the padding calculation so it can't wrap around in that case.
Ewen

We should probably (a) check that the start address is actually a 256-byte sector boundary (and fail out at the beginning if it is not), and (b) adjust the padding calculation so it can't wrap around in that case.

I've added both of those.  I believe the only regularly used programming locations are 0x28000 (start of user image) and 0x00000 (start of flash, for the bootloader), but it'll now throw an exception if someone tries to program somewhere else that is legal but not a multiple of 256.  I also changed the code to calculate the offset from the start of the 256 octet sector boundary in the case where it's going to try to do a short write, and avoid wrapping in that case.  (But in practice I think the only way you could trigger that behaviour and get sane results would be to be writing less than 256 octets total at an offset that is not a multiple of 256; which is now no longer supported.)
Ewen
PS: The programming offsets are shown in hex, but I only realised after testing that --addr would accept hex values (0xNNNN) due to the way parse_int() worked; I'd assumed it just did int()...
PPS: The "you can't do that" stack trace is ugly, but in theory should "never happen", so I've left it as just a safety check.
(LX P=tinyfpga_bx.minimal F=micropython R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$ tinyprog -m
[
  {
    "boardmeta": {
      "name": "TinyFPGA BX",
      "fpga": "ice40lp8k-cm81",
      "hver": "1.0.0",
      "uuid": "1012cf3a-e168-4cf0-b517-d0e002789cfc"
    },
    "bootmeta": {
      "bootloader": "TinyFPGA USB Bootloader",
      "bver": "1.0.1",
      "update": "https://tinyfpga.com/update/tinyfpga-bx",
      "addrmap": {
        "bootloader": "0x000a0-0x28000",
        "userimage": "0x28000-0x50000",
        "userdata": "0x50000-0x100000"
      }
    },
    "port": "/dev/ttyACM0"
  }
]
(LX P=tinyfpga_bx.minimal F=micropython R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$ perl -le 'print 0x28000;'
163840
(LX P=tinyfpga_bx.minimal F=micropython R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$ tinyprog --program-image build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+micropython.bin --addr 163845

    TinyProg CLI
    ------------
    Using device id 1d50:6130
    Only one board with active bootloader, using it.

    Programming /dev/ttyACM0 with build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+micropython.bin
    Programming at addr 028005
    Waking up SPI flash
    442884 bytes to program
Traceback (most recent call last):
  File "/src/fpga/litex-buildenv/build/conda/bin/tinyprog", line 11, in <module>
    load_entry_point('tinyprog', 'console_scripts', 'tinyprog')()
  File "/src/fpga/dev-litex-tools/TinyFPGA-Bootloader/programmer/tinyprog/__main__.py", line 463, in main
    if not fpga.program_bitstream(addr, bitstream):
  File "/src/fpga/dev-litex-tools/TinyFPGA-Bootloader/programmer/tinyprog/__init__.py", line 612, in program_bitstream
    return self.program_sectors(addr, bitstream)
  File "/src/fpga/dev-litex-tools/TinyFPGA-Bootloader/programmer/tinyprog/__init__.py", line 510, in program_sectors
    format(minor_sector_size, addr))
ValueError: Address must be a multiple of 256, not 163845
(LX P=tinyfpga_bx.minimal F=micropython R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$ tinyprog --program-image build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+micropython.bin --addr 163840

    TinyProg CLI
    ------------
    Using device id 1d50:6130
    Only one board with active bootloader, using it.

    Programming /dev/ttyACM0 with build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+micropython.bin
    Programming at addr 028000
    Waking up SPI flash
    442884 bytes to program
    Programming and Verifying: 446kB [00:11, 40.1kB/s]
    Success!

(LX P=tinyfpga_bx.minimal F=micropython R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$

Ah, it looks like self.write actually also tries to handle this 256 octet sector boundary (and do a short write at the start).  So in theory writing at arbitrary addresses might actually work, it's just very inefficient (as program_sectors will self.erase in 4k chunks, which will do a bunch of rewriting at the margins to impedance match the SPI flash erase chunks to the relevant offsets, only for them to be erased/rewritten on the next sector written....).  I'm dubious whether that's ever actually been used in anger though.
I'll repush a version that just adjusts the padding to reach the end of the 256 octet offset but not overrun it, and only apply if the original start address was a multiple of the minor sector size.  Ie, only apply in the case where we were having problems.
Ewen

Now the special padding work around will only happen if we're aligned, which should be safe.  But we don't try to force programming at aligned addresses.  (I'm really not convinced that programming starting at unaligned addresses via program_sectors works, but it should work as well as it did previously.)
Still seems to sole the FIRWWARE=none problem.
While testing this I noticed that our MicroPython image is actually not a multiple of 256 octets (it's 4 bytes longer), nor is our stub image, yet both of always worked (give or take timeouts):
FIRMWARE=none: 180192 = 703 * 256 + 224
FIRMWARE=micropython: 442884 = 1730 * 256 + 4
FIRMWARE=stub: 203904 = 796 * 256 + 128
So there's apparently something Yet More Magical (tm) about the 224 byte value of FIRMWARE=none somewhere inside tinyprog, the TinyFPGA-Bootloader verilog, or the SPI flash, which causes it not to work when the others (4 bytes, 128 bytes) do work.
But:
(a) I've run out of patience to try to figure out precisely why the 224 byte partial sector is failing;
(b) rounding up aligned writes to a full SPI flash sector works; and
(c) as far as I'm aware, writing 0xff to a SPI flash is a no-op, because my understanding is that one gets a "1" in a flash cell by erasing a block of cells and not writing a "0" to it, and one gets a "0" in a flash cell by explicitly writing a zero to it (ie, draining that cell), and the extra 0xffs write no "0"s ; and
(d) in practice I think all the uses of TinyProg are writing at aligned addresses anyway, and that's the case of all the "address range" boundaries defined (eg, in the metadata); and
(e) AFAICT the TinyProg Bootloader doesn't have support for choosing among multiple images, so the only case where there is potentially trailing data that matters, after what is being written, is someone attempting to do a short write to patch the middle of their image, which honestly feels really really unlikely in practice.
So "this seems to work in practice", and I think at this point it shouldn't break anything in practice.
But if someone else wants to figure out why writing a 224 byte partial sector is the thing that fails (with weird data wrap around being observed -- see https://github.com/timvideos/litex-buildenv/issues/137) then feel free. to do that instead.
Ewen
PS: Either way I think we should increase the read/write timeouts, which means if we're not going to merge this pull request we should probably cherry pick that change and the pip-wheel-metadata changes out of this branch and apply them separately.

Joy.  I was doing a final reprogram of MicroPython into the board before I put it away, and finally triggered a timeout.  It seems there's something where even 5 seconds isn't enough and/or the bootloader/SPI state machine gets itself wedged.
tinyprog --program-image build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+micropython.bin

    TinyProg CLI
    ------------
    Using device id 1d50:6130
    Only one board with active bootloader, using it.

    Programming /dev/ttyACM0 with build/tinyfpga_bx_base_lm32.minimal//image-gateware+bios+micropython.bin
    Programming at addr 028000
    Waking up SPI flash
    442884 bytes to program
    Programming and Verifying:  94%|████████▍| 418k/443k [00:15<00:00, 26.8kB/s]
Failed to write to serial port:
Write timeout
targets/tinyfpga_bx/Makefile.mk:31: recipe for target 'image-flash-tinyfpga_bx' failed
make: *** [image-flash-tinyfpga_bx] Error 1
(LX P=tinyfpga_bx.minimal F=micropython R=default-target-tinyfpga-bx) ewen@parthenon:/src/fpga/litex-buildenv$

It worked fine when I did another "make image-flash" to program it a second time.  So there might well be value in extracting out the "try programming each sector multiple times" logic from https://github.com/tinyfpga/TinyFPGA-Bootloader/pull/16 as well as these timeouts changes.  Especially for BootLoader reprogramming (which unfortunately get recommended to basically every new user, as a bunch of TinyFPGA BX boards shipped with an incorrectly programmed bootloader :-( )
Ewen

@ewenmcneill Maybe we should have very large timeouts on programming the bootloader?

Maybe we should have very large timeouts on programming the bootloader?

@mithro: That's a timeout on writing a single 256 octet minor sector!
5 seconds is a long timeout for writing 256 octets (bytes).  Honestly even 1 second is a long timeout:  that's roughly one octet ever 4ms.  Even over 9600 bps serial that ought to be enough time, and it looks like the serial runs around 38400 bps in practice.  5 seconds/256 octets allows around 19ms per octet written.   So I don't think it's the serial transfer itself that's the limiting factor.
5 seconds to write 256 octets is also a very long time in the context of writing to SPI flash.  The write speed there is typically in the many hundreds of Kbps, if not Mbps (give or take how quickly they're clocked into the SPI flash).
So I expect we could set, eg, a 30 second timeout and still hit it in the same conditions as the 5 second / 1 second write timeout.
What still hitting a write timeout sometimes, even at 5 seconds, says to me is that there's a race condition in the USB / serial / TinyFPGA-Bootloader / SPI Flash interaction (and they're all basically chained together :-( ) which sometimes causes it to stop clocking through additional serial bytes.  Causing the write to backlog indefinitely, and thus hit the really long timeout.  No idea if that's on the USB/serial to BootLoader side or the BootLoader to SPI Flash side, or somewhere in between.
Short of debugging that TinyFPGA-Bootloader to eliminate all possible race conditions (formal methods?) the next best option seems to be to retry to the sector write instead of immediately bailing out with "oh, it didn't work, hope that didn't brick your board", which is the traditional/existing tinyprog handling of errors (ie, see https://github.com/tinyfpga/TinyFPGA-Bootloader/pull/16, who report that retrying writes made it reliable for them).
Ewen

I think retry is a good idea.
My guess is the timeout could occur because of issue in things like USB forwarding when using a VM.

FTR, I'm not using a VM.  Ubuntu 16.04 LTS on Dell XPS9360,direct on the hardware.  And still got the write timeout above (and a few others, with the same combination) in the past.  Same TinyFPGA BX, USB cable, USB port, laptop, etc, all work properly the other 95%+ of the time.  Hence thinking there's an edge condition in the TinyFPGA Bootloader verilog (which AFAICT just got developed until it "almost always" worked).
The retry logic is in https://github.com/tinyfpga/TinyFPGA-Bootloader/pull/16.  Feel free to pull out the relevant bits and rework that and merge it too.  (I've have many, many, other things on my todo list that need doing, so if it waits for me to pull out/rework bits of https://github.com/tinyfpga/TinyFPGA-Bootloader/pull/16 it may well be Some Time (tm).)
Ewen

FTR, this commit from Trammel Hudson (in a tree mentioned on Twitter) might be worth considering as well: https://github.com/osresearch/TinyFPGA-Bootloader/commit/43672180fbcd60b4e30bbce99744e37b011fe3ce.  It skips erase/write on identical 4k sectors, and skips write/read back on 0xff filled 256-octet mini sectors (since 0xff is the erased value).  Having merged this pull request it'll need a tiny bit of tweaking to insert in the right place, but the individual code blocks seem sensible to me.
Ewen

@ewenmcneill - I think that commit is merged now?

@ewenmcneill - I think that commit is merged now?

It's not merged yet.  But it is in https://github.com/tinyfpga/TinyFPGA-Bootloader/pull/54, so if that gets merged we'll pick it up.  (Otherwise I think we probably want that erase/write optimisation logic anyway.)
Ewen

@ewenmcneill - This pull request is merged now....


Add senzing/senzing-debug container

Is your feature request related to a problem? Please describe.
In the docker formation, add a senzing/senzing-debug container for doing debugging and using the Senzing tools in /opt/senzing/g2/python.

Pull requests:

https://github.com/Senzing/docker-compose-demo/pull/30


ROBOCOPY docs missing /J switch

This page for ROBOCOPY is missing documentation on the /J switch.  /J forces unbuffered copies.  Might have some other cross-impacts that should be documented.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bf2281d4-1470-0658-ec5a-687fe2cd451c
Version Independent ID: 2ce2be3e-e3a2-790c-3bff-90efe7318450
Content: robocopy
Content Source: WindowsServerDocs/administration/windows-commands/robocopy.md
Product: windows-server
Technology: windows-commands
GitHub Login: @JasonGerend
Microsoft Alias: jgerend

#assign:xelu86

Hey @ipsos-md-jimw,
Thanks again for contacting Microsoft. We value your feedback. A documentation change has been made and will be live after the next business day. We will be closing this as resolved. Thank you and have a most pleasant day. #please-close


Added LitmusChaos and MayaData in README

This adds Litmus Chaos to the list of HacktoberFest TypeScript repositories and MayaData to participating companies
Litmus is a toolset to do cloud-native chaos engineering. Litmus provides tools to orchestrate chaos on Kubernetes to help SREs find weaknesses in their deployments. SREs use Litmus to run chaos experiments initially in the staging environment and eventually in production to find bugs, vulnerabilities. Fixing the weaknesses leads to increased resilience of the system.
More Details about the Litmus could be found here on the official website
Signed-off-by: Sayan Mondal<EMAIL_ADDRESS>

Cheers


Display search results Land Library

Hi,
Can we adjust the display of the search results in the LAnd Library?
The thumbnails are placed above the resource title, which makes it confusing as to which resource the picture belongs to. Possibly to align it on the left of the text information of the same resource?
Can we have a line between one resource and the other to make the distinction between the different resources? Similar to the way news items are divided on this page: http://landportal.info/debate/news ?
Antonella mentioned that that is a CSS issue, so not for her to fix.
Thanks!!

should be fixed.
The search results is now using a 'normal' / regular rending method.
The view is now entirely different and display proper fields- cleanup is still in the process though.


LightSpeech fine-tuning

Could you tell me if it is possible to fine-tune the model? If yes, what steps do I need to take to do this?

@texpomru13  It is possible to fine-tune the model, though I do not try it. You can place the trained checkpoint (e.g., our trained checkpoint) under the corresponding path (e.g., Step 7 in README) and the program will automatically reload that model.
mkdir -p checkpoints/lightspeech
cd checkpoints/lightspeech
wget https://msramllasc.blob.core.windows.net/modelrelease/LightSpeech/model_ckpt_steps_100000.ckpt
cd ../../
export CUDA_VISIBLE_DEVICES=0,1,2,3 #may change this to the appropriate number on your machine
export PYTHONPATH=.
### continue training
### remember to edit the lightspeech.yaml file to modify the parameters ( e.g., training steps, your data path, learning rate ... ) or pass them via --hparams
python tasks/lightspeech.py \
    --config configs/tts/lj/lightspeech.yaml \
    --exp_name lightspeech \
    --reset


Introduce result-clause

Purpose
Fixes #40531

We need to also change the names in BallerinaParser.java and BallerinaErrorhandler.java eg: https://github.com/ballerina-platform/ballerina-lang/blob/e1ef9867d68a0cadf149687bd24098254d6230e6/compiler/ballerina-parser/src/main/java/io/ballerina/compiler/internal/parser/BallerinaParserErrorHandler.java#L447

We need to also change the names in BallerinaParser.java and BallerinaErrorhandler.java eg:
https://github.com/ballerina-platform/ballerina-lang/blob/e1ef9867d68a0cadf149687bd24098254d6230e6/compiler/ballerina-parser/src/main/java/io/ballerina/compiler/internal/parser/BallerinaParserErrorHandler.java#L447

Changed

Did u check the BallerinaParser.java as well for endClause occurrences?

Need to rename the following variable.
https://github.com/ballerina-platform/ballerina-lang/blob/e1ef9867d68a0cadf149687bd24098254d6230e6/misc/formatter/modules/formatter-core/src/main/java/org/ballerinalang/formatter/core/FormattingTreeModifier.java#L2831

Did u check the BallerinaParser.java as well for endClause occurrences?

Seems no occurrences there.


Add ability to handle scopes from api keys

In Barong we have dynamic scopes.
Point of scopes:
User can implement a finance bot that can monitor monthly budget.
But bot should be ably only read trades and other actions should be prohibited.
We should think of other scopes, implement it and add documentation how to use it.
Related Barong issue is https://github.com/rubykube/barong/issues/540

Just FYI .. I was able to implement scopes by utilising the declared scopes on the grape api.
It requires no change at all to barong.  However, you may be wanting to do something completely different so that's why I'm adding this simply as FYI.


Updated json-eval($.cardNo) to json-eval($.patient.cardNo)

Purpose

Updated json-eval($.cardNo) to json-eval($.patient.cardNo)

Goals

Get the card number value from the request

Approach

Updated json-eval($.cardNo) to json-eval($.patient.cardNo)

User stories

As a developer I want to be able to get the card number from the HTTP request

Release note

Brief description of the new feature or bug fix as it will appear in the release notes

Documentation

Link(s) to product documentation that addresses the changes of this PR. If no doc impact, enter “N/A” plus brief explanation of why there’s no doc impact

Training

Link to the PR for changes to the training content in https://github.com/wso2/WSO2-Training, if applicable

Certification

Type “Sent” when you have provided new/updated certification questions, plus four answers for each question (correct answer highlighted in bold), based on this change. Certification questions/answers should be sent to<EMAIL_ADDRESS>and NOT pasted in this PR. If there is no impact on certification exams, type “N/A” and explain why.

Marketing

Link to drafts of marketing content that will describe and promote this feature, including product page changes, technical articles, blog posts, videos, etc., if applicable

Automation tests

Unit tests

Code coverage information

Integration tests

Details about the test cases and coverage

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes/no
Ran FindSecurityBugs plugin and verified report? yes/no
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes/no

Samples

Provide high-level details about the samples related to this feature

Related PRs

List any other related PRs

Migrations (if applicable)

Describe migration steps and platforms on which migration has been tested

Test environment

List all JDK versions, operating systems, databases, and browser/versions on which this feature/fix was tested

Learning

Describe the research phase and any blog posts, patterns, libraries, or add-ons you used to solve the problem.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Allow for negative numbers to be parsed

YARA recognizes having negative numerical values, but plyara will crash if it encounters a HYPHEN with a NUM type when parsing a rule.
Related external issue: https://github.com/CybercentreCanada/assemblyline/issues/231

@cccs-rs Thanks for all of this!
I cherry-pick merged almost all of these changes. I needed to decouple the keyword changes and the black formatting changes. Also, it looks like in that commit you removed two keywords that are still part of yara by accident. I added them back plus the one you added. I also brought the keywords in line with the current documentation. I also added all the missing modules that exist in the source code only line dex and macho as well as lnk which only exists in yara-x. This project uses single quotes rather than double. There may be a black setting for that? I like some of the other black formatting and adopted those. I (and the linters) disagree with black on the other formatting changes. Finally, those docstring looking single quotes in core.py are deliberate. They're not docstrings. They are a quirk of PLY. I had to adjust my linters to accomodate them. One of the projects that I'd like to complete is the transition to SLY, dabeaz's updated project that replaces PLY. I took his compilers course during the pandemic and learned how to use it. I started on that before I had to stop working on plyara for a bit.
If you're amenable, I'd like to add you as a maintainer for this project so we can work on it together.
I will be reworking the unit tests so that they're more organized over the weekend and will be cutting a 2.2 on pypi when that's done and they're passing.

Sure thing! I was thinking of writing a YARA parser in Lark because I noticed there was some issues with processing keywords/functions from imports like PE (and I also didn't know if this project was being actively maintained 😅).
Sample code for some basic parsing that I wrote as a hobby:
yara_parser = Lark(r"""
    start: yara_rule*
    yara_rule       : rule_imports [rule_scope] "rule" rule_name [":" rule_tags] "{" ["meta:" meta] ["strings:" strings] "condition:" condition_terms "}"
    rule_name       :   /\w+/
    rule_scope      :   /(private|global)/

    rule_tags       : rule_tag*
    rule_tag        : /\w+/

    rule_imports    :   rule_import*
    rule_import     :   "import" "\"" /(pe|elf|lnk|dotnet|macho|console|hash|math|string|time)/ "\""

    // META
    meta        : meta_pair*
    meta_pair   : meta_key "=" meta_value
    meta_key    : /[a-z0-9_]+/
    meta_value  : ESCAPED_STRING | SIGNED_NUMBER | /(true|false)/

    // STRINGS
    strings         : string_pair*
    string_pair     : string_key "=" string_value
    string_key      : /\$[a-z0-9_]+/
    string_value    : text_pattern | hex_pattern | regex_pattern

    // STRINGS: HEX
    hex_pattern     : "{" [byte|jump]* [hex_regex]* [byte|jump]* "}" hex_modifiers
    hex_regex       : "(" [byte|jump]* [/\|/ [byte|jump]*]* ")"
    hex_modifiers   : [hex_modifier]*
    hex_modifier    : /(private)/
    byte            : /(~)?[a-fA-F0-9?]{2}/
    jump            : "[" [SIGNED_NUMBER] "-" [SIGNED_NUMBER] "]" | "[" SIGNED_NUMBER "]"

    // STRINGS: TEXT
    text_pattern    : ESCAPED_STRING text_modifiers
    text_modifiers  : [text_modifier]*
    text_modifier   : /(xor(\(0x[0-9a-fA-F]{2}-0x[0-9a-fA-F]{2}\))?|base64(wide)?(\(".+"\))?|nocase|wide|ascii|fullword|private)/

    // STRINGS: REGEX
    regex_pattern:   /\/(.+)\/(i)?(s)?/ regex_modifiers
    regex_modifiers : [regex_modifier]*
    regex_modifier  : /(nocase|wide|ascii|fullword|private)/

    // CONDITIONS
    condition_terms : condition_term*
    condition_term : /\w+/ | string_key

    %import common.ESCAPED_STRING
    %import common.SIGNED_NUMBER
    %import common.HEXDIGIT
    %import common.WS
    %ignore WS
""")

But working on this together would be a great opportunity for collaboration! I haven't used SLY before but it's not like I'm an expert in writing parsers anyways so it would be neat to learn😁

I've added another fellow threat intel traveller to the project as a maintainer. I'll send you an invite as well. No need to keep different forks, and I would love to collaborate.

@larsborn @cccs-rs
Lets plan of a little get together via whatever kind of chat or comms you all like so we can do some coordination. Zoom works or if you have a jitsi instance or whatever works for you two

I enabled discussions on the repo. We can move this all here:
https://github.com/orgs/plyara/discussions/140


[.NET][WPF] GetResourceInformation API doesn't retrieve audio or video

Platform

.NET WPF

Version of SDK
master
Details
When using the GetResourceInformation the API in .NET doesn't retrieve video or audio information compared to its counterpart in the C++ shared model where this elements are actually retrieved

Let's confirm that the desired behavior is to return information for audio/video as well. Does this work in javascript?
If it already works everywhere except .NET we should just fix it.

Fixing this issue is not currently on our roadmap. Feel free to continue the conversation on this issue. If you feel strongly about this issue, please let us know and we can reevaluate.


Test Art peace to snforge correct version

Description
I've added the artpeace test Cairo in onchain/cairo/tests/art_peace_tests.cairo.
We need to upgrade the snforge version and all the tests with the current one.

[ ] Uncomment the import in the lib.cairo tests of the art_peace_tests
[ ] Upgrade with the current version of snforge
[ ] Change tests if different way to do the tests

Hi @MSghais, can i work on this?


Rethink trigger rules

Right now the mappers' constructor accepts a TriggerRule value, which is then completely ignored.
Instead there's an update_trigger_rules() method in parser, which sets them according to whether the particular task is a fail task (ONE_FAILED), any other task (ONE_SUCCESS, although it should rather be ALL_SUCCESS) or when it can't be determined which type a task is, it's set to DUMMY.
One approach would be to completely remove the constructor setting of the trigger rules and rely comletely on the update_trigger_rules() method.
However other ideas are welcome.

Currently, mapper is not responsible for updating trigger rules. ParsedNode is responsible for that, but mapper requires triggering rules, because they are part of the task object that mapper creates. Each mapper has trigger rules in the constructor to be able to set the default value.


Add jsdoc support for function return type in JavaScript

Suggestion

🔍 Search Terms

jsdoc
✅ Viability Checklist

My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
[x] This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion

function fun(level: 'warn' | 'error', ...args: string[]): 'foo' | 'bar' {
    return 'foo';
}

We can write the corresponding js code
function fun(/** @type {'warn'|'error'} */ level, /** @type {string[]} */ ...args) /** @type {'foo'|'bar'} */ {
    return 'foo';
}

But the return type seems to be ignored.
Can we add support for jsdoc function return type?

Use @returns. See https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html#param-and-returns

Here is another example:
/**
 * @template T
 * @param {(mode : 'dev'|'prod') =>T} arg
 * @returns {[T]}
 */
function foo(arg) {
    return [arg('dev')];
}

const bar1 = foo(
    /**
     * @returns {'a'|'b'}
     */
    (input) => {
        return 'a';
    },
);

const bar2 = foo((input) /** @type {'a'|'b'} */ => {
    return 'a';
});

In TypeScript, we may write
const bar2_ts = foo((input): 'a'|'b' => {
    return 'a';
});

The practice of bar2 is closer to the habit of TypeScript (bar2_ts) than bar1

I don't see how this is an improvement, really. You can already use returns, can already write this,
const fun = (/** @type {'warn'|'error'} */ level, /** @type {string[]} */ ...args) => {
    return /** @type {'foo'|'bar'} */('foo');
}

and the line length implied by putting @type at the => doesn't seem like something people would do


Arrangement and execution of the meeting for constitutive equation of the orthotropic layer in the arrangement long fibers and in the rotated arrangement (issue #1031 related)

The goal is to arrange and execute the meeting for constitutive equation of the orthotropic layer in the arrangement long fibers and in the rotated arrangement. The following things should be prepared:

[x] scope and schedule of meeting (in one week);

[x] conducting team (in two weeks);

[x] guide or instructions with examples (one week before meeting);

[x] announcement (in Slack and Teams channel) with basic information (conducting persons, date,scope,schedule) (one week before meeting);

[x] exemplary code in Jupyter (link) (one week before meeting).

Meeting has been executed.


ci(github): code coverage report

Calculate and report code coverage.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@0bf606d). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             main        #4   +/-   ##
========================================
  Coverage        ?   100.00%
========================================
  Files           ?         1
  Lines           ?       149
  Branches        ?         0
========================================
  Hits            ?       149
  Misses          ?         0
  Partials        ?         0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0bf606d...a0c20a2. Read the comment docs.


Write a unittest that exercises ReadWindowAggregate and gets tables back

Currently, the flux client in storage returns column readers, rather than tables.
DOD:

Ensure the storage/flux client gets flux tables, with correct data and shape.
A test exists to assert this is correct

Fixed by #18288.


It is very basic necessity to modularize the scripts in any programming. I am very very disappointed that Azure load test does not support basic controllers Include and module controllers. This is a very very big draw back for using Azure load test. Please let us know if there is any plan to implement it in this product.

It is very basic necessity to modularize the scripts in any programming. I am very very disappointed that Azure load test does not support basic controllers Include and module controllers. This is a very very big draw back for using Azure load test. Please let us know if there is any plan to implement it in this product

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 25693cda-4d10-649e-ea6b-d3bb80a2701a
Version Independent ID: a5e58364-5670-2353-ee0c-e432f91e0ecb
Content: Supported Apache JMeter features - Azure Load Testing
Content Source: articles/load-testing/resource-jmeter-support.md
Service: load-testing
GitHub Login: @ntrogh
Microsoft Alias: nicktrog

It is very basic necessity to modularize the scripts in any programming. I am very very disappointed that Azure load test does not support basic controllers Include and module controllers. This is a very very big draw back for using Azure load test. Please let us know if there is any plan to implement it in this product.

It is very basic necessity to modularize the scripts in any programming. I am very very disappointed that Azure load test does not support basic controllers Include and module controllers. This is a very very big draw back for using Azure load test. Please let us know if there is any plan to implement it in this product.

@vpayahealthcare
Thanks for your feedback! We will investigate and update as appropriate.

@vpayahealthcare Thank you for your feedback. To make sure this is logged in the right place, please use the Report a problem tool to report the issue. This allows you to engage directly with the engineering team.

#please-close


Fixed "ERROR: Using ${var} in strings is deprecated, use {$var} instead" and added logError and logErrorDetails variables on the Shutdown handler

Fixed php error on view user action (src\Application\Actions\User\ViewUserAction.php):
ERROR: Using ${var} in strings is deprecated, use {$var} instead"

Fixed php error on view user action:
ERROR: Using ${var} in strings is deprecated, use {$var} instead"

Why "Tests / Tests PHP 8.1 (pull_request)" is failed?

Why "Tests / Tests PHP 8.1 (pull_request)" is failed?

Because of whitespace convention.
Using
$short   = "hi";
$loooong = "hi";

instead of just
$short = "hi";
$loooong = "hi";

is a convention but seems against the coding guidelines here...

I fixed the php 8.1 test issues but now I'm getting a new error from Coveralls, why?

Because the new changes aren't covered in the unit tests...but I'm not sure if we should do it that way. If the error given to the shutdown handler is an error in the logger itself, we shouldn't give it to the logger. On the other hand, if it wasn't, we should, so I think we should make some try/catch around it. (Only my personal view, not a maintainer.)


Adding functionality to allow accessing authenticated user outside of…

Adding functionality to allow accessing authenticated user outside of superglobal(s).
Currently, the middleware authenticates, but retrieving the authenticated user
is still done via $_SERVER["PHP_AUTH_USER"] or $request->getServerParams()["PHP_AUTH_USER"].
There is something unnatural about both, as with any framework you're normally
not directly interacting with any of the superglobals, or their variables.
IE you access $request->getUri()->getPath() in the application, rather than
$_SERVER["REQUEST_URI"] , or $request->getServerParams()["REQUEST_URI"].
With this PR, you will be able to access the authenticated user with
$request->getAttribute("authorized_user") in your route, or other middleware ( IE ACL ),
and the attribute name is customizable through the existing options, which the
README has been updated to provide clarity on.

Going through unanswered issues and PRs.
This is a good addition and quite common use case. Currently you can do same thing with the before setting.
$app->add(new Tuupola\Middleware\HttpBasicAuthentication([
    "users" => [
        "root" => "t00r",
        "somebody" => "passw0rd"
    ],
    "before" => function ($request, $arguments) {
        return $request->withAttribute("user", $arguments["user"]);
    }
]));

I need to think a bit if the use case is common enough to warrant it's own specific config.


Cleanup gen to allow for the plethera of overrides.

https://github.com/heptio/sonobuoy/pull/267/files#diff-24099c431cb8fcc6e65f2293b7346162R51
Also update to add an function signature to the client.Interface.
/cc @liztio

This also touches #166 - so I will close that one as the listed todos are a superset.


Kubernetes Azure implementation

Hi , Thanks for perfect job. I need help for kubernetes azure implementation. I I tried but can't.
Do you think , or plan for azure.

we use it in AKS , without any issue ,kudos to opendistro.
Including TLS1.2 with openid/oauth integration to Azure AD

Hi @tolahs it has been 8 days since we last talk.
We will close this issue for now, please feel free to re-open it.
Thanks.


Ci docs preview

This PR enables branches named docs-preview-* on our repo to deploy preview docs. This enables us to push specific PR branches from forks to our repo, to quickly preview.

From pairing w/ Rich, this PR is broken because it requires #360 . I've tested by pushing this with #360 merged to a branch, and the preview appears to work.


Update provider to support Vapor 1.1 specs

Conform the PostgreSQL provider class to the updated Vapor Provider Protocol.
This fixes #7

Current coverage is 74.61% (diff: 12.50%)

Merging #8 into master will decrease coverage by 4.58%

@@             master         #8   diff @@
==========================================
  Files             2          2
  Lines           125        130     +5
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits             99         97     -2
- Misses           26         33     +7
  Partials          0          0

Powered by Codecov. Last update 3b56fda...030e7ac

Looks great @natebird! Any way we could bump the test cov? If not, this is good to tag as 1.1


Nullable enums with an empty string fail to get generated

In #59, we added support for an empty string as a valid case in a string enum, however, we only tested it on enum schemas not marked as nullable.
Turns out, that when marked as nullable, OpenAPIKit parses the values slightly differently, and treats the empty string as a nil string instead. That nil value, when converted to OpenAPIKit's AnyCodable, gets turned into a Void.
Before we introduce a workaround in the generator, @mattpolzin is this intentional behavior, and if so, can you say more about it please?

To reproduce, we can just use:
type: string
nullable: true
enum:
  - foo
  - ""

The second enum value will be an empty string when the schema is not nullable, and Void when the schema is nullable.

Unfortunately this is a known issue with Yams. https://github.com/jpsim/Yams/issues/301
OpenAPIKit has special case decoding of nullable string enums because otherwise a null case gets read in as the string "null" instead of the value null, but it is unintentional that the empty string also gets read in as null.
There is a proposed fix for the issue against Yams and a fork with that fix under the CreateAPI project but no indication of whether the fix will be upstreamed.

Thanks for the find, @mattpolzin - I wonder if @jpsim can tell us more about the prioritization of landing that fix?

I suspect we'll need to check for this ourselves in the generator, it's not a big deal, just wanted to try to see if Yams could fix it, but I suspect that'd be a breaking change anyway, so might not be likely.


material input autocomplete

Is possible to make material input autocomplete working?
 <material-input label="Email address" type="email" required autocomplete="email">
    </material-input>

Currently we haven't released auto suggest, but this is on the roadmap and will be coming soon.


Races are no fun

prefs.js (and maybe other files?) have Time-of-check-to-time-of-use (TOCTTOU) bugs. If you call prefs/set quickly enough, multiple rows will be created in the database. This is not a security issue (at least not from what I've found...) but should still be fixed. This probably should be fixed (table locking?)

@thatoddmailbox I dont know what table locking is but races are fun

Would this be considered a TOCTTOU bug?

To simulate: Click on HW view many times fast before it loads in sidebar

@RobotXLabs No, that's the tab loader thingy not canceling the HW View loader.

@thatoddmailbox GO ON SLACK

:disappointed:

This seems to be fixed (along with another really bad bug with prefs!)


Fanzy Zones - No different Layouts on different virtual desktops

Microsoft PowerToys version
0.49
Running as admin

[ ] Yes

Area(s) with issue?
FancyZones
Steps to reproduce
Running under Windows 11 Pro.
One physical Display, but four virtual Desktops. I used Fancy Zones to have different Layouts on different Desktops under Windows 10 Pro. That worked fine for me.
Now, the last assigned Layout affects all desktops.
In the Fancy Zones-Layout-Editor there's on the top just one single Display shown, no possibility to assign different layouts to different desktops any more.
Unusuable version for me.
✔ Expected Behavior
Possibility to assign different layouts to different virtual desktops.
❌ Actual Behavior
The last assigned Layout affects all desktops.
Other Software
No response

Can you please attach a bug report?
Right-click on the tray icon and click Report Bug. Then drag-and-drop the generated file into the GitHub comment to upload it. Thanks!

I suffer from this too.


MB 3484 Pre Demo Cleanup

Description
This branch makes the changes requested in this document which includes:

[x] adding space before the optional label
[x] spacing on form section headers
[x] radio button spacing
[ ] try switching over everything to Public Sans
[ ] margin and sizing of hhg move page header

Reviewer Notes
I am having trouble figuring out where I need to go to switch us off of Source sans. I also could use help figuring out how to navgate to the page where this header is located so I can adjust its styling:

Setup
In storybook:
yarn install
yarn stroybook

In the app
make server_run
make client_run

Code Review Verification Steps

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] User facing changes have been reviewed by design.
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

References

Jira story for this change
Google Doc outlining requested changes explains more about the approach used.

Screenshots

Warnings

:warning:
This PR does not include changes to unit tests, even though it affects app code.

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against fb7888a6e1a8c0c20f2ca3db0305daf6716a1aa3

The changes look good to me in storybook. Unfortunately, the styles don't show up locally.  Sounds like the reason is related to Suz's comment about the ghc_index.scss file not being imported into the customer-facing app.

Re-reviewing, based on local envt:
Spacing between the radios for Delivery address look better
I'm not seeing spacing between optional and the label
I do see better spacing between form section headers
There's changes to buttons that I don't think are intended:
The button is now width of section, don't think it was before

and
The button colors look funny, and on the browser (non-phone view) the buttons have moved directly under the form.  It looks more like a mobile view on the app view.


Add PgVectorChatMemory

An option to use PgVectorChatMemory for users that want to use pgvector as chat memory.

@leijendary which advisor you should use with that ChatMemory to process saving ?

@ogbozoyan You can use MessageChatMemoryAdvisor and pass in the instance of the PgVectorChatMemory. Then for RAG, you can use RetrievalAugmentationAdvisor. This is what I did for my chatClient bean:
@Bean
fun chatClient(builder: ChatClient.Builder, vectorStore: VectorStore, chatMemory: ChatMemory): ChatClient {
    val memoryAdvisor = MessageChatMemoryAdvisor(chatMemory)
    val documentRetriever = VectorStoreDocumentRetriever.builder()
        .vectorStore(vectorStore)
        .build()
    val ragAdvisor = RetrievalAugmentationAdvisor.builder()
        .documentRetriever(documentRetriever)
        .build()

    return builder
        .defaultAdvisors(memoryAdvisor, ragAdvisor)
        .build()
}

@ogbozoyan You can use MessageChatMemoryAdvisor and pass in the instance of the PgVectorChatMemory. Then for RAG, you can use RetrievalAugmentationAdvisor. This is what I did for my chatClient bean:
@Bean
fun chatClient(builder: ChatClient.Builder, vectorStore: VectorStore, chatMemory: ChatMemory): ChatClient {
    val memoryAdvisor = MessageChatMemoryAdvisor(chatMemory)
    val documentRetriever = VectorStoreDocumentRetriever.builder()
        .vectorStore(vectorStore)
        .build()
    val ragAdvisor = RetrievalAugmentationAdvisor.builder()
        .documentRetriever(documentRetriever)
        .build()

    return builder
        .defaultAdvisors(memoryAdvisor, ragAdvisor)
        .build()
}

in this example you passing VectorStore, may be you meant ?
fun chatClient(builder: ChatClient.Builder, **pgVectorChatMemory: PgVectorChatMemory**, chatMemory: ChatMemory): ChatClient {
     val memoryAdvisor = MessageChatMemoryAdvisor(**pgVectorChatMemory**)
     val documentRetriever = VectorStoreDocumentRetriever.builder()
         .vectorStore(vectorStore)
         .build()
     val ragAdvisor = RetrievalAugmentationAdvisor.builder()
         .documentRetriever(documentRetriever)
         .build()

     return builder
         .defaultAdvisors(memoryAdvisor, ragAdvisor)
         .build()
}

@ogbozoyan

@ogbozoyan You can use MessageChatMemoryAdvisor and pass in the instance of the PgVectorChatMemory. Then for RAG, you can use RetrievalAugmentationAdvisor. This is what I did for my chatClient bean:
@Bean
fun chatClient(builder: ChatClient.Builder, vectorStore: VectorStore, chatMemory: ChatMemory): ChatClient {
    val memoryAdvisor = MessageChatMemoryAdvisor(chatMemory)
    val documentRetriever = VectorStoreDocumentRetriever.builder()
        .vectorStore(vectorStore)
        .build()
    val ragAdvisor = RetrievalAugmentationAdvisor.builder()
        .documentRetriever(documentRetriever)
        .build()

    return builder
        .defaultAdvisors(memoryAdvisor, ragAdvisor)
        .build()
}

in this example you passing VectorStore, may be you meant ?
fun chatClient(builder: ChatClient.Builder, **pgVectorChatMemory: PgVectorChatMemory**, chatMemory: ChatMemory): ChatClient {
     val memoryAdvisor = MessageChatMemoryAdvisor(**pgVectorChatMemory**)
     val documentRetriever = VectorStoreDocumentRetriever.builder()
         .vectorStore(vectorStore)
         .build()
     val ragAdvisor = RetrievalAugmentationAdvisor.builder()
         .documentRetriever(documentRetriever)
         .build()

     return builder
         .defaultAdvisors(memoryAdvisor, ragAdvisor)
         .build()
}

The code I provided is correct. I am using it right now. The ChatMemory instance is automatically picked up by spring as PgVectorChatMemory because of PgVectorChatMemoryAutoConfiguration

I wonder if this should be more generic and instead providing a JdbcChatMemory instead. I would be also nice to provide the proper scripts to create the tables, similar to what spring security and spring session does.

@eddumelendez I agree. I also originally designed it to be the same as the existing CassandraChatMemory and just used the same columns. I thought of changing the columns to the following:
CREATE TABLE ai_chat_memory (
    session_id character varying(36) NOT NULL,
    content text NOT NULL,
    type character varying(10) NOT NULL, -- USER/ASSISTANT
    "timestamp" timestamp without time zone NOT NULL DEFAULT NOW()
);

Do you also mean that the table should not be automatically created and instead be created manually by the developer?

Do you also mean that the table should not be automatically created and instead be created manually by the developer?

it should be created automatically by detecting the database. I meant having something like https://github.com/spring-projects/spring-session/blob/main/spring-session-jdbc/src/main/resources/org/springframework/session/jdbc/schema-postgresql.sql that spring ai can execute, so, if new database should be supported then adding a script should be enough

type character varying(10) NOT NULL, -- USER/ASSISTANT

maybe add check ?
CREATE TABLE ai_chat_memory (
    session_id character varying(36) NOT NULL,
    content text NOT NULL,
    type character varying(10) NOT NULL CHECK (type IN ('USER', 'ASSISTANT')), -- USER/ASSISTANT
    "timestamp" timestamp without time zone NOT NULL DEFAULT NOW()
);


Reduced Font size

I have been successfully running the emulator for a few weeks. Today the font size mysteriously shrank to an illegible 1 point font. I have not updated the npm package, or made any other changes that I know of. Where does the application pull the font size from? I am using version "3.5.36".
Thanks in advance.

Please ignore - Ctl-+ ... Face palm...


Add Apache Guacamole

Issue description
Add clientless remote desktop gateway: https://guacamole.incubator.apache.org/
Steps to reproduce
Create nix expression.
Technical details
https://guacamole.incubator.apache.org/doc/gug/installing-guacamole.html
I will look into packaging this. It may be more fruitful than gateone or cloud9.

I actually have a local setup of this, but it is done in a very hacky way, so probably better off if you work on a service expression yourself. Probably need some improvements to the existing Tomcat module first though, from what I remember.

@Shados Please post any expressions you may have build.
For the client, i just grab the binary (do you build it?):
wget https://sourceforge.net/projects/guacamole/files/current/binary/guacamole-0.9.9.war/download\?use_mirror\=heanet -O guacamole-0.9.9.war
I've not made a service yet. You've hit on the issue, the tomcat setup is more convoluted, especially with DB authentication and the properties/user-mappings.
@Shados: this works for the server (depends on #17936) (note removing a warning due to a deprecated function in glibc).
{stdenv, fetchFromGitHub, autoreconfHook, cairo, libjpeg_turbo, libpng, libossp_uuid
,freerdp, pango, libssh2, libvncserver, libpulseaudio ,openssl, libvorbis, libwebp, pkgconfig, perl, libtelnet,
makeWrapper}:

stdenv.mkDerivation rec {
  name = "guacamole-${version}";
  version = "0.9.9";

  src = fetchFromGitHub {
      owner = "apache";
      repo = "incubator-guacamole-server";
      rev = "6c0862e82da8ee737ee655c7815b1851de6d0488";
      sha256 = "1wzlg06yzr3lbxdjjj9rkhdv6g6nr62sdmkklq5f99fj1jclgvic";
  };
  buildInputs = with stdenv; [ autoreconfHook pkgconfig cairo libpng libjpeg_turbo libossp_uuid freerdp pango libssh2 libvncserver libpulseaudio openssl libvorbis libwebp libtelnet perl makeWrapper];
  # propogatedBuildInputs = with stdenv; [ autoreconfHook pkgconfig cairo libpng libjpeg_turbo libossp_uuid freerdp pango libssh2 libvncserver libpulseaudio openssl libvorbis libwebp inetutils];
  patchPhase = ''
    substituteInPlace ./src/protocols/rdp/keymaps/generate.pl --replace /usr/bin/perl "${perl}/bin/perl"
    substituteInPlace ./src/protocols/rdp/Makefile.am --replace "-Werror -Wall" "-Wall"
  '';
  postInstall = ''
    echo "Wrap"
    wrapProgram $out/sbin/guacd --prefix LD_LIBRARY_PATH ":" $out/lib
    '';

  meta = with stdenv.lib; {
    description = "Clientless remote desktop gateway";
    homepage = "https://guacamole.incubator.apache.org/";
    maintainers = [ stdenv.lib.maintainers.tomberek ];
    license = licenses.cc-by-40;
    platforms = platforms.linux;
  };
}

@tomberek
Yeah, I just grab the WAR as well, arguably it is more the Nix way to build it from source but I was just looking to get something working rather than something clean :).
My expression for the server is pretty similar, but instead of wrapping guacd I used dontPatchELF = 1; to avoid removing the RPATHs for the dynamically linked libraries. Also I didn't bother to support as many protocols (e.g. RDP) as I didn't need them for that environment.
Here's my service expression, but I doubt it will be very helpful. It's kind of a mess, fairly specific to my own setup (which e.g. has a reverse-proxy in front of guacamole, not covered therein), and I haven't altered the tomcat service expression to be any better.

@tomberek @shados Not strictly on-topic, but since it sounds like you both have or did have this running, can you comment on the performance of remote access via guac relative to other options? I've found everything but teamviewer to be frustratingly slow. Is this tree worth barking up? :)

well... it's possible i got this working again... still needs some cleanup, but i combined @Shados 's work and it's up and running.

I'd also be interested in native nix expression. Currently I just run it in docker.

I'm interested in this (please un-stale it).

also very interested in this!

I'm interested in this (please un-stale it).

I doo am interested in it

I'm interested as well.

This is a must-have. Thank you for opening this, we can thumbs up this comment for anyone else who's interested in making this a native nix expression.

For anyone interested - I created a working flake (based on the linked PRs): https://github.com/jbuchermn/guacamole-nixos/tree/main
Configuration is not yet done in a nice way, but if you need to get guacamole running, this could help.

@jbuchermn Cool news, I will most probably try it out soon.
By the way, how about submitting the PR in nixpkgs directly so we could have a NixOS module?

great work @jbuchermn are you planning to make a PR?

Hi all,
I just created a draft PR to add Guacamole server and client.
This PR does not include the NixOS module since @jbuchermn has plan to update it.
In the meantime, could you review that new draft PR: https://github.com/NixOS/nixpkgs/pull/237824 ?

Just as side note, I landed here from https://nixos.wiki/wiki/Remote_Desktop - where it mentions guacamole not yet being available.

Just as side note, I landed here from https://nixos.wiki/wiki/Remote_Desktop - where it mentions guacamole not yet being available.

Thanks for noticing it!
Any user having a GitHub account is free to edit the wiki. Are you willing to do it?

Can you provide a sample configuration? I'm also unclear of the difference between services.guacamole-client.enable and services.guacamole-client.enableWebserver

Can you provide a sample configuration? I'm also unclear of the difference between services.guacamole-client.enable and services.guacamole-client.enableWebserver

Here's my own server configuration: https://github.com/drupol/nixos-x260/blob/master/hosts/nixos/guacamole.nix
@BananaAcid Are you willing to edit the Wiki page: https://nixos.wiki/wiki/Remote_Desktop ?

Thanks for the configuration. However I still can't get it to run. Am I right to assume, that once it's setup, you should get a tomcat webview in which you can select what connection you want to open (defined for your user in the user-mapping.xml). That's how it works on an ubuntu install in a vm that I did. I surf to it, login and then I have a bunch of connections (or can setup a new one).
nginx as reverse proxy just prints 502 bad gateway error.
I have this in my configuration.nix:
    # Guacamole
    services.guacamole-server = {
        enable = true;
        host = "<IP_ADDRESS>";
        userMappingXml = /data/guacamole/user-mapping.xml;
    };

    services.guacamole-client = {
        enable = true;
        enableWebserver = true;
        settings = {
            guacd-port = 4822;
            guacd-hostname = "localhost";
        };
    };

    # Enable nginx
    services.nginx = {
        enable = true;
        recommendedTlsSettings = true;
        recommendedOptimisation = true;
        recommendedGzipSettings = true;
        recommendedProxySettings = true;
        clientMaxBodySize = "500m";
        virtualHosts = {
            "my.domain.tld" = {
                forceSSL = true;
                sslCertificate = "/data/acme/fullchain.cer";
                sslCertificateKey = "/data/acme/domain.tld.key";
                http2 = true;
                locations."/" = {
                    proxyPass = "https://<IP_ADDRESS>:4822/";
                };
            };
        };
    };
    # Setup Networking
    networking = {
        firewall = {
            enable = true;
            allowPing = true;
            allowedUDPPorts = [ 80 443 4822 ];
            allowedTCPPorts = [ 22 80 443 4822 ];
            # OpenSSH: 22
            # Web: 80 443
            # Guacamole: 4822
        };
    };

journalctl -b0 -u guacamole-server.service
Aug 09 10:17:45 dns3 systemd[1]: Started Apache Guacamole server (guacd).
Aug 09 10:17:46 dns3 guacd[1182]: Guacamole proxy daemon (guacd) version 1.5.2 started
Aug 09 10:17:46 dns3 guacd[1182]: guacd[1182]: INFO:        Guacamole proxy daemon (guacd) version 1.5.2 started
Aug 09 10:17:46 dns3 guacd[1182]: guacd[1182]: INFO:        Listening on host <IP_ADDRESS>, port 4822
Aug 09 10:17:46 dns3 guacd[1182]: Listening on host <IP_ADDRESS>, port 4822

journalctl -b0 -u nginx.service
Aug 09 10:20:20 dns3 systemd[1]: Starting Nginx Web Server...
Aug 09 10:20:20 dns3 nginx-pre-start[1461]: nginx: the configuration file /nix/store/phbyim3xpcw9vbrb7xcrbcsfryzwgzj9-nginx.conf syntax is ok
Aug 09 10:20:20 dns3 nginx-pre-start[1461]: nginx: configuration file /nix/store/phbyim3xpcw9vbrb7xcrbcsfryzwgzj9-nginx.conf test is successful
Aug 09 10:20:20 dns3 systemd[1]: Started Nginx Web Server.

I also tried with                     proxyPass = "https://<IP_ADDRESS>:4822/guacamole"; but no luck either.

I have the feeling that the issue is becoming more about how to setup the proxy than Guacamole itself.
I've never used nginx, so I can't help on this, I'm always using Caddy... you should try it ;)
What I would suggest is to just try to setup guacamole without the proxy, this should be quite straightforward. Once it works, setup the proxy on the top.

It seems to me that the discussion is shifting more towards the configuration of the proxy rather than focusing on Guacamole itself.
I must confess, I'm unfamiliar with nginx, as my experience is primarily with Caddy — which I highly recommend you give a try, by the way! 😉
To approach this issue more systematically, I suggest first setting up Guacamole without the proxy. This should be a fairly straightforward process. Once you've successfully configured Guacamole, you can then layer the proxy on top. Breaking it down into these two separate steps may simplify the process and help identify any underlying issues more clearly.

How to use it without proxy? I tried to access it directly by ip address and port - no luck.

The webserver is by default on port 8080, that's the reason why I used caddy in my config, see line 27: https://github.com/drupol/nixos-x260/blob/17393d655712f5a01175a910ae4e553a49c20510/hosts/nixos/guacamole.nix#L27
I guess you just have to access http://<IP_ADDRESS>:8080/

The webserver is by default on port 8080, that's the reason why I used caddy in my config, see line 27: https://github.com/drupol/nixos-x260/blob/17393d655712f5a01175a910ae4e553a49c20510/hosts/nixos/guacamole.nix#L27
I guess you just have to access http://<IP_ADDRESS>:8080/

Ok, now that makes sense... I always tried port 4822 :)

Glad you found out. 4822 is the port of the guacd service (guacamole-server).

Ok, figured it out:
Caddy
    services.caddy = {
        enable = false;
        virtualHosts."domain.tld.ch".extraConfig = ''
            tls /data/acme/fullchain.cer /data/acme/domain.tld.key
            handle_path /* {
                rewrite * /guacamole{path}
                reverse_proxy <IP_ADDRESS>:8080 {
                    flush_interval -1
                }
            }
        '';
    };

nginx
    services.nginx = {
        enable = true;
        recommendedTlsSettings = true;
        recommendedOptimisation = true;
        recommendedGzipSettings = true;
        recommendedProxySettings = true;
        clientMaxBodySize = "500m";
        virtualHosts = {
            "domain.tld" = {
                forceSSL = true;
                sslCertificate = "/data/acme/fullchain.cer";
                sslCertificateKey = "/data/acme/domain.tld.key";
                http2 = true;
                locations."/" = {
                    proxyPass = "http://<IP_ADDRESS>:8080";
                    extraConfig = ''
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_buffering off;
                        proxy_http_version 1.1;
                        proxy_set_header Upgrade $http_upgrade;
                        proxy_set_header Connection "upgrade";
                        access_log off;
                    '';
                };
            };
        };
    };

Excellent, I might reuse what you did in the documentation :)
Caddy is amazing, glad you're using it :)

Excellent, I might reuse what you did in the documentation :)

Sure, go ahead.

Caddy is amazing, glad you're using it :)

Well, just for testing on that machine. The actual machine where I run guacamole (up to know in an ubuntu vm - so I can scratch that now) is already using nginx as reverse proxy for several things (plex, roundcube, wetty, photoprims, home assistant).

Do you maybe have a user-mapping.xml entry on how to connect to Win10 or Win11? I tried rdp with and without NLA but can't get achieve any connection.

I'm not using Windows machine, sorry...

I have a problem with sound/file sharing
tried version 1.5.3, 1.5.5. does someone knows what's going on?
Cannot create static channel "rdpdr": failed to load "guac-common-svc" plugin for FreeRDP.
WARNING:        Cannot create static channel "rdpdr": failed to load "guac-common-svc" plugin for FreeRDP.
WARNING:        Support for the RDPDR channel (device redirection) could not be loaded. Drive redirection and printing will not work. Sound MAY not work.
WARNING:        Cannot create static channel "rdpsnd": failed to load "guac-common-svc" plugin for FreeRDP.
WARNING:        Support for the RDPSND channel (audio output) could not be loaded. Sound will not work. Drive redirection and printing MAY not work.
Support for the RDPDR channel (device redirection) could not be loaded. Drive redirection and printing will not work. Sound MAY not work.
Cannot create static channel "rdpsnd": failed to load "guac-common-svc" plugin for FreeRDP.
Support for the RDPSND channel (audio output) could not be loaded. Sound will not work. Drive redirection and printing MAY not work.

this is what i have in journalctl --unit=guacamole-server.service
apparently there's freerdp2 shared libraries in guacd, so idk :\