Use a ThreadLocal shared list for absolute sequence propagation

[x] Depends on https://github.com/ppy/osu-framework/pull/4723

Working under the following requirements:

No changes to API
Assumption that this is going to be a hot path in most projects (not just osu!) and should be expected to not allocate more than necessary
Should work from load thread or update thread, as it currently does

This seems like about the best we can do. I can understand if this approach is a tad hard to swallow, but wanted to push this out for discussion. Using "static" lists with the paradigm I've used here is safe enough, but the lists do still have the ability to grow and reside in memory indefinitely. Maybe this is fine?
If we are not happy with ThreadLocal, it is also enough to cater to the most common usage (ie. only use a shared list on the Update thread, and construct a new one in all other cases).
Note that this is a similar issues to #4683 and we may want to decide on a path forward to use in cases like these as they come up. #4683 is more simple as it can only ever run on the update thread (and also know the length from the outset).
Before:

Method
Mean
Error
StdDev
Gen 0
Allocated

NonRecursive
1,008.6 us
19.83 us
17.58 us
-
187 B

Recursive
970.0 us
15.64 us
14.63 us
1.9531
24,259 B

SlightlyNestedNonRecursive
1,003.6 us
18.42 us
16.33 us
-
191 B

SlightlyNestedRecursive
1,058.8 us
20.21 us
17.91 us
5.8594
72,334 B

VeryNestedNonRecursive
1,031.8 us
20.08 us
23.91 us
-
187 B

VeryNestedRecursive
1,124.8 us
21.70 us
22.28 us
3.9063
49,555 B

After:

Method
Mean
Error
StdDev
Allocated

NonRecursive
1.006 ms
0.0200 ms
0.0231 ms
214 B

Recursive
1.039 ms
0.0205 ms
0.0191 ms
267 B

SlightlyNestedNonRecursive
1.010 ms
0.0161 ms
0.0179 ms
211 B

SlightlyNestedRecursive
1.135 ms
0.0206 ms
0.0192 ms
267 B

VeryNestedNonRecursive
1.009 ms
0.0173 ms
0.0153 ms
211 B

VeryNestedRecursive
1.105 ms
0.0137 ms
0.0121 ms
267 B

Note a slight allocation overhead due to the thread check. Feels best to have it in place at least initially?
Real world, this remove the array allocations associated with BeginAbsoluteSequence (blue line):

Closing this one (see https://github.com/ppy/osu-framework/issues/4761 for reasoning)


[dv] Rework (aes_)shadow_reg_errors

With #4781 we have reworked how the shadow register errors (update and storage errors) translate into alerts and the implications they have on the AES module. Unfortunately, I wasn't careful enough and didn't realize that the aes_shadow_reg_errors test is neither run in CI nor in the default config I used to verify the changes. As a result, the design and DV are now out of sync. I tried to fix it myself but since I am not familiar enough with UVM and the CSR tests, I need some help.
In particular, the main changes I did and that we would need to adjust the test infrastructure for are:

Update and storage error no longer necessarily have an own alert called "csr_name"_storage/update_err. In the case of AES, we have two alert signals fatal and recoverable that can both collect multiple alert conditions. What would probably be needed here is a tag in the hjson description saying which alerts the storage and update errors are connected to (Like the tag specifying the path of the shadow register in the design).
Fatal alerts (e.g. produced by a storage error) should lock up the peripheral. In AES it is now no longer possible to get out of such a condition except by a reset. Writes to the shadow reg are ignored after a storage error is detected and the alert remains asserted - even if the backdoor interface is used again to put the two copies back in sync. This behavior is not inherent to the shadow register but to AES. The test framework might need to adapt on a per case basis.
In AES, the recoverable alert (produced by an update error) remains asserted until the next write to the shadow register. (This could also be changed in the RTL.)
In AES, we now have a separate status bit for both fatal and recoverable alerts in the status register. Previously, we just had one for storage errors/fatal alert. As a result, the test now sometimes stops because it reads an unexpected value in the status register.

I am sorry for not realizing this earlier and would be happy if @cindychip and/or @weicaiyang could have a look. Also pinging @rasmus-madsen and @sriyerg for visibility.

Thanks @cindychip ! Re 3.: I will now do a PR make the update error being asserted for just one cycle as this is required by the new alert guidance. However, the corresponding status register bit will remain high until the next write of the control register. I will ping you in the PR to keep you in the loop.

Thanks @cindychip ! Re 3.: I will now do a PR make the update error being asserted for just one cycle as this is required by the new alert guidance. However, the corresponding status register bit will remain high until the next write of the control register. I will ping you in the PR to keep you in the loop.

Sorry I missed the PR: https://github.com/lowRISC/opentitan/pull/4909. I will make an update in shadow_reg test to support recoverable_error assert one clock cycle and fatal_error keeps firing until reset by the end of the week.

One remaining item on DV: check fatal error continuously triggered without enabling scb. (Make sure this can be also used in keymgr)

Hi @vogelpi! Just want to follow up on this:

However, the corresponding status register bit will remain high until the next write of the control register. I will ping you in the PR to keep you in the loop.
Have you implemented this already? I just want to double check to make sure I did not miss your PR.

Thanks,
Cindy

Hey @cindychip ,
the corresponding PR (#4909) got merged quite a while ago. Sorry, I didn't ping you in here, but just in the PR.
The behavior of the two alerts is as described initially:

recov fires for just one clock cycle. The corresponding status bit in the STATUS register remains high until the next write to the shadow register occurs. This second part (keep status big asserted until next write) probably doesn't need to be tested. It's not part of the shadow register specification but more specific to how the shadow register is used inside AES. I can imagine some shadow registers will not have status bits at all.
fatal remains asserted until reset. There is also a status bit in the STATUS register that remains asserted until reset. But again, that's quite specific to AES probably.

So, I think from the AES side everything looks good. Thanks a lot for the work you put into this Cindy!
Or is there anything else you wanted to know from me?

Thanks @vogelpi for getting back to me! I think the AES sides should be good as well. I am just reviewing this issue and see if i missed anything. The status register checking is covered in the shadow_reg_err sequence.
I will close this PR since all the updates are covered in the shadow_reg_errors sequence.
Please feel free to open it again if you find anything missing :)
Thanks,
Cindy


Use mirror of curse API

Hello! I propose to add the useCurseMirror clause in this class. Here's a view of it: curseforge.com/api/v1/mods/${ProjectID}/files/${FileID}/download.
The point is that not everyone has access to api.curse.tools. (see my issue)

As the official API requires an API key, this option is not considered for the time being.


Consider renaming Async::new()

Since Async::new() should really only be used for networking and not for stdin/stderr/stdout/files, what if we renamed Async::new() to Async::socket() to prevent people from putting files etc. inside Async and expecting things to work?
That said, there are some things that aren't sockets and can be used with Async like timerfd and notify, but we can say those are socket-like things.
On Windows, Async::new() only accepts T: AsRawSocket so you really can only use sockets with it - the situation there is perfect. On Unix, there's just AsRawFd and there's not really any way to distinguish between sockets and other kinds of file descriptors.

What about removing the Async::new() entirely and use Async::try_from() for all T: AsRawFd?
Maybe I should open a PR with that?

Maybe instead of "socket", "pollable fd" would be a better term? It seems weird to call the various other kinds of pollable fds on Linux "sockets" :)

On Windows, Async::new() only accepts T: AsRawSocket so you really can only use sockets with it - the situation there is perfect.

This also seems like a good reason to rename, as one might want to add an Async::handle() in the future that works on HANDLEs, for example.

@shekohex Interesting, haven't thought of that! But I worry people would do Async::try_from(file)?
@sdroege I like that - so we can explain Async is only for pollable things, which files and stdio aren't.

@sdroege I like that - so we can explain Async is only for pollable things, which files and stdio aren't.

stdio/out/err are actually pollable or not (just not on Windows)? I remember doing that quite a few times.

@sdroege As long as the AsRawFd bound is kept, you just can't stop people writing Async::try_from(file)?. However, you shouldn't because the method's name already suggests a known idiomatic approach most rust developers are familiar with.

How about Async::pollable()?

stdio/out/err are actually pollable or not (just not on Windows)? I remember doing that quite a few times.

That of course depends on what your stdio streams happen to be. If they're on-disk files, they're not pollable. If they're ptys, or sockets, or pipes, they are :slightly_smiling_face: So it's definitely true that you cannot assume they're always pollable.

How about Async::pollable()?

Perhaps Async::polling(smth)? Though that sounds a bit Swift-ish.

I decided to not change this :) It seems that the original issue hasn't come up again since the initial release of smol. That may be partly because more people have become familiarized with Async vs Unblock distinction, and partly because there are now higher-level crates like async-net and async-fs.


Laboratory work #3, Tormashova Valeriya - 19FPL2

don't forget about target score if you don't want 10

But I haven't added realisation of the last function for score 8 and not sure that I'll do it later


Inconsistent sorting between formatting all files and a single file

What version of prettier-plugin-tailwindcss are you using?
v0.5.11
What version of Tailwind CSS are you using?
v3.4.1
What version of Node.js are you using?
v20.10.0
What package manager are you using?
npm, pnpm
What operating system are you using?
Windows (WSL)
Reproduction URL
https://github.com/lesha1201/tailwind-prettier-plugin-inconsistent-sorting-issue
Describe your issue
https://github.com/tailwindlabs/prettier-plugin-tailwindcss/assets/10157660/8590929d-0f29-4c0c-8752-a2be1496bd5a
prettier-plugin-tailwindcss formats a certain file differently when you format all files and only that file. In the reproduction example, there are two components: ButtonSocialApple and ButtonSocialFacebook which have similar tailwind classes.
// src/app/button-social/button-social.apple.tsx
// ...
export function ButtonSocialApple({ children }: ButtonSocialAppleProps) {
  return (

      {children}

  );
}

// src/app/button-social/button-social.apple.tsx
// ...
export function ButtonSocialFacebook({ children }: ButtonSocialFacebookProps) {
  return (

      {children}

  );
}

When you format all files by running:
npx prettier --write .

It produces one result for ButtonSocialFacebook:
export function ButtonSocialFacebook({ children }: ButtonSocialFacebookProps) {
  return (

      {children}

  );
}

When you format only ButtonSocialFacebook:
npx prettier --write ./src/app/button-social/button-social-facebook.tsx

It produces a different result:
export function ButtonSocialFacebook({ children }: ButtonSocialFacebookProps) {
  return (

      {children}

  );
}

However, when you remove ButtonSocialApple component, it formats ButtonSocialFacebook consistently.

If I remove env.context.getClassOrder from sortClassList then it works fine so I guess the issue is in env.context.getClassOrder.
export function sortClassList(classList, { env }) {
  let classNamesWithOrder = env.context.getClassOrder
    ? env.context.getClassOrder(classList)
    : getClassOrderPolyfill(classList, { env })

  return classNamesWithOrder
    .sort(([, a], [, z]) => {
      if (a === z) return 0
      // if (a === null) return options.unknownClassPosition === 'start' ? -1 : 1
      // if (z === null) return options.unknownClassPosition === 'start' ? 1 : -1
      if (a === null) return -1
      if (z === null) return 1
      return bigSign(a - z)
    })
    .map(([className]) => className)
}

Hey — so this is a problem in Tailwind CSS itself coupled with the fact that we have to do some things in the Prettier plugin for performance reasons.
I've merged in a fix in https://github.com/tailwindlabs/tailwindcss/pull/12911 and I'm going to work on back porting to v3.4 and get a patch release out this week.
Thanks for pointing this out!


No way to link a java class definition back to the project explorer?

I find this dependency view of limited use unless I can link to it from an open class definition, e.g. to browse classes in the same package as one that I opened directly with CTRL-Click, or similar.

https://github.com/Microsoft/vscode-java-dependency/pull/89


Missing break-line when sending chat messages

When typing a message and one inserts an break-line and the message is sent to another user, the break-line seems to be deleted from the message and does not show in the sent message.
Steps to reproduce:

In any chat, type a message and insert a break-line.
Send the message and notice how the break-line has disappeared in the sent message.

This issue has been fixed in this commit: https://github.com/waveyboym/Chat-App/commit/cec508c1226547fd4626a4d58680cff16cf9de4b
external ref: https://stackoverflow.com/a/57160503


nixos/freshrss: easier nginx vhost configuration

This patch simplifies configuring the nginx virtualHost used by freshrss, by exposing a new services.freshrss.nginx.virtualHost option.
I renamed the services.freshrss.virtualHost option to services.freshrss.hostname and deprecated virtualHost to avoid confusion with the new nginx.virtualHost option and to make this module more consistent with other modules that provide a nginx.virtualHost option.

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[X] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@ofborg freshrss.passthru.tests


Intrinsic definition collision for ARMv7-A/Cortex-A9

Compiler: GCC 13.2.1 (Arm GNU Toolchain)
Target: Cortex-A9 (-mcpu=cortex-a9 -mfpu=neon) bare metal, no OS
Flags: -DNEON
With the addition of #219, building now fails due to redefinition of certain intrinsics and macros previously defined at CMSIS/Core/Incclude/cmsis_gcc.h:938. There's a note at https://github.com/ARM-software/CMSIS-DSP/blob/4404d3e65bd367754da83a8dd1f5669496677231/Include/arm_math_types.h#L115-L116 that seems relevant, but if that were the case there wouldn't be a name collision.
Attempting to build without providing an inclusion against CMSIS/Core/Include instead raises a different error: "fatal error: cmsis_compiler.h: No such file or directory".
I assume that Core must have its includes available for the DSP module, but doing so creates the collision. Deleting the check against ARM_MATH_NEON in arm_math_types.h removes the issue.

@stellar-aria You can't build by just defining ARM_MATH_NEON and __GNUC_PYTHON__ ?
__GNUC_PYTHON__ will disable the inclusion of cmsis_gcc.h (and the dependency to CMSIS-Core) and ARM_MATH_NEON will enable the Neon variants when available.

Above flags are for make. With cmake it should be -DNEON=ON and -DHOST=ON

I have made a commit for this issue. It should solve the problem with CMSIS Core if you build with above defines.

Yes, removing the CMSIS Core reference and adding HOST ON worked, thanks so much!


Fix request: "module" entrypoint does not exist

Building with<EMAIL_ADDRESS>it complains about dist/use-interval-ts.esm.js and it's actually right, the file does not exists.
🚨 Build failed.

@parcel/core: Failed to resolve '@use-it/interval' from './app/components/Match.js'

  /Users/damiano/Desktop/tt/app/components/Match.js:7:25
    6 | import useAxios from 'axios-hooks';
  > 7 | import useInterval from '@use-it/interval';
  >   |                         ^^^^^^^^^^^^^^^^^^
    8 | import AudioToggle from '../AudioToggle.js';
    9 | import Board from './Board.js';

@parcel/resolver-default: Could not load './dist/use-interval-ts.esm.js' from module '@use-it/interval' found in package.json#module

  /Users/damiano/Desktop/tt/node_modules/@use-it/interval/package.json:57:13
    56 |   ],
  > 57 |   "module": "dist/use-interval-ts.esm.js",
  >    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ './dist/use-interval-ts.esm.js' does not exist, did you mean './dist/interval.esm.js'?'
    58 |   "size-limit": [
    59 |     {

Any news here, we ran in the same issue?

This is addressed in #34, but @donavon hasn't yet chimed in or merged it.


Improved unit test coverage.

Relates to #62. Coverage is now at 64%.

Refactored code for testing purposes.
Misc. comment changes.

@ConnorDoyle PTAL. I updated it.

I don't think the bot picked-up the comment from your approval. I will self lgtm.

/lgtm

/lgtm


feat(core): specify default Cache-Control for responses when not set …

…by upstream (#177)

Add default_cache_control for core and traefik

Move adding default Cache-Control header to bridge.go

Add to core docs

Add to traefik plugin docs

Add for Caddy plugin

Add to Caddy plugin docs

Add code and docs for Tyk plugin

Add Caddy plugin docs to top-level README

@Menci Can you push on this branch instead of your fork please because it will trigger automatically the tests workflow.

You can add on.pull_request to your GitHub Actions workflow files to make every PR's commit tested in CI.

Sadly I have no permission ...

Let's close it and continue in #183.


flash_attn not found when using docker, and Alibaba-NLP/gte-Qwen2-1.5B-instruct

System Info
Using docker latest (July 21st), with the following params:
v2   --port 8000 	--model-id Alibaba-NLP/gte-Qwen2-1.5B-instruct --model-id jinaai/jina-embeddings-v2-base-en
(note, jina-embeddings on its own works fine).
Information

[X] Docker
[ ] The CLI directly via pip

Tasks

[X] An officially supported command
[ ] My own modifications

Reproduction
ERROR:    Traceback (most recent call last):
  File "/app/.venv/lib/python3.10/site-packages/starlette/routing.py", line 732, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/app/infinity_emb/infinity_server.py", line 59, in lifespan
    app.engine_array = AsyncEngineArray.from_args(engine_args_list)  # type: ignore
  File "/app/infinity_emb/engine.py", line 256, in from_args
    return cls(engines=tuple(engines))
  File "/app/infinity_emb/engine.py", line 64, in from_args
    engine = cls(**engine_args.to_dict(), _show_deprecation_warning=False)
  File "/app/infinity_emb/engine.py", line 50, in __init__
    self._model, self._min_inference_t, self._max_inference_t = select_model(
  File "/app/infinity_emb/inference/select_model.py", line 65, in select_model
    loaded_engine = unloaded_engine.value(engine_args=engine_args)
  File "/app/infinity_emb/transformer/embedder/sentence_transformer.py", line 55, in __init__
    super().__init__(
  File "/app/.venv/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 287, in __init__
    modules = self._load_sbert_model(
  File "/app/.venv/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 1487, in _load_sbert_model
    module = Transformer(model_name_or_path, cache_dir=cache_folder, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py", line 54, in __init__
    self._load_model(model_name_or_path, config, cache_dir, **model_args)
  File "/app/.venv/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py", line 85, in _load_model
    self.auto_model = AutoModel.from_pretrained(
  File "/app/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 550, in from_pretrained
    model_class = get_class_from_dynamic_module(
  File "/app/.venv/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 501, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
  File "/app/.venv/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 326, in get_cached_module_file
    modules_needed = check_imports(resolved_module_file)
  File "/app/.venv/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 181, in check_imports
    raise ImportError(
ImportError: This modeling file requires the following packages that were not found in your environment: flash_attn. Run `pip install flash_attn`

Expected behavior
Expected: no error
Actual: error
J

Thank you.
Can this be added to the default docker image?
I prefer to stick with 'official' releases.

#17 #306 #

Adding it to the docker image results in a ~130mb increase.

Fixed. FA2 is not even used efficently in this implementation, they could have also used F.sdpa.
I created a special release with FA2 wheels installed in the docker file. Does this work?
docker run -it michaelf34/infinity:0.0.52-fa  v2 --port 8000 --model-id Alibaba-NLP/gte-Qwen2-1.5B-instruct

@michaelfeil you rock!
1/ IMO, 134mb is nothing - I would recommend for everyone (including yourself) to keep it simple, and have only one Docker as much as possible (wait until you find dependencies that are in the GBs for having multiple artifacts
2/ else - could you please create a ':latest-fa' ?
3/ I'm giving it a go now, and will shortly report back.

Problem is the automated CI. Flash-attn does not support pep517 builds (poetry) & pip install flash_attn often fails -> wheel is directly hacked into this docker version.

Just tested it, confirmed it solves the issue. Thank you so much.
Please consider (2) - create a :latest-fa release.


Lap BTLx Feature // T-Butt & L-Butt

This PR introduces a new BTLxLap processing feature and Updates the T-Butt and L-Butt joints so that they follow the new system of joint definition (ie the use of the alt constructors of the JackRafterCut and Lap classes).
This means that the ButtJoint module can eventually be removed.
Also this PR fixes a bug in the calculations of angle and inclination of the JackRafterCut.
What type of change is this?

[X] Bug fix in a backwards-compatible manner.
[X] New feature in a backwards-compatible manner.
[ ] Breaking change: bug fix or new feature that involve incompatible API changes.
[ ] Other (e.g. doc update, configuration, etc)

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[X] I added a line to the CHANGELOG.md file in the Unreleased section under the most fitting heading (e.g. Added, Changed, Removed).
[X] I ran all tests on my computer and it's all green (i.e. invoke test).
[X] I ran lint on my computer and there are no errors (i.e. invoke lint).
[X] I added new functions/classes and made them available on a second-level import, e.g. compas_timber.datastructures.Beam.
[X] I have added tests that prove my fix is effective or that my feature works.
[X] I have added necessary documentation (if appropriate)

I haven't removed the ButtJoint module, although it has been completely removed from the T-Butt and L-Butt joints. Let me know if you would like to do this in this PR or in a separate one.

Adjustments based on the comments from @chenkasirer  done 👍

I worry that we are implementing overly-simplified geometric implementations of the BTLx Processings. I would prefer that we make these more complete so that we don't have to go back later. This is particularly true for the Lap Processing, which will lose functionality with this. I know this is currently only for the Butt Joints, but it's worth considering if we should just implement fully immediately.

I agree with your point, and I think in this case it can be resolved by implementing an additional alternative constructor. But in my opinion in some of the joint definitions this simplification could be valuable. Let me know what you guys think @chenkasirer  @obucklin

@obucklin After discussing with @chenkasirer, we agreed it would be wise to split the Lap Processing development into two PRs. The main reason is that this PR’s primary goal is to update the old implementations, at least when it comes to machining processes. Since lap joints currently only allow for geometric representation without the ability to extract machining parameters, we can accept that the alternative constructor introduced here is mainly intended for butt joints.
In a follow up PR, we can develop a new constructor that allows for the generation of a more advanced Lap processing, that would handle lap joints and would be able to extract the necessary machining parameters. (And also have the lap be represented as a volume rather than as a box).


Auto-hide taskbar issue.

I have my taskbar set to auto hide, and I have a secondary monitor that is above my primary monitor. When I connect my secondary monitor the taskbar hides as intended, but when it hide the transparent/blurred bar falls onto my primary monitor. This leaves a burred bar on the top part of the primary screen, covering things like Chrome tabs and the min, max, and close buttons on my windows.
My work around has been to open TaskbarX and restart it. This issue occurs every time I reconnect my HDMI cable to my laptop.

You should close your duplicated issues, #526 and #527 which are basically the same as this one (#528).

You should close your duplicated issues, #526 and #527 which are basically the same as this one (#528).

Odd, I didn't realize this posted multiple times

Odd, I didn't realize this posted multiple times

@Aharper9917 No problem, i closed them for you. I get back to you when i have some time...

@Aharper9917 No problem, i closed them for you. I get back to you when i have some time...

To be able to focus on major/current bugs I had to close all issues. The issues section has gotten very messy to a point that I couldn't follow it anymore. Most issues are duplicates, very old or are already fixed/solved. Feel free to reopen this issue if you still have this issue on the latest version of TaskbarX to get my attention. All feature requests have been put in a list here: 💡| Feature requests #870
I hope you understand ❤


Use a keyring directory for RAUC and add a devel certificate

Adding a devel certificate that is deployed (but not enabled) on all LXA TAC images makes it easier for newcomers to build and install their first custom bundle.
The install-time detection of which certificates to trust is required for our planned release channels for official bundles:
First the bundles are built and signed with the rauc-devel.key.pem key and tested on hardware.
If the tests are passed the existing bundle will be automatically re-signed with the nightly release key.
If a new stable release is to be released an existing nightly bundle that has been manually tested is manually re-signed with the stable release key.
All of these bundles are deployed with the same /etc/rauc/certificates-* but a stable bundle should only be able to install other stable bundles, so we need to detect the release channel the bundle belongs to at install-time.

The PR now contains the newly generated certificates and should be ready for merging.


[1929] Switch request access URL from ui to frontend

Context
Requesting access has been ported from the old C# application to the new Rails application, but the frontend still links to the old one and needs to be updated.
Changes proposed in this pull request

Update the request access link to the manage-courses-frontend implementation rather than the C# implementation

Guidance to review

I added a do not merge to this but then removed it because I think we can safely ship this now; I've tested access requests just now and they work great in the new frontend.
@defong Nah no need to test this


Fix incorrect detection of unclosed  tag

In the current master branch, entering an <aside> tag causes the detection of an unclosed <a> tag (resolving issue #230).
This PR applies the code changes from #678 to the current master branch, rather than attempting to fast forward that PR through ~3 years of changes.
On merging, this should resolve #230 and close #678.

We can probably apply this to markdownLegacy as well since it shouldn't affect rendering of any brews.

Alright, LGTM.


Be able to use url helpers in components without prefix

From #16

I can't use URL's inside components without prefixing them with main_app

We should be able to use url helpers without any prefix, just like a normal app.
Probably has something to do with isolate_namespace.

Closed in #18

Hey @Fodoj it is now possible to use url helpers inside components without having to add the prefix main_app
Cheers!


feature request: print a error message including the failed create materialized view statement when recovery

Motivation: it would be helpful both for us and users:

For users: we will pop-up an alter to user on our Cloud UI to inform him about the failure
For us: the full create materialized view will help us to diagnose the problem

We might need to also consider https://github.com/risingwavelabs/risingwave/issues/12673

@kwannoel could you please help to di this after you finish recoverable background ddl?


Sign metadata when configured to do so

Also open up the metadata generation API for customization.

LGTM Can you open the same PR for master? Thanks
CC @mmoayyed


Details about GPU run times in Table 2 of the paper

Hello @cypw. First of all, thanks to your amazing research. I had a question regarding Table 2 in your paper. This is not an issue but rather a follow up question. You reported CPU inference times of Resnet-50 for various values of alpha. I'm wondering if you had observed a similar trend of decreasing run times on GPU ?
I've replicated your implementation in Tensorflow and here are my inference times on 2080Ti for Resnet-50 imagenet model. Image size 224x224x3

alpha
GPU runtime (ms)

0
13.7

0.125
22.8

0.25
23.1

0.5
23.8

0.75
22.6

From the above table, octave convolution performance that I get is worse on GPU.
Did you see a similar performance on GPU from your side? Does this suggest that Octave convolution improvement is more suitable for CPU rather than GPU?
Thank you !!

According to https://github.com/koshian2/OctConv-TFKeras experiment he observed good speed up on both GPU and CPU. I don't have time to optimize CUDA version but I think at least it will be significantly better if you try to compile with AutoTVM rather than directly run with TF.

Thanks @antinucleon. I took a look at his experiment with Cifar-10 Wide ResNet (32x32x3) which is why I got confused when I wasn't observing such trend. I'm using TF 1.13 and cuda-10.0. Thank you for pointing out TVM. I will look into it for a better comparison.

For AutoTVM you can start from: https://docs.tvm.ai/tutorials/autotvm/tune_relay_cuda.html#sphx-glr-tutorials-autotvm-tune-relay-cuda-py

It is relevant to know if the paper relies on TVM to make it faster.

@miguelvr TVM is an automatic way to generate fast kernels. We can also hand write code to achieve theoretical speed up too. So actually it is not about DL framework / compiler, it is about how to get fast code for kernels which is not well optimized by vendor lib such as MKLDNN, CUDNN etc.

@miguelvr TVM is an automatic way to generate fast kernels. We can also hand write code to achieve theoretical speed up too. So actually it is not about DL framework / compiler, it is about how to get fast code for kernels which is not well optimized by vendor lib today (Usually written by perf experts) such as MKLDNN, CUDNN etc.

I know what it is. I am facing this issue on my own implementation of OctConv in pytorch.
But my benchmarks show that the OctConv2d op on its own is much faster (4-6x) than a regular convolution, but the the OctResNet models are about 3x slower than the vanilla counterpart.

@miguelvr There are many reasons to effect end to end perf with a framework aside of operator quality, eg dynamic graph / static graph, multi-stream executor quality and so on. When we have bandwidth we can tune with TVM CUDA to get desirable speed (and kernels).


Do we have a consensus about the wallet migration strategy?

Hello everyone,
I would like to ask you some questions:

How do you see the structure of a transaction when we look at a block explorer? Would you say that it would be like in the figure below?

where

-In the Seraphis-0-0-16 draft page 15, you say:

Linking tags will have the form (k b,recipient /(k a,sender + k a,recipient )) ∗ J. Even if a transaction author sends many e-notes to the same spend key K s , and all of those e-notes are spent, the author cannot use linear algebra on the resulting linking tags to associate those linking tags with the e-notes they sent out. This avoids the ‘linearity’ problem for fixed-base-point linking tag constructions noted by the CryptoNote whitepaper [37], which breaks sender-receiver anonymity. [[[formalize better? proof?]]]

Seraphis depends a lot on division operations, up to now, we did not do it that frequently so are there any known issues or special care when massively dealing with these operations? Did you start formalizing this proof?

I would consider a very very simple wallet with the functions below:

Open/Save wallet file with private keys.
Scan the blockchain for e-notes (already done)
Show amounts
Make a transaction (already done)

At this point, it should not be too hard (in terms of creativity) to create this simple wallet that would read/write from the blockchain and have these basic functionalities (as you have already written 10k+ LOC with the core of it).
Although, we cannot read/write from the blockchain as we dont have the structure to save the data on it yet! Can we start working on a very very primitive blockchain (just a reference file or so) with the blocks containing the reference transactions so we have a standard to work on? Like scanning the txs, showing the amounts, adding new transactions to this file and so on?

Do we have a consensus about the desired approach ?

I'm in favor of this double wallet. We start creating a new wallet_seraphis (which would be the similar of wallet2) and we add the necessary functions (that koe is working on) to make a legacy/seraphis transaction on wallet2.
What do you think?

I think it is also important to work in parallel in implementing the necessary structures to store the data in an actual blockchain style so we can have a testnet as soon as possible. If we agree with that structure shown in figures 1 and 2, we have to basically modify the file so the DB schema would account for that, right?

SpTxSupplementV1 also includes the enote ephemeral pubkeys.

Seraphis depends a lot on division operations, up to now, we did not do it that frequently so are there any known issues or special care when massively dealing with these operations? Did you start formalizing this proof?

Not that I know of, and no I haven't started. Up to this point I have just assumed that inversion works without issue. You could look at the Lelantus-Spark paper to see if they have any special remarks, since their linking tag structure is very similar.

I would consider a very very simple wallet with the functions below:

Keep in mind that all the stuff I have written is not persistent state - no saving to file, just pure RAM mockups.

Can we start working on a very very primitive blockchain (just a reference file or so) with the blocks containing the reference transactions so we have a standard to work on?

You could fairly easily adapt/upgrade the MockLedgerContext to do this.

I'm in favor of this double wallet. We start creating a new wallet_seraphis (which would be the similar of wallet2) and we add the necessary functions (that koe is working on) to make a legacy/seraphis transaction on wallet2.

A seraphis wallet should be able to exist completely independent of wallet2 (i.e. users shouldn't need wallet2 at all), since I have built up legacy balance recovery into the seraphis library. There may need to be a new release where tons of old code touching wallet2 is deleted. An intermediate step would be refactoring out parts of wallet2 that need to be reused in a seraphis wallet.

I think it is also important to work in parallel in implementing the necessary structures to store the data in an actual blockchain style so we can have a testnet as soon as possible.

I agree that blockchain/db stuff is largely independent of wallet stuff, so those tasks can be worked on in parallel.

About the first part of your issue, with the diagram about a possible structure of a Seraphis transaction as shown e.g. in a block explorer: As far as I understand, that's basically the question how to serialize Seraphis transactions to store them in places like the blockchain and wallet cache files, so I will comment on that:
Maybe you, @DangerousFreedom1984 , know all what I write in the following already, or maybe not, and other readers might not as well, so there:
I worked a fair bit with object serialization and deserialization as it's implemented in the existing Monero code base. E.g. for the MMS I defined the structure for a fourth wallet file, "walletname.mms" from the ground up, and I watched jberman over the shoulder when he extended the transaction serialization to include viewtags.
Thus I can say that the way it's usually done if you have data in RAM that you have to make permanent i.e. store into files so you can re-create it later is "going forward". I mean with this that first you don't care at all how things will look when they are stored; your starting point is all the classes that describe that the data you want to store, so no "going backwards". You add code to serialize and deserialize, and whatever kind of structure that will create in the files will be ok.
Let's get concrete to illustrate. @UkoeHB 's Seraphis library currently defines a SpTxSquashedV1 struct as the object to manage Seraphis transactions in RAM. (I removed things like comments and methods to simplify.)
struct SpTxSquashedV1 final
{
    std::vector<LegacyEnoteImageV2> m_legacy_input_images;
    std::vector<SpEnoteImageV1> m_sp_input_images;
    std::vector<SpEnoteV1> m_outputs;
    SpBalanceProofV1 m_balance_proof;
    std::vector<LegacyRingSignatureV3> m_legacy_ring_signatures;
    std::vector<SpImageProofV1> m_sp_image_proofs;
    std::vector<SpMembershipProofV1> m_sp_membership_proofs;
    SpTxSupplementV1 m_tx_supplement;
    DiscretizedFee m_tx_fee;
    SemanticRulesVersion m_tx_semantic_rules_version;
}

Le'ts assume that the order of the members of this structure makes some sense, so we will just use the same order to write them into the file. This is how that structure might look with the necessary code added to use the same epee binary serialization system that is used e.g. for the existing Monero transactions:
struct SpTxSquashedV1 final
{
    std::vector<LegacyEnoteImageV2> m_legacy_input_images;
    std::vector<SpEnoteImageV1> m_sp_input_images;
    std::vector<SpEnoteV1> m_outputs;
    SpBalanceProofV1 m_balance_proof;
    std::vector<LegacyRingSignatureV3> m_legacy_ring_signatures;
    std::vector<SpImageProofV1> m_sp_image_proofs;
    std::vector<SpMembershipProofV1> m_sp_membership_proofs;
    SpTxSupplementV1 m_tx_supplement;
    DiscretizedFee m_tx_fee;
    SemanticRulesVersion m_tx_semantic_rules_version;

    BEGIN_SERIALIZE_OBJECT()
      VERSION_FIELD(0)
      FIELD(m_legacy_input_images)
      FIELD(m_sp_input_images)
      FIELD(m_outputs)
      FIELD(m_balance_proof)
      FIELD(m_legacy_ring_signatures)
      FIELD(m_sp_image_proofs)
      FIELD(m_sp_membership_proofs)
      FIELD(m_tx_supplement)
      FIELD(m_tx_fee)
      FIELD(m_tx_semantic_rules_version)
    END_SERIALIZE()
}

(Really only for illustrations. There are other macros like e.g. VARINT_FIELD that we may have to use for some members, and maybe some things like the fee are only "cached" in RAM for easier handling and don't belong stored permanently into any file.)
Most of these members are itself complex objects with again a number of members. They will all get such code sections to serialize them as well, and the recursive execution of all this will then decide what gets written to file and "how it looks" there.

I opened a new issue #11 about question no 5, the question about a blockchain file that can hold Seraphis transactions.

@rbrunner7 I'd like to actively discourage making changes to any of the existing files in seraphis_lib. For serialization, it would be better to see separate ObjName_serializable objects, and separate functions serialize(ObjName, [serialization config], ObjName_serializable &out) deserialize(ObjName_serializable, [deserialization config], &ObjName_out). I realize an intermediate step like this adds some allocation overhead, but imo it is very worth it to maintain clean code at scale.

Beside epee serialization there is a second system of serialization provided by the Boost library that is - or at least was - also used in the Monero codebase.
Quite some time ago a number of files, e.g. ".mms" files, were switched from using Boost serialization to use epee serialization, if I remember correctly because there had been some bugs and exploits with Boost serialization, and epee serialization was considered more robust and more trustworthy.
But, and that's why I mention this here, with Boost serialization you do not write the serialization code directly into the classes, but you write separate templated classes.
You can see this in my message_store.h file with declarations that support both systems. The Boost classes start here.
So maybe for Seraphis and the Seraphis wallet a re-evaluation of the pros and contras of those two serialization systems would be in order, where Boost would have a big "pro" on its side to seamlessly support your proposal to keep any serialization code out of the Seraphis library and the advantages that this can bring.

Wouldn't even serializing a single tx lead to construction of possibly dozens of intermediate serialization objects for that "going down"?

It shouldn't be that bad...

With standard epee based macros the generated code is able to both serialize and deserialize. Would that be achievable as well using your proposed separate serializing classes, or would the code basically have to double?

The ObjName_serializable would be direct serialize/deserialize like normal. On top there would be serialize/deserialize glue code to translate into the actual objects. We are already doing that for e.g. Bulletproofs, except it is a huge pain to actually find the code that does that.
Since I already know how this should work, maybe I should just add it to my todo.

I just realized that my "meta transaction" approach to serialize and deserialize transactions of any kind (see issue #7) probably won't work if legacy transactions use the epee system and Seraphis transactions use Boost, which could develoop into a serious problem.

I just realized that my "meta transaction" approach to serialize and deserialize transactions of any kind (see issue https://github.com/seraphis-migration/wallet3/issues/7) probably won't work if legacy transactions use the epee system and Seraphis transactions use Boost, which could develoop into a serious problem.

If epee serialization ever gets replaced (there are now three proposals: perfect-daemon's, vtnerd's, and jeffro256's), we will probably end up using that.

We are already doing that for e.g. Bulletproofs

If the linked code implements the special serialization classes, where are the "true" Bulletproof classes then?

If epee serialization ever gets replaced

I did not follow developments closely: Do those systems allow for the desired de-coupling of class declarations and serialization code?
And well, it really depends when any replacement will actually happen. Seems to me that serious implementation work on the Seraphis wallet could already start in 2 months or so, with considerable manpower, and ideally the serialization question would get decided early.

If the linked code implements the special serialization classes, where are the "true" Bulletproof classes then?

This is an example of serialization spaghetti. The Bulletproof struct has C++ members for its C++ representation, and serialization members for its serialization representation, and those are not 1:1.

I did not follow developments closely: Do those systems allow for the desired de-coupling of class declarations and serialization code?

No idea, maybe @j-berman or @vtnerd can comment.

And well, it really depends when any replacement will actually happen. Seems to me that serious implementation work on the Seraphis wallet could already start in 2 months or so, with considerable manpower, and ideally the serialization question would get decided early.

I can do a proof-of-concept for serialization within 2 weeks (as soon as I get the demo for legacy input integration working - getting closer).

This is an example of serialization spaghetti.

I think you coined a nice new term here :)

I can do a proof-of-concept for serialization within 2 weeks

That's a very attractive offer; you are probably by far the best person to try to prove that this approach works, as we probably should hope, or to find out that it turns into spaghetti code as well, just in a different way - which would be unfortunate of course, a "back to the drawing board" moment so to say.

If epee serialization ever gets replaced (there are now three proposals: perfect-daemon's, vtnerd's, and jeffro256's), we will probably end up using that.

Follow up for anyone reading: these proposed serialization updates are for the p2p network, not for the blockchain, so they aren't relevant here.

Thank you for the discussion. I would suggest the serialization work package to be part of the wallet development as we could replace the blockchain stuff by only a file (only for simplification) if we know the correct formats to read/write the bytes from the bc (file) initially.

Today I was investigating the subject of serialization a bit more.
I showed here earlier in this issue that in the simplest possible case you can get classes to serialize by adding some quite simple lines, starting with BEGIN_SERIALIZE_OBJECT(), then having FIELD for every class member, and ending with END_SERIALIZE().
I think that if (and that's probably a big "if") things could be arranged in a way that makes it possible that serialization code stays so simple, not much would speak against having it directly in the classes as shown, and thus as part of the Seraphis library, which would be a simple and easy configuration.
So I went and tried to find out what's up with all that terrible "spaghetti" serialization code for cryptonote::transaction, rctSigBase and so on. See in the code e.g. here or here. What is at the core of the complexity of that code, what are the reasons it's complex? And would there have been a better way to do it? E.g. would it have helped to extend the functionality of that binary_archive serialization mechanism?
My (somewhat naive) hope was that after reaching understanding I would indeed find out that there are better ways to do it, or at least ways to prevent code degeneration to such an extent, and keep everything so simple that we could afford to keep the code right in the Seraphis library after all.
Unfortunately, I did not get yet very far. Biggest problem are things I don't understand yet. There is for example PR #1067 from @moneromooo-monero titled rct: rework serialization to avoid storing vector sizes. It complicated things considerably, but sadly no idea what was the problem with storing vector sizes ...
If neither @DangerousFreedom1984 nor @j-berman want to pick up at this point and investigate further, I will probably wait how @UkoeHB 's approach looks that mirrors every relevant Seraphis library object with a serializable object. I confess I am quite sceptical that turns out pretty and performant, but let's see.

@rbrunner7 I didn't really understand your question. Are you asking about the file structure where the serialization will happen? Or are you asking about the tasks to do?
I believe we could have a similar approach, as you stated, in the corresponding file in Seraphis. Maybe this one ? But we may want to discuss it in another thread.

@DangerousFreedom1984 , it should become clear if you read the whole discussion between me and @UkoeHB. Most important and consequential statement, as I see it, is this one: If you want to avoid any code regarding serialization in the Seraphis library, things could get tricky.

@rbrunner7 Thanks for pointing. Yeah, I agree with that approach from @UkoeHB  and I believe that his answer was not to touch the actual files or at least not change them drastically. But I guess we can insert other files inside the src/seraphis folder to make the serialization and have the proper objects as suggested. But sorry, I might have not understood still.

@UkoeHB recently completed a demo / proof-of-concept for the serialization of Seraphis transactions. The principal commit where you can see how it works is this one.
I read the code which is quite simple, clean, not hard to understand, and indeed smaller and less complicated than I myself would have predicted when I first had the general idea explained: Don't make the seraphis_lib objects themselves serializable, but make a new, second set of objects which are, and provide ways to convert back and forth between those two "families" of objects.
Still, I am also more confused than ever about the motivation for this exercise: What does this want to achieve, at its core, on a fundamental level? Which "bad scenarios" does this protect the seraphis_lib against? Which potential problems does it want to prevent, now and in the future?
If you drill down and look beyond those two families of objects, the 50 or so methods needed to convert between their members, and all the other "infrastructure" and go to the place where the actual serialization happens, in the file erialization_demo_types.h, you see code parts like the following one:
BEGIN_SERIALIZE()
    FIELD(A)
    FIELD(A1)
    FIELD(B)
    FIELD(r1)
    FIELD(s1)
    FIELD(d1)
    FIELD(L)
    FIELD(R)
END_SERIALIZE()

And, well, this code looks like I claimed it would look: Trivial.
I am really looking forward to clarification and explanation from @UkoeHB because it must be that I overlook something important. Currently this all looks to me like one pretty elaborate exercise to keep absolutely trivial code parts like the one showed out of the core of the Seraphis library.

The goal is to make trivial what otherwise would not be. Take the deserialization of CLSAGs and bulletproofs as the canonical example of a mess.

I have a question that may clear things up for me: Do you think that 2 or 3 years after the initial Seraphis hardfork, with maybe 2 or 3 semantic tweaks in place, with pruning fully implemented, and maybe other things I can't come up with right now, the full serialization code will also look quite complicated? Maybe not a complete "mess" like the examples from the existing code thant you linked, but still quite complex?

Do you think that 2 or 3 years after the initial Seraphis hardfork, with maybe 2 or 3 semantic tweaks in place, with pruning fully implemented, and maybe other things I can't come up with right now, the full serialization code will also look quite complicated?

It should not be necessary to make the serialization more complex. More lines - yes, more complex - no.

semantic rule changes: use a version number that informs post-deserialization semantics checkers
structure changes: use new structs to represent new structures
overloading a space with multiple potential structures: use a variant of types

This solution would introduce a little bit of spaghetti: calls to is_pruned(ser_SpTxSquashedV1)

Maybe all that is needed to avoid that is to include an isPruned boolean into the serialized object, written into the stream early, so you can read it early later on and react accordingly?
And I am thankful for that pruning code sketch, because it shows how the introduction of pruning forces a split into two classes, and the "upper" class, the full serializable transaction, gets a sub-object. You probably would not want this to happen to the "real" transaction class, you want to keep that "flat".

Maybe all that is needed to avoid that is to include an is_pruned boolean into the serialized object, written into the stream early, so you can read it early later on and react accordingly?

I suppose that's another option. Tbh I don't know much about how pruned txs are handled, so it's hard for me to come up with the best solution. All I can say is: there must be solutions that minimize spaghet!


Delayed connection to opened ports on self hosted pwd

I run two simple nginx server in a docker instance and open port 80 for both.
The connection to nginx A works and l2 shows "Proxying http connection to..." in its logs.
When I try to connect nginx B, I get the content of the website from nginx A and l2 doesn't show any new logs.
When I refresh the site of nginx B after about a minute, I get the correct contents and l2 shows the connection. Then I have to wait again to get a new connection. No matter which nginx website I try to reload.
Even multiple reloads of one nginx only show one connection in l2. Then I have to wait again to get a new connection.
Does l2 have any kind of cache or time constraints for consecutive connections?

@z3ky forgot to ask. Can you please provide a bit more information about the environment you're using in PWD? Are you performing this test in your local machine? Which environment (OS, version, etc) are you working with

I tested on ubuntu 20.04 and macOs.
On ubuntu, I changed the host in config.go to the server's DNS. I also tried to wget the opened port URLs from within instances. Same issues. Only the first call returns the correct data and a l2 log. The wget in the second instance returns the data from the other one and no l2 log. After about a minute or maybe less, I get connection to l2 again.
On macOs, I didn't changed any file. Here, the connection to direct.localhost only works in Chrome browser, but with the same issue.
To reproduce:

docker-compose up
create 2 instances
instance 1

mkdir data && echo test_1 > data/index.html
docker run --name some-nginx -p 8080:80 -v /root/data:/usr/share/nginx/html:ro -d nginx
open port 8080

instance 2

mkdir data && echo test_2 > data/index.html
docker run --name some-nginx -p 8090:80 -v /root/data:/usr/share/nginx/html:ro -d nginx
open port 8090

Are you connecting to the l2 directly or through haproxy?
I'm testing this locally and I can't reproduce. Here are the steps I'm following:
1 - docker-compose up -d
2 - Create  instance 1 and run docker run -d -p 80:80 nginx
3 - Create  instance 2 and run docker run -d -p 80:80 nginx
4 - Test with curl ip<local_ip>-<session_id>.direct.localhost on both instances and not getting any connection issues.

Let the servers show different contents. The curl the first, then the second and the the first again.

Can you try adding option http-server-close in the haproxy.cfg file under the defaults section and try again?

option http-server-close fixes the issue. now, all requests go through haproxy to l2.


The tests are temporarily disabled

:warning: Testing-farm support is disabled. We are moving to a new test runner. :warning:
The cluster with the old testing-farm runner has died, it will never work again and we are finalizing the support for a new one.

[x] We have prepared (#875) implementation of the new API of the testing-farm.
[x] Miro V. has been working hard on the new API. You can expect full tmt support so be patient, please.
[ ] Jirka is finalizing our part and making sure the connection works smoothly (#875).

We are sorry about that but give us some time, please.
More information about that can be found in #802 or in packit/packit#1078.
You will now get Testing farm is temporarily disabled. result until the new testing-farm is ready. We set the result status to success to not mark your PRs as failed. The neutral state is not possible with our current procedure how we set the status (#760).

Miro V. is working hard on the new API but it's not ready yet.

I guess that part has been done, right? @thrix?

Miro V. is working hard on the new API but it's not ready yet.

I guess that part has been done, right? @thrix?

Yes, I'm working on implementing it on our side.

Yes, I'm working on implementing it on our side.

@psss I've updated the description to be fair..;)

@psss I've updated the description to be fair..;)

Thanks for the update :)

Thanks for the update :)

This seems to be already in production, right?

yes, it's been in production since this week


More functionalities related to cart state management

Hi,
Thank you for providing this project as an open-source for us to use. Do you use this cart state management in your product apps as well? Asking this to know whether it is a good approach for the production app.
Also, can you do us a favor and provide a code snippet for implementing quantity update functionality and its synch with local storage (in this case it is shared pref)?

What do you mean by cart state management?
And for the second part of your question,  the api that I use doesn't have a service / endpoint to update quantity.

Cart state management means shopping cart in short which includes adding, incrementing, decrementing, removing, etc.
Would be great if you include the updating as well.


fix: allow text-selecting in dialogs & reset cursor

What does it do ?

Enable text selection in keyboard modal
Convert cursor inside modal from crosshair to text

Do we really want that.. it affects all dialogs.

Do we really want that.. it affects all dialogs.

Which is good, no? I suspect the fact we disallow text selection is a regression, not a feature.

Which is good, no? I suspect the fact we disallow text selection is a regression, not a feature.

Do we really want that.. it affects all dialogs.

Right now the cursor and text selection is disabled in the body tag affecting the whole app.

Do we really want that.. it affects all dialogs.

Right now the cursor and text selection is disabled in the body tag affecting the whole app.

Here is what I mean... the cursor shouldn't apply everywhere (if at all)

Here is what I mean... the cursor shouldn't apply everywhere (if at all)

Here is what I mean... the cursor shouldn't apply everywhere (if at all)

@lipis I have removed cursor:text now

Here is what I mean... the cursor shouldn't apply everywhere (if at all)

@lipis I have removed cursor:text now

Merge?

Merge?


hipchat: upgrade to v3.3.6-202

Thank you for the contribution. It was merged directly as 89161e7b82c4a2ee6598a4b2aefaa98a43e3b48f to keep commit history cleaner. Your contribution is still credited to you.


heigh ho

heigh_ho.webm
cc @a-moreira @Davidson-Souza

to allow plebs to aggregate the hashrate of their multiple ASIC devices into a single solo mining "lottery ticket".

using their own node? Like we've discussed prev offline?

yes, each miner deploys their own bitcoind as Template Provider
currently it's https://github.com/Sjors/bitcoin/tree/2023/11/sv2-poll
while we mentioned GBT on the previous discussion, I don't think RPCs are really necessary, since Sjors' fork is able to exchange SV2 messages

a correction on the message above:
the initial development cycles of pleblottery are based on GBT
Sv2 TDP alternative is on the roadmap


Support reading KOF files with locations

Add functionality to use https://github.com/norwegian-geotechnical-institute/kof-parser to merge locations from KOF files to an SGF file.

The kof-parser is now published on PyPi as well: https://pypi.org/project/kof-parser/


repository: (Mostly) index-related cleanups

What does this PR change? What problem does it solve?
This PR simplifies some of the index code and removes several functions that are only used for testing. In addition, it slightly improves the separation between the index and repository code. These changes are nearly sufficient to allow splitting the index from the repository package.
Was the change previously discussed in an issue or on the forum?
No.
Checklist

[x] I have read the contribution guidelines.
[x] I have enabled maintainer edits.
[ ] I have added tests for all code changes.
[ ] I have added documentation for relevant changes (in the manual).
[ ] There's a new file in changelog/unreleased/ that describes the changes for our users (see template).
[x] I have run gofmt on the code in all commits.
[x] All commit messages are formatted in the same style as the other commits in the repo.
[x] I'm done! This pull request is ready for review.

repository.Loader is also unused.

So, then let's get this merged.


Add toggle for referential integrity

This adds a configuration option called disable_referential_integrity which will put everything in the seeds file into a block where the database connection's referential integrity is disabled using a method called on that connection.
This allows more flexibility when seeding the database.  We ran into an issue where seeding the db with referential integrity enabled caused constraint failures on some many-to-many tables.  Seeding without worrying about referential integrity up front (similar to what a mysql database dump file does) got around this.

Thank you so much!!
I added a few comments, the code looks good, but I think we could simplify it a bit.
Could you add a test for this? And also squash the commits into one, including an explicit title (less than 72 chars for the the first line) and a quick description in the commit body?

Thanks pjambet, I will work on the changes you suggested and push them up soon.  I'll let you know if I have any questions.


WinAppDriver cannot find elements (neither can inspect)

We have run into a strange problem where some elements cannot be found by WinAppDriver, yet they appear on the screen.  In debugging the issue, we have determined that they are not properly being added to the UI tree in inspect either.
There is a listview in our app that is on the landing screen.  In our automated tests, the listview is almost never able to be found by WinAppDriver, which then causes the tests to fail.
Opening the app manually and viewing it with inspect shows some 'custom' nodes that do not have the children that we expect (although we can clearly see them on the screen). After navigating away from this screen and returning, the elements are added as expected, but this is not an acceptable workflow to ensure automation works for our testing.
In the image below, the left side shows the tree upon first arrival of the listview.  The list should be within the 'custom' node in red.  On the right side shows the tree after returning.  The list is there along with several other nodes including the one listview item.

However, when using inspect, even on first arrival, we CAN use the mouse to select the listview.  When we do this, the item details are shown including our assigned automation id and the element information.  Except it does not auto-navigate to anywhere on the treeview on the left side because evidently there is no parent in that tree.  It seems to be an orphaned element.

Unfortunately, I cannot post screenshots of the application as it contains sensitive product information (and I have blocked out such text).
We aren't sure this is a WinAppDriver issue, since the problem is also perceived in inspect.  But we don't know who else to bring this to.  Any advice/help would be greatly appreciated.
Windows 10 1903 build 18362.329
WinAppDriver v <IP_ADDRESS>

Dump the actual XML rather than relying in inspect.
Something like the code below inserted appropriately wilt give you the various xml tag names
GuiRootAccess.FindWindowSession(window, 1); gets the session corresponding the title of the Window containing the control of interest
Once you have the xml you can determine the tags and search keys needed for searchs by XPATH to navigate item by item to the control of interest.
public static void SaveSourceToFile(string window,
string strFile)
{
WindowsDriver session;
bool bExists;
session= GuiRootAccess.FindWindowSession(window, 1);
Assert.IsNotNull(session);
var source = session.PageSource;
Assert.AreNotEqual(source.Length, 0);
bExists = Utilities.OSFileUtilities.DoesFileExist(strFile);
if (bExists)
{
Utilities.OSFileUtilities.DeleteFile(strFile);
}
System.Xml.XmlDocument xmlDoc = new System.Xml.XmlDocument();
System.Xml.XmlTextWriter w = new System.Xml.XmlTextWriter(strFile,
System.Text.Encoding.BigEndianUnicode);
xmlDoc.LoadXml(source);
xmlDoc.WriteTo(w);
w.Close();
}

@DLightstone,
Thanks for the reply.
We have checked the xml page source and we see the exact same issue.  I didn't post that because it seemed redundant, but the page source also does not have the node in the tree.

My wild guess is that Microsoft has not yet implemented a means to correctly identify the control you wish to have identified.
If you play with the TreeView control at all you will quickly discover that
(1) image icons, checkboxes, and node tags are "invisible" (is no xml tags suitable for observation)
(2) node expand/contract  and checkboxes not considered to be a controls ( or subcontrol depending on perspective.)
Finally there is a possibilty that a different set of project build options my expose a different interpretation of the items

Hi @DLightstone,
Thanks again for the reply.  For the case I have posted on, the issue is not consistent.  For many people on my team, the elements all appear correctly, while some others see this issue and cannot run our tests.  One thing we have noticed, although it is anecdotal, is that it seems that machines running Windows 10 1903 seem to see issues like this more commonly than those running 1809.  Again, this is anecdotal.
After installing a build of our app, I tested this behavior 3 times, the first 2 times the elements showed up as expected in the UIAutomation tree.  The third time the elements were not there, but in all 3 cases the application looked exactly the same to the user.

@DGCBio
Not consistent on the same computer suggests a race condition between the application under test and the test procedure or perhaps something related to task scheduling
One thing to consider is that controls can be created dynamically. This I observe to occur with the TreeView control. If this is a possibility capture the XML during the test just before you expect the missing controls to be available.

HI @DLightstone,
We can reproduce the issue manually as well by starting the application and simply viewing the UIAutomation tree with inspect.  Therefore, if there is a race condition, it seems to be within Windows involving adding elements of the application to the UIAutomation tree.
I have already captured the details of the UIAutomation tree when the condition occurs, and when it does not in my original post.
Thanks.

Some updates to add:

This continues to be an important issue for our team.  It is critical we resolve it or it will be a showstopper for us being able to use WinAppDriver for test automation (we have been building our UWP automation around WinAppDriver for the last couple years).

We had a machine (tablet) that was running Windows 10 1809 and not once did we observe this problem (it was our last machine capable of running our automated tests reliably).  This morning it was updated to Windows 10 1903.  After the upgrade, this bug can now be reproduced reliably.  Again, this issue does not happen 100% of the time, but for most of our Windows 10 1903 machines and tablets it happens more often than not.  We had already been harboring suspicion that it was related to some change in the Windows 1903 update, and this seems to confirm it.  Perhaps there was an optimization change in the 1903 update that affected the UI Automation server?   That is a guess.

We have explored:

Altering the value for the control's AutomationProperties.AccessibilityView property.
Ensuring all updates for Windows 10 1903 have been downloaded and installed.
Ensuring all devices are running WinAppDriver <IP_ADDRESS>.
Attempting to debug the issue using the Accessibility Insights tool.
Testing across many devices with different operating system versions.

With respect to capturing XML of dynamically created controls
Perhaps dynamically created controls can be precluded. Perhaps you capture strategy considered the possibility. I was just trying to indicate a potential observe-ability problem.  Inspect and Accessibility Insights will not observe such conditions. They are basically samplers and the sample has to be taken at the appropriate time.
As an example to clarify my intent
If you capture the XML of an application containing a treeview control immediately before and during a treenode relabel you will observe
(1) the absence of an edit control in the XML captured immediately before the relabel
(2) the presence of an edit control in the XML capture created during the relabel.
With respect to possible bugs
I definitely have observed defects related to adding elements to the UIAutomation tree, so this is not something that can be precluded.
Although is not likely to be the same situation with which you are presented. I have a build where TreeItemControls are converted to TreeControls in one project configuration, and properly recognized an a different project configuration.
My bet is Microsoft Bug

Does WinAppDriver use the Accessibility dlls in the Net Framework installation?
I noticed that Windows 1809 includes Net Framework 4.8.3761, and updating to 1903 downgrades the Net Framework version to 4.8.3572.
Could this be the issue? If so, how can I manually update our 1903 tablet to check if the newer Net Framework version would fix the issue? I tried manually installing Net Framework 4.8 from Microsoft, but it just says I already have 4.8 installed and does nothing.

@hassanuz we are hoping for a comment from you or someone on your team.

@DGCBio
Don't hold your breath.
For reasoning, see the book, Crossing the Chasm by Geoffrey Moore
Microsoft's goals are to preempt competition by getting firms to adopt a technology prior to it being fully implemented.
You can see this by looking for the tag "Help Wanted", and evaluating whether it is possible for someone other than a Microsoft employee to be able to provide the help

@DGCBio, can you try contacting me directly at<EMAIL_ADDRESS>There's not much we can do from the WinAppDriver side given the context, but it would be worthwhile looking into seeing why the app isn't correctly attributing the UI elements - if this is a .NET framework issue, then we'd want to contact the owners for that to resolve this.

Was there ever any resolution to this?
I have a similar scenario where I am trying to detect the download bar in Chrome using WinAppDriver.  Chrome is started by Selenium, and the WinAppDriver simply grabs the process.
On the development machine WinAppDriver is able to correctly find the download bar every time.  However, on a fully updated and patched separate machine WinAppDriver is never able to find the download bar.

@jeremeguenther I'd be interested to know also. I am having the same issue with Chrome Extensions.
I have had some success though with passing --force-render-accessibility into chrome on startup.
That's made things render similar to my machine but it is no populating all the automationId properties I expected (for the HTML inside the Chrome popup).


FormModel tests

On this PR:

Update testing-library to @testing-library/...
Remove empty tests
Add tests for the components that are part of the FormModel rendering process.
Move testing dependencies to devDependencies

The remaining failing tests are from the Shex components and custom hooks that were not part of the refactor.

Reviewed the tests - they look good. Adding some fixes to the other tests that are failing so all tests run successfully.


MainApp.java: Edit version to v1.2

This PR must be merged ASAP.
Signed-off-by: jjlee050<EMAIL_ADDRESS>

Pull Request Test Coverage Report for Build 242

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 86.507%

Totals

Change from base Build 233:
0.0%

Covered Lines:
1917

Relevant Lines:
2216

💛  - Coveralls


Joins to user create a weird query

The query with no fields selected is:
SELECT Id,LastModifiedBy.Account.LastModifiedBy.Account.LastModifiedBy.Id,LastModifiedBy.Account.LastModifiedBy.Account.Id,(SELECT Id FROM Users) FROM Account
Also get the "cannot use" warning when trying to select from User.
Fixing this is quite difficult so recording it here to see if it is important to users. If so, please add a comment below.

also handled the thrown error better and clearing the from entity for now


Add k8s ResourceDetector

I need a way to specify pod/namespace/node as otel attributes from env vars
OTEL_RESOURCE_ATTRIBUTES will not work for k8s with dynamic values, because k8s can't map(k=v).join(',') required attributes. It should be separate envs
- name: K8S_POD_NAME
  valueFrom:
    fieldRef:
      fieldPath: metadata.name
- name: K8S_NAMESPACE_NAME
  valueFrom:
    fieldRef:
      fieldPath: metadata.namespace
- name: K8S_NODE_NAME
  valueFrom:
    fieldRef:
      fieldPath: spec.nodeName

I can do it manually, but may be i can get a better solution
// Use default resource
let resource = Resource::from_detectors(Duration::from_secs(0), vec![
    Box::new(SdkProvidedResourceDetector),
    Box::new(EnvResourceDetector::new()),
  ])
  // Merge with custom or write own ResourceDetector
  .merge(&Resource::new(vec![
    opentelemetry_semantic_conventions::resource::K8S_POD_NAME.string(std::env::var("K8S_POD_NAME")?),
    opentelemetry_semantic_conventions::resource::K8S_NAMESPACE_NAME.string("K8S_NAMESPACE_NAME"),
    opentelemetry_semantic_conventions::resource::K8S_NODE_NAME.string("K8S_NODE_NAME"),
  ]));

Hello @cijothomas @sergeyshaykhullin, is this currently being worked on, please?

Hello @cijothomas @sergeyshaykhullin, is this currently being worked on, please?

No. Would you like to contribute?

Yes, I'd like to give it a try. I'm an outrachy applicant and first time contributor to this repo

Submitted a PR for this, would appreciate feedback please @cijothomas


string mappings processor

I would like a way to map string values to int values so that i can use them in prometheus.
I expect maybe that the config would look something similar to this
processors:
  int-mappings:
    value-mappings:
      value-names:
        - "/interfaces/interface/state/admin-status"
        - "/interfaces/interface/state/oper-status"
	string-mappings:
	  - UP: 1
          - DOWN: 0
  bgp-state-mappings:
    value-mappings:
      value-names:
        - "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state"
	  string-mappings:
            - IDLE: 0
            - CONNECT: 1
            - ACTIVE: 2
            - OPENSENT: 3
            - OPENCONFIRM: 4
            - ESTABLISHED: 5

And the result will be like this
# Before
{
  "name": "interface_state",
  "timestamp":<PHONE_NUMBER>370627152,
  "tags": {
    "interface_name": "et-7/3/1",
    "source": "<removed>",
    "subscription-name": "interface_state",
  },
  "values": {
    "/interfaces/interface/state/admin-status": "UP",
    "/interfaces/interface/state/oper-status": "DOWN"
  }
}

# After
{
  "name": "interface_state",
  "timestamp":<PHONE_NUMBER>370627152,
  "tags": {
    "interface_name": "et-7/3/1",
    "source": "<removed>",
    "subscription-name": "interface_state",
  },
  "values": {
    "/interfaces/interface/state/admin-status": "1",
    "/interfaces/interface/state/oper-status": "0"
  }
}

# Before
{
  "name": "bgp_state",
  "timestamp":<PHONE_NUMBER>515931016,
  "tags": {
    "neighbor_neighbor-address": "<removed>",
    "network-instance_instance-name": "master",
    "source": "<removed>",
    "subscription-name": "bgp_state",
  },
  "values": {
    "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state": "CONNECT"
  }
}
{
  "name": "bgp_state",
  "timestamp":<PHONE_NUMBER>557577400,
  "tags": {
    "neighbor_neighbor-address": "<removed>",
    "network-instance_instance-name": "<removed>",
    "source": "<removed>",
    "subscription-name": "bgp_state",
  },
  "values": {
    "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state": "ESTABLISHED"
  }
}

# After
{
  "name": "bgp_state",
  "timestamp":<PHONE_NUMBER>515931016,
  "tags": {
    "neighbor_neighbor-address": "<removed>",
    "network-instance_instance-name": "master",
    "source": "<removed>",
    "subscription-name": "bgp_state",
  },
  "values": {
    "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state": "1"
  }
}
{
  "name": "bgp_state",
  "timestamp":<PHONE_NUMBER>557577400,
  "tags": {
    "neighbor_neighbor-address": "<removed>",
    "network-instance_instance-name": "<removed>",
    "source": "<removed>",
    "subscription-name": "bgp_state",
  },
  "values": {
    "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state": "5"
  }
}

it's possible to achieve what you need using event-strings processor with transfom replace: https://gnmic.kmrd.dev/user_guide/event_processors/event_strings/
The equivalent config to your suggestion would be:
processors:
  int-mappings:
    event-strings:
      value-names:
        - "/interfaces/interface/state/admin-status"
        - "/interfaces/interface/state/oper-status"
      transforms:
        - replace:
            apply-on: "value"
            old: "UP"
            new: "1"
        - replace:
            apply-on: "value"
            old: "DOWN"
            new: "0"
  bgp-state-mappings:
    event-strings:
      value-names:
        - "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state"
      transforms:
        - replace:
            apply-on: "value"
            old: "IDLE"
            new: "0"
        - replace:
            apply-on: "value"
            old: "CONNECT"
            new: "1"
        - replace:
            apply-on: "value"
            old: "ACTIVE"
            new: "2"
        - replace:
            apply-on: "value"
            old: "OPENSENT"
            new: "3"
        - replace:
            apply-on: "value"
            old: "OPENCONFIRM"
            new: "4"
        - replace:
            apply-on: "value"
            old: "OPENCONFIRM"
            new: "5"

Note that you can use regular expressions to match values and tags as well as do regular expression based replaces, i.e the fields value-names, tag-names, tags, values and .transforms[].replace.old can be regular expressions.

thanks i must have missed the apply-on: value when i read the docs.


Add resourceFilter attribute for all subscriptions

Description
Blocked by altinn/altinn-resource-registry#153
Considerations

Only AT environments might have subscriptions based on a source that isn't on the app domain.

Deployment

[x] Deploy migration script that introduces resourceFilter column for subscription table
[x] Deploy feature where resource filter is set for all app subscriptions
[x] Deploy feature where resource filter is required for generic subscriptions

After deployment of migration scripts:

[ ] Run manual script for modifying existing existing subscription without resource filter

[ ] 	Cannot be a part of migration script at altinn host must be defined per environment

In a later release:

[ ] Deploy migration script dropping old insert_subscription, get_subscription & find_subscriptions function

SQL
Check what resource filter will be
SELECT sourcefilter,
CONCAT('urn:altinn:resource:altinnapp.',
    SPLIT_PART(sourcefilter, '/', -2) ,
    '.',
    SPLIT_PART(sourcefilter, '/', -1)) AS resourcefilter
FROM events.subscription
where sourcefilter like CONCAT('https://%.apps.', 'at22.altinn.cloud', '/%')
and resourcefilter IS NULL;

Add new column
ALTER TABLE events.subscription ADD COLUMN resourcefilter character varying;

Add column value
update events.subscription
set resourcefilter = CONCAT('urn:altinn:resource:altinnapp.',
    SPLIT_PART(sourcefilter, '/', -2) ,
    '.',
    SPLIT_PART(sourcefilter, '/', -1))
where sourcefilter like CONCAT('https://%.apps.', 'at22.altinn.cloud', '/%')
and resourcefilter IS NULL;

Task

[x] Ensure all app subscriptions are given a resourceFilter attribute upon creation, if it doesn't exists
[x] Require resource filter for all subscriptions on generic events
[x] Implement unit tests
[x] Manual test locally
[ ] Run manual db script

[x] AT
[x] YT
[ ] TT
[ ] PROD

[ ] Ensure all existing subscriptions in database have a resource filter property
[ ] QA
[ ] Manual test

Test cases

Post subscription for app resource: resourceFilter included in response object and persisted in DB.
Post subscription for generic resource: resourceFilter included in response object and persisted in DB.
Post duplicate subscription: same subscription id returned
Post almost identical subscription, everything but resource identical: new subscription id returned
Get subscription by id: resourceFilter included in response object
Get subscription by consumer: resourceFilter included in response
Verify that events are pushed to subscription that have matching source filter. Support for matching on resource will be developed later

Acceptance criteria

[ ] All Subscriptions have a resourceFilter attribute defined in the database
[ ] When an app subscription is posted,  a resourceFilter should be generated based on the source filter
[ ] A subscription for an "unknown" event source should always require a resourceFilter

Ready for testing in AT.
Remaning dev tasks are related to the manual work that needs to be done after deploy to an environment.

Testing completed for now. Created two issues and one open question.

#351
#352
It's possible to create subscriptions for events from an non existent resource. Is this handled by a separate issue?


Args test: use test table to write more tests

following https://github.com/massalabs/massa-as-sdk/issues/16 and the test table for envy test, we will be able to use test table to add more test and keep some demonstrative tests (with comments and how-to use guide).
This issue is blocked by https://github.com/massalabs/massa-as-sdk/issues/29

This issue is now block by massalabs/as#39


Added domain for Horizon Community College

@ukjbrown Pull request merged. Thank you!

@ukjbrown Pull request merged. Thank you!


changed function call tf.nn.deconv2d to tf.nn.conv2d_transpose for TensorFlow 0.7.0

I have changed function call tf.nn.deconv2d in ops.py to tf.nn.conv2d_transpose to reflect the name change in TensorFlow 0.7.0.

thanks for commit but we'd better use both of them for legacy. What about change the line from
deconv = tf.nn.conv2d_transpose(input_, w, output_shape=output_shape,

to
try:
    deconv = tf.nn.conv2d_transpose(
catch:
    deconv = tf.nn.deconv2d(

Yes, that is a much better idea. Do you want me to send another pull request or do you want to just do it yourself?

@terrybroad If you make another commit then I'll merge it. You don't need to make another pull request and just commit in this branch and push it :+1:

Ok I have fixed that now with:
`         try:
deconv = tf.nn.conv2d_transpose(input_, w, output_shape=output_shape,
strides=[1, d_h, d_w, 1])
     # Support for verisons of TensorFlow before 0.7.0
     except AttributeError:
         deconv = tf.nn.deconv2d(input_, w, output_shape=output_shape,
                              strides=[1, d_h, d_w, 1]) `


Make NModbus4 portable

Blocked until following issues are open

[ ] Reimplement TCP transport to rely on Socket (#27)
[ ] Reimplement UDP transport to rely on Socket (#28)
[ ] Exclude serial transport into separate dll (#29)
[ ] Make NModbus4 assemebly portable

This is something that I'm very interested in. I'm currently working on porting some of my .NET / NModbus4 code over to Linux. I am available to test anything you come up with.

@rquackenbush thanks. I think future target will be dotnet so #27 and #28 become unnecessary.


[Compatible] Run integration tests on nightly

Explain your changes:
Remove condition in dhall files not to run test executive tests on compatible branch
Explain how you tested your changes:
Unfortunately it's only possible to merge it and check if on compatible tests are executed
Checklist:

[ ] Dependency versions are unchanged

Notify Velocity team if dependencies must change in CI

[ ] Modified the current draft of release notes with details on what is completed or incomplete within this project
[ ] Document code purpose, how to use it

Mention expected invariants, implicit constraints

[ ] Tests were added for the new behavior

Document test purpose, significance of failures
Test names should reflect their purpose

[ ] All tests pass (CI will check this if you didn't)
[ ] Serialized types are in stable-versioned modules
[ ] Does this close issues? List them

Closes #0000

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me


MNT: Re-render the feedstock [ci skip]

Hi! This is the friendly conda-forge-admin automated user.
I've re-rendered this feedstock with the latest version of conda-smithy (1.4.2) and noticed some changes.
If the changes look good, then please go ahead and merge this PR.
If you have any questions about the changes though, please feel free to ping the 'conda-forge/core' team (using the @ notation in a comment).
Remember, for any changes to the recipe you would normally need to increment the version or the build number of the package.
Since this is an infrastructural change, we don't actually need/want a new version to be uploaded to anaconda.org/conda-forge, so the version and build/number are left unchanged and the CI has been skipped.
Thanks!

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Closing this. It needs to be re-rendered with the latest conda-smithy to work.


Use a transitionend listener instead of timeout

https://github.com/follywolly/web-app-from-scratch-18-19/blob/129e3c7de7e32b32a2eef8e74dfca60315437b4b/src/js/modules/slider.js#L9

Interesting, wasn't aware of this event. Implemented!


Unexpected token 'export' when pulling in single-spa as a dependency

Sorry if this is the wrong place to ask this kind of specific question.  I am using this library to generate some import maps.  One of the dependencies is single-spa.  The package is referencing version 5.9.3.  When I pipe it through the generator using these settings
    defaultProvider: 'jspm',
    env: ['production', 'browser']

I am getting this
<EMAIL_ADDRESS>However, at the end of that file that it is pulling down is
;export default n;
Is there a reason that is being added? I don't see that in the npm version and I am not setting the env to 'module'.  Am I missing some configuration somewhere?  Thanks!

I might have opened this ticket too soon.  I switched the default provider to jspm.system and that seems to give me a version within the export being added.

@SomethingSexy you're going to want to add the "module" condition in the env there and then it will use the "module" field in the package.json defined. JSPM will convert CommonJS and UMD modules into ES modules, hence it adds a default export to make them importable.

@guybedford  should it add that default export at the end of the file, outside of the System.register wrapper?

@SomethingSexy System modules aren't supported as input on JSPM - UMD modules are interepreted as CommonJS and thus given a default export. Add the "module" condition to use the correct entry point for SingleSPA.

Got it! Thanks for the explanation.  I think I might have something else going on with the version of single-spa being return from JSPM vs jsdelivr but I need to investigate further.  This issue can be closed.  Thanks again.

@SomethingSexy using:
    defaultProvider: 'jspm',
    env: ['production', 'browser', 'module']

should fix the issue as discussed above.


perf(VectorSource): unbind FeaturesCollection when disposing

Related to #14466

Thanks for your contribution, @KotoriK. To make it easier to review (and to make the checks pass), can you please run npm run lint --fix to remove the lint that's currently in the pull request?


Subscribe to the onclose event via JS

Hello, I want to subscribe to the onclose event via JS.
Here is an example of what I need:
https://youtu.be/11I4lK00KeI
How do I do this without changing "/_framework/blazor.server.js" ?

Blazor Server side: 0.5.1
/_framework/blazor.server.js

We don't currently expose any API like that. When we implement connection management and automatic reconnection, we'll provide APIs to respond to disconnection.

Would #1341 meet your needs?

@SteveSandersonMS , yes.

Is it dangerous to open access to a signalr connection?
i = (new s.HubConnectionBuilder)

@cores-system Yes, anything you do with undocumented APIs is dangerous!

@SteveSandersonMS, Thanks for the temporary API, this fix #1339 can be used in 0.6?


:telemetry.execute blocks the calling process until UDP is sent

Hi,
I have noticed that when using telemetry_metrics_statsd invoking :telemetry.execute is blocking the calling process until the metric is send over UDP. That was a bit surprising, as I thought the library would forward all the events to TelemetryMetricsStatsd process that would send things over UDP itself?
Is that an intentional design decision? While UDP sending is usually pretty fast, it can sometimes block up (especially when resolving DNS names and especially in K8s/docker environment) and that means the caller is being delayed by the metrics for quite a significant time. I'd like to change this and the easiest would be to modify this library to instead have a background process send metrics over UDP, rather than the caller. Please let me know if you agree with this approach however. :)

Forwarding all the events for publishing to a single process (or even a pool of processes) in an asynchronous manner would easily flood this process in a busy system. Having the process executing the event send the metric over UDP applies natural backpressure and adds an overhead to each process sending a metric proportional to the number of metrics it publishes.
I understand that this it might be not ideal in certain circumstances, but it's hard to decide on a best approach without more information. Would you be able to capture a profile of your system to see where the most time is spent when sending a metric?

Right, that makes sense.
I think the problems stems from the fact our system is extremely latency driven and we are doing everything we can to minimize the latencies in our services.
We have previously seen weird behavior when running inside a Docker container. A lookup in the OS DNS cache would take several milliseconds (much slower than on the host machine where its microsends) and that meant that telemetry is delaying us a lot. After switching the BEAM's DNS lookup to its own local cache things got a lot faster - but now perhaps it seems we are reaching a limit again as we are sending more and more metrics. I am also a little worried that the spikage we are seeing in the 99th percentile and above could be related to that.
I'll do a bit more investigation, but maybe the solution is the one we already applied in one of the services - cache metrics locally and then use telemetry_metrics_poller to actually send them on an interval and tuning them carefully not to overload the system. I suppose that's a bit like folsom used to do.

Here's a sample trace where I sort of reproduced it - it seems that it is, in fact, the DNS lookup that is the worst part of it.
I suppose it means I have to make the solution on our end, rather than modifying the library - that's ok, feel free to close this issue. :)

I think i'm also facing this issue. I've been forced to remove ecto reporting from production, because it adds too much overhead. At first I thought it was because we were sending all metrics through a single genserver (that's why I contributed the pool to this library).
But doesn't seem to be the case, or at least not the only issue. I never really thought about dns being a problem here, I'll have to investigate it. My production is also in kubernetes, so it might as well be the exact same situation you are on right now

@epilgrim
Here's a comparison when talking to statsd using hostname and IP address (both localhost)

The difference is quite big and it seems to be a docker issue on the DNS resolution. The solutions for us was to use an inetrc.conf and feed it into the BEAM
%% -- ERLANG INET CONFIGURATION FILE --
{resolv_conf, '/etc/resolv.conf'}.
{hosts_file, '/etc/hosts'}.
{cache_size, 10000}.
%% refresh every 60s
{cache_refresh, 60000}.
{lookup, [dns, native]}.

Hope it helps. :)

Perhaps something that could be of help here would be to add a function to change the statsd host dynamically through a call to TelemetryMetricsStatsd? That way maybe we could configure it with an IP address but have a background process that does dns resolution and updates every now and then. 🤔  @arkgil what do you think?

Thank you both for contributing to the discussion, this is very valuable data!
Reconfiguring the reporter in flight would make sense, but would require reattaching all the event handlers, meaning that some metrics would be lost. Still, that might be a valid solution.
What would be interesting to see is a mix of synchronous and asynchronous reporting implemented by Elixir's Logger, i.e. send metrics asynchronously but switch to a synchronous mode once the message queue grows over a configured length.

I can whip up a PR for the reconfigure. I'm not so familiar with the internals of Logger, but I'll also take a look. How about having a configuration option for now that allows for either async or sync sending of metrics? Given how there's already a pool, in some system configs (I hope ours) they should be able to handle the load.

@haljin I looked at #46 and I'm not sure the async send is the approach we should follow. Accumulating metric updates in the message queues of processes sending them over UDP looks very expensive in terms of memory. It seems like with your requirements you might be better off aggregating metrics inside the BEAM, e.g. by using https://github.com/beam-telemetry/telemetry_metrics_prometheus, or using the approach with poller you described before.
Let's focus on reducing the overhead of DNS lookups, as this is what's hurting performance the most. I like the idea with dynamic update of IP/port, but maybe we can go with something more automatic.

That's fair enough, @arkgil ! I was going to built a DNS poller on top of the library, but I suppose we can add it inside as well and have TelemetryMetricsStatsd process do a DNS resolution on an interval and update its configuration. I'd make the resolution itself and polling period configurable. Should I keep the function to update it manually? What do you think? :)


Make JwtCirce.parseHeader public

It's sometimes necessary to parse the header prior to validating tokens.

No time right now. You can always extend JwtCirce and expose the method (probably).
Will think about it. Please ping me in 2 weeks if no news in this thread.

Extending doesn't appear to work.
Do you see some issues with the trivial fix in this PR?

I might. Even if you parse it, how can you be confident about reading any data from it without checking it has not be compromised? Nearly all exposed methods of the lib will check if the token is matching its signature before giving you any result.

What's the root issue you are facing? Maybe we can think for a global / best solution for it.

@pauldijou What do you think? Can I somehow validate Firebase tokens with the latest version of jwt-scala or are some changes needed? What kind of solution would be optimal in your opinion?

Okay, so you will need to parse the header first indeed. One easy way would be to split the token, parse the first part using any JSON lib, and grab the kid directly. No need to fully parse it as a JwtHeader.
I've released 0.17.0 which should allow you to extend jwt-scala parsers if you still want to use parseHeader.
Hope it will solve your issue.

It's not that implementing header parsing is hard - I've that implemented. It's just that since this is already implemented in JwtCirce I'd like to reuse that code and eliminate my own code.
I'd just like to be able to invoke JwtCirce.parseHeader directly or to extend JwtCirce without needing to implement any methods. AFAIK, I can't currently extend JwtCirce and at the same time reuse parseHeader and parseClaim.

Done in 0.18.0