Draft: Adding stable baselines PPO agent

Here is a running example of stable baselines 3 PPO agent. It requires more custom strategy class, so this does not inherit from the RLBaseStrategy in the avalanche-rl repo.
Main changes:

Add choose_actions(Observations) -> Actions to be called in both training & evaluation to get actions for specific observations. This replaces RLBaseStrategy.sample_actions(), which was only used during training
Add receive_transitions(Vectorized[Transitions]) to be called during rollout with the result of calling env.step(actions). This replaces the Rollout object that was previously returned from rollout. This is also where strategies can put training logic.

NOTE: A Transition is a tuple of (observation, action, reward, done, next observation)

Rework rollout() to enable usage in both training_exp() and eval_exp(). It now generates episode data rather than returning rollouts.
Use Timestep as experience limits and periodic eval frequencies.

Things I'm unsure about:

PPO manages the pytorch Module, optimizer & criterion itself, so what should I pass to the BaseStrategy? Will there be support for Strategies that manage those themselves rather than passing to BaseStrategy?
PPO requires knowing both number of environments (for vectorization) and observation/action spaces up front. I required them in the constructor, but that limits what scenarios this can be used in.

My current thinking for way ahead is:

Extract most of the functionality of this example into either a reworked RLBaseStrategy, or a new base strategy class

choose_actions() and receive_transitions() would be abstract methods of the new base class

Move the PPO specific things to a subclass of the new base strategy. This would only implement choose_actions() and receive_transitions().

Hey Corey, thanks a lot for you contribution!
The main problem with encapsulating sb3 algorithms is that we're not capable of directly exposing internals (e.g. loss) the way Avalanche does, thus limiting the customization capabilities of Plugins. This is why a2c and dqn have been implemented from scratch.
Regarding your changes and rightful concerns:

The intended way to choose an action was to implement both the RLBaseStrategy.sample_actions() to choose actions during training and PPOModel.get_action() for inference. The idea is that you can have different behaviors like eps-greedy during training (https://github.com/ContinualAI/avalanche-rl/blob/master/avalanche_rl/training/strategies/dqn.py#L138) and a simple argmax in inference (https://github.com/ContinualAI/avalanche-rl/blob/master/avalanche_rl/models/dqn.py#L16). This should enable to "throw away" the Strategy once you're done training and have everything you need for inference in a nn.Module class. I'm aware this isn't explicitly written anywhere atm so I really appreciate a feedback on whether you believe this pattern to be confusing.
I was anticipating this change for PPO and I agree with you, personally I would add the action to the Rollout instance in the RlBaseStrategy.rollout() so this becomes sort of a default behavior, while refactoring the Rollout object into the Transition you propose so that it is also optimized to be used along with some replay memory.  I also think receive_transitions logic could be implemented in the RlBaseStrategy.update() method by taking in the gathered transitions, do you agree?
This is kinda dependent on 1. but I like the formulation, although we'd have to take into account VectorizedEnv quirks.
👍🏻

Overall, I think if we do want to support sb3 algorithms we should make a separate class which addresses the problems you mentioned, specifying the limits in customization (e.g. no optim, criterion, loss fiddling..) and your implementation could serve as a base for that. On the other hand, I'm unsure whether having a less flexible SBStrategy could have advantages over re-writing PPO to fit RLBaseStrategy guidelines and possibly add missing features to our base class (like the Transition object).
What are you thought on that @AntonioCarta?
P.S. A workaround to the n_envs init requirement could be delaying the actual initialization until an experience is passed, although it's not the cleanest solution.

Hey @coreylowman, thanks for the PR!
I need a bit of time to look into it but I can give some high-level comments.

Extract most of the functionality of this example into either a reworked RLBaseStrategy, or a new base strategy class

Like @NickLucche said, SB3 doesn't give access to some attributes that we use in Avalanche. Therefore, we will probably need a custom Template for them.
Is it possible to make a generic wrapper for SB algorithms? I would like to avoid having lots of wrappers for all the different algorithms.
Notice that we did some work in avalanche and we are experimenting with templates, which allow to define different training loops, with different events/callbacks, and internal state. I would prefer to limit the number of them since they add a bit of complexity, but I think RL is a place where we will need a lot of customization, which is now supported by the main avalanche branch.

PPO requires knowing both number of environments (for vectorization) and observation/action spaces up front. I required them in the constructor, but that limits what scenarios this can be used in.

Maybe a clean solution would be to provide all these attributes inside the experience and do the configuration before each experience. Alternatively, we can put the attributes in the benchmark object and pass that object to the strategy.

I'm unsure whether having a less flexible SBStrategy could have advantages over re-writing PPO to fit RLBaseStrategy guidelines and possibly add missing features to our base class (like the Transition object).

Of course, the ideal solution would be to have PPO implemented inside Avalanche so that we have full control of it and we can easily make it compatible with the rest of the library. The problem is that:

we don't have the manpower to do it (at least right now)
SB3 is well known and probably uses lots of tricks that would take a lot of time to reimplement on our side

So, if the wrapper is not too expensive to mantain, I think it can be a good middle ground.
Of course this only makes sense if we can support at least part of avalanche features with the wrapper, such as:

metrics. I expect some to be incompatible with the wrapper.
benchmarks
some plugins. Many plugins will probably be incompatible.

Hey all thanks for the feedback!

The intended way to choose an action was to implement both the RLBaseStrategy.sample_actions() to choose actions during training and PPOModel.get_action() for inference.

Since the strategy has access to the self.is_training, you could implement that with the single function. I think it's also nice to let the strategy control that in a single place rather than having to implement it in both the strategy for sampling and the model for inference. It also makes reusing rollout easier since there's only 1 function, but you could just pass in the action function as a parameter to rollout if you want to keep them separate!

I also think receive_transitions logic could be implemented in the RlBaseStrategy.update() method by taking in the gathered transitions, do you agree?

I think so! We found passing in the vectorized transitions to a strategy and letting them handle it any way they want easier for both parties.

This is kinda dependent on 1. but I like the formulation, although we'd have to take into account VectorizedEnv quirks.

Yes the rollout function expects only vectorized environments now - I had changed the make_eval_env() to vectorize too. We should annotate the env passed into rollout as VectorEnv instead of Env

Is it possible to make a generic wrapper for SB algorithms? I would like to avoid having lots of wrappers for all the different algorithms.

I believe its possible to make generic wrapper for on-policy and off policy algorithms separately, but not a base algorithm. This is due to the differences in their collect_rollout() functions. The code in choose_actions() and receive_transitions() is directly from OnPolicyAlgorithm.collect_rollouts(). Technically the code in this PR may be usable for any on policy algorithm, I have not tested that yet though.

Maybe a clean solution would be to provide all these attributes inside the experience and do the configuration before each experience. Alternatively, we can put the attributes in the benchmark object and pass that object to the strategy.

I think including the spaces in the experience would be useful, but most algorithms will need access to the spaces when they construct the model. In our framework we just pulled the spaces from the first experience in the scenario, which seemed to work.
I don't think num_envs belongs there because that really depends on what computer you are running on.

Of course this only makes sense if we can support at least part of avalanche features with the wrapper, such as:

Agree I think many will be incompatible at this point. We could probably extract the nn.Module from the underlying SB3 objects, but getting hooks into the SB3 training function may be difficult.

Do you have a sense for how easy it would be to rework existing algorithms into this SB3 structure? If you're thinking this SB3 structure is good, it'd probably be less work overall to just move existing algorithms to this SB3 structure now. Perhaps do that update first before any of the other updates for this PR?
I think the rest of what you say makes sense! I do think just doing OnPolicy class would be straightforward.

I think I'd rather keep sb3 and avalanche-rl algorithms separated, due to the limitations in flexibility of the former.
So I'd definitely go for an implementation of OnPolicySB3 class, but trying to re-use as much as possible of what's already in place (like the Rollout object).

Ok I've refactored your implementation in this branch https://github.com/ContinualAI/avalanche-rl/blob/pr/avalanche_rl/training/strategies/sb3_onpolicy_strategy.py, let me know what you think of it.
There are many rough edges atm, but the core idea is the following: we can handle model and rollouts/transitions collection from the "outside", while we let sb3 compute the loss and optimization step internally.
The model is managed externally as it shouldn't depend on the experience to be instantiated (tho it can use the task labels supplied to branch-off internally).
The data collection part is like the one you made but it re-uses avalanche-rl components, perhaps we can even improve it at the RLBaseStrategy abstraction level borrowing from your code, but this way we don't "over-specialize" the sb3 class.
Things that should be worked out in order to have the bare-functionalities version are:

rollout buffer integration: I'm somewhat sure that we can swap sb3 RolloutBuffer with our Rollout, perhaps adjusting the api of the latter. This would allow the collection of multiple transitions at once (as long as we add the ppo values to it) before the optimization step, without needing to have this kind of check if self.steps_since_last_train >= self.ppo.n_steps before doing ppo.train().   Currently we're stuck with having max_steps_per_rollout=1 due to this. Perhaps we'd find better solution by looking more into the matter.
Model creation should be wrapped in an intuitive api which hides the workaround needed to create a model that depends on sb3 algorithms args. The only requirement here is that the model shouldn't depend on some experience (we want to create it only once) .
Generalization to OnPolicy.


slf4j (logback) logs not getting forwarded to open-telemetry-collector

I have created a spring-boot micro-service with spring-cloud-sleuth. I have also configured a open-telemetry-collector to forward the logs to datadog. The otel-collector is running in a docker container along with the micro-service. I have used logback-appender to forward the logs also to otel-collector along with traces. However, in datadog, I can only see traces and not logs. The logs printed are using the same traceId and spanId as that of trace.
This issue can be reproduced using a POC with very few files and is uploaded at https://github.com/ff-sdesai/distributed-tracing-spring. The readme.md has the steps to run the application. You will need to enter a valid datadog API key in the .env file.I will not be able to share the key since the key is specific to my organization.
Please let me know if there is something else that I should specify along with these details.

However, in datadog, I can only see traces and not logs.

Since Sleuth is a solution for distributed tracing, as far as I can see everything works fine from Sleuth's perspective, not having logs in datadog with the OTel collector/appender should be an issue outside of Sleuth.


Improve robustness for scenario: slow network/dex not reachable

update github.com/lestrrat-go/jwx/jwk
set httpClient timeout as default
option to set httpClient
option to set refreshInterval

please reopen if not solved with the given PR


Build clang_builtin_headers with Bazel. [phab:D195]

Created by zarko at 2015-04-16 21:37:14:

schroederc accepted this revision at 2015-04-16 21:40:27

zarko closed via commit c0b86fabd655b876b4df7e94754fb0a3fce66649 at 2015-04-16 21:52:50


Unable to create streams for google nest camera entities - no entity_id

Trying to use go2rtc using this integration to create a stream for the my google nest camera so I can use it in more places.
Following the readme to configure it using hass entities (which are working directly in hass) in the go2rtc docs here: https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-hass
I create a configuration like this:
streams:
  left_side_camera:
    - hass:entity_id=camera.left_side_camera

No errors on startup, but when I try to preview the stream using http://myhassinstance:1984/stream.html?src=left_side_camera I get the following error:
undefined error=streams: hass: no entity_id caller=github.com/AlexxIT/go2rtc/internal/mjpeg/init.go:166
Looking at go2rtc issues, this was reported before and was due to missing config info, but I can't see any way / need to do that when running as an integration?

If i should be using the different syntax in the readme in this repo, I'm a bit confused.
If I use hass://supervisor?entity_id=camera.left_side_camera then I get the error undefined error=streams: hass: wrong type: auth_invalid caller=github.com/AlexxIT/go2rtc/internal/mjpeg/init.go:166
If I use hass://myhassinstance?entity_id=camera.left_side_camera then I get undefined error=streams: hass: no token caller=github.com/AlexxIT/go2rtc/internal/mjpeg/init.go:166
So it's like it needs to treat the request as if it's an external system, even though it's running as an integration?

I'm also getting this issue with my Nest Doorbell camera. Go2RTC is installed on my Home Assistant as an Addon. The code is as follows:
streams:
  bedroom_2:
    - hass:Camera-Hub-G3-E44E
    - ffmpeg:aqara_g3#audio=aac#audio=opus
  doorbell:
    - hass:doorbell

Worth noting that the bedroom_2 camera works fine.

hass:entity_id=camera.left_side_camera syntax completely wrong.
How have you installed Hass?
How have you installed go2rtc?

Yeah, I tried a lot of different variations on the syntax, since nothing I tried worked. The second message lists the variations.
Home Assistant is a simple hassos install, nothing special.
go2rtc is installed via this integration, which I relied on to install go2rtc (which works for other camera types) My configuration is:
api:
  password: <password>
  username: admin
rtsp:
  password: <password>
  username: admin
hass:
  config: "/config"
streams:
  <other cameras which work...>
  nest_camera:
    - hass://supervisor?entity_id=camera.left_side_camera&token=<my token>

Yeah, I tried a lot of different variations on the syntax, since nothing I tried worked. The second message lists the variations.
Home Assistant is a simple hassos install, nothing special.
go2rtc is installed via this integration, which I relied on to install go2rtc (which works for other camera types) My configuration is:
api:
  password: <password>
  username: admin
rtsp:
  password: <password>
  username: admin
hass:
  config: "/config"
streams:
  <other cameras which work...>
  nest_camera:
    - hass://supervisor?entity_id=camera.left_side_camera&token=<my token>

You need to use hass://supervisor and not use token when go2rtc runs as addon on same server.
And use hass://ip and token when hass installed on another server

ok thank you - here's my updated findings:

hass://supervisor - fails with "auth invalid" error, no matter what I do.
hass://hostname - fails due to using the container-local IP (172....) to resolve the name of the server.
hass://ip - fails due to using port 80
hass://ip:8123 - works if you supply a token!!

So the ip, port and token approach works in the integration with automatically installed go2rtc scenario! Thank you for your help. I guess it's meant to work with the supervisor URI, which would certainly be more convenient, but it's enough for me!
Thanks.

I also did the tests. hass://supervisor doesn't work for build-in go2rtc version. Only way is to use hass://ip:8123 with long live token. This is a Hass limitation and cannot be fixed.


cache_datasets.py is missing

I am trying to use the checkpoint "logs_sae_goodlight_view3" on test folder provided in the dataset.
I encountered several errors and warnings while attempting to run the
bash ./tools/test_from_config_nll_local.sh ./configs/mocap/early_fusion_zed_mmwave_audio_ucla.py 1
Most important one is:
python: can't open file '/src/mmtracking/tools/cache_datasets.py': [Errno 2] No such file or directory
I use Ubuntu 22.04 and installed all packages according to the github except the lap package.
Installation the lap package through setup.py does not work on my system.
I attached the output of the bash script.
verbose.txt

Updating this line
python $WORK/src/mmtracking/tools/cache_datasets.py $CONFIG
in
test_from_config_nll_local.sh
Resolved the error.


php.packages: Add back meta attributes

Description of changes
This addresses a regression from https://github.com/NixOS/nixpkgs/pull/185683 where all meta attributes went away. Including maintainers and description.
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[X] Fits CONTRIBUTING.md.

Oof... Sorry team!


patches react-native-video+5.2.0.patch

Hi! 👋
Firstly, thanks for your work on this project! 🙂
Today I used patch-package to patch<EMAIL_ADDRESS>for the project I'm working on.

Here is the diff that solved my problem:
diff --git a/node_modules/react-native-video/android-exoplayer/build.gradle b/node_modules/react-native-video/android-exoplayer/build.gradle
index d005a58..6a9922b 100644
--- a/node_modules/react-native-video/android-exoplayer/build.gradle
+++ b/node_modules/react-native-video/android-exoplayer/build.gradle
@@ -28,7 +28,7 @@ android {

 dependencies {
     implementation "com.facebook.react:react-native:${safeExtGet('reactNativeVersion', '+')}"
-    implementation('com.google.android.exoplayer:exoplayer:2.13.2') {
+    implementation('com.google.android.exoplayer:exoplayer:2.13.3') {
         exclude group: 'com.android.support'
     }

@@ -37,7 +37,7 @@ dependencies {
     implementation "androidx.core:core:1.1.0"
     implementation "androidx.media:media:1.1.0"

-    implementation('com.google.android.exoplayer:extension-okhttp:2.13.2') {
+    implementation('com.google.android.exoplayer:extension-okhttp:2.13.3') {
         exclude group: 'com.squareup.okhttp3', module: 'okhttp'
     }
     implementation 'com.squareup.okhttp3:okhttp:${OKHTTP_VERSION}'

This issue body was partially generated by patch-package.

you should switch to V6  version !

following branch includes this fix if needed: https://github.com/react-native-video/react-native-video/pull/2795/files


feature request: improve panels

From this Gist my only feedback: it's not ideal to attach panels to the columns - panels have padding which will effect the row size and break the grid in many instances. Also if you're putting columns inside a twelve column section, you can just omit the twelve column piece. Does that help?

HOWEVER -- those panels are a really nice graphic effect...attractive,
set things off nicely, etc.
Feature request: Make them compatible with the row size!!
Source: Foundation newsgroup, Google Groups

Generally, panels would work fine attached to columns if you removed their padding. However since we can't tell columns to be `width: 50% - 40px' we can't make them work on columns and still pad around them. For that we need another element.

Thank you for responding and explaining.
Perhaps someday you can include this in the documentation about the padding etc., and perhaps comment upon what additional element would be needed as you say above, because those panels really do look purdy. :) :)


Remove beta warnings from docs

As black became stable, the beta warning in the docs should be removed

Let's have a stable release first.

Oh sorry I thought it was already stable after seeing the tag stable and stable in the readthedocs link (https://black.readthedocs.io/en/stable/)


Update pyflakes to version 2.2.0

The new version introduces support for assignment expressions (changelog).
Currently when lint is ran with the expression in the code, pyflakes will crash flake8.
Updating the version fixes this but a noqa ignore is still necessary because pycodestyle did not yet receive an update and does not recognize the new syntax with a colon.
The update will introduce these lint errors: https://paste.fuelrats.com/rotukasohe.txt
where F999s are f-strings with no expressions inside of them and the F821s referring to this method definition's literals in the calls inside the typehints
https://github.com/python-discord/bot/blob/b44caed7c5b26f13b4b31d237ffe07f3157aadfb/bot/cogs/moderation/management.py#L49-L56

How do we feel about F999? I recall a discussion in #dev-contrib regarding multi-line strings such as:
string = (
    f"First line with some {param} "
    f"second line without params"
)

If I remember correctly, the majority of our core devs are in favour of keeping the second f as it aligns the literals.

Looks like the joined f-strings are taken care of, taking your snippet we only get an error about param
D:\pycon>pyflakes splitfstrings.py
splitfstrings.py:2:29 undefined name 'param'

I don't know about the walrus but it can be nice to use in some simple cases without sacrificing any readability.
The pyflakes update still brings useful things so if the walrus is decided against; a notice could be posted or the flake8-walrus package can be used which forbids them.

Looks like the joined f-strings are taken care of ...

That's good to hear, thanks for the clarification.
For the record, I'm not opposed to this. I'm just hoping to get a discussion started to see whether we're all collectively on board.
I like the walrus myself, and I recently bumped pyflakes to 2.2.0 in one of my projects. It is, however, disappointing that even then, the lines still need to be noqa to avoid the problems you mentioned.

Sorry, I did not properly read yesterday and didn't realize pyflakes has had a release. I thought we'd still have to grab the specific commit the adds assignment expression support. Let's do this then, I don't see a reason to wait, and apologies again.

Good news is that pycodestyle already supports walrus on master and is only waiting on a proper release: https://github.com/PyCQA/pycodestyle/issues/911.

Ah, looks like flake8 is incompatible with the newest pyflakes and the above missing placeholder error codes are wrong because of that, the only thing incompatible changes are the error codes which were fixed in this merge commit thatfixes it by simply updating a dict https://github.com/PyCQA/flake8/commit/6efb15cecb5a53e4f605cb0639ddd7f6f33c4a41
Do we care about the error codes enough to delay this until a release? Couldn't find any hints on when that may be.

Hmm, I'd probably lean towards just waiting on a proper flake8 release then.

Let's just wait for the flake8 3.8.0 release. The alpha is out and it doesn't make sense to add a bunch of manual dependency pins just to chase the walrus.

flake8 3.8 is now released: https://pypi.org/project/flake8/3.8.0/

flake8 version will be bumped to 3.8 in #947


Add a landing page explaining what this is

The white Helvetica on a pink background mimics the Hotline Bling album cover:
https://en.wikipedia.org/wiki/Hotline_Bling
It's the exact shade of pink from that cover.

Old version:

New version, with more legible text:

Ready for re-review.

Oh, yeah otherwise this looks totally good. I may do a style pass on this at some point to redesign it, but I'll run designs by you first.


Update German translations

Following the same process as #2917 (thanks DeepL! 🙏) I have updated the German translations in this PR. I also made a few tweaks to existing translations, but of course let me know if I got something wrong there.
Unfortunately, there isn't a way that I know of to ask DeepL to produce translations with the informal tone that we settled on previously, rather than the formal tone, so those translations will definitely need correcting I'm afraid.
All corrections from native German speakers are welcome, especially previous translators including @PythooonUser, @cdfzo and @imbstt.
Thanks! :)

I will have a look later!

@surfdude29 i'll have a look today :) could you please rebase?

@surfdude29 i'll have a look today :) could you please rebase?

That was good timing! 😄 When I saw your email I was actually in the middle of working on an update (I started from scratch as I thought that was easiest).
The new PR is #4742 so I'll close this one.


Java agent file name in Dockerfile not working with OTEL Operator

As the operator has the agent file hard coded as:
https://github.com/open-telemetry/opentelemetry-operator/blob/main/pkg/instrumentation/javaagent.go#L69
I think the easiest way to adopt this would be changing the default file name to javaagent.jar if possible in
https://github.com/signalfx/splunk-otel-java/blob/main/Dockerfile#L3
So we can directly use your built image with the OTEL operator.
Is there any hard rule we have to use existing name?

@owais can you take a look at this?

Splunk does not officially support the community operator but you can use it with a custom image derived from official Splunk image(s) or configure operator differently.
Splunk does have it's own operator for Splunk OpenTelemetry Collector which supports Java auto-instrumentation. It is in beta though. github.com/signalfx/splunk-otel-operator

Thanks for the sharing @owais .
Could you please introduce the major difference between opensource version otel operator vs splunk otel operator?
As a Splunk customer, we'd like to leverage the best practise from vendor, however we don't want to be too far away from the open source project, what would be your recommendation?

Biggest difference is that Splunk Operator deploys Splunk's distributions of agents (Collector, Java instrumentation, etc) with default configuration that makes more sense for Splunk customers.
Another major difference is the CRDs supported by the operators. Community operator lets people deploy a general purpose otel deployment which can work with any vendor but with considerable effort on behalf of the user (configuration, etc). Splunk operator CRDs are optimized to simplify deployment for Splunk customers as much as possible. For example, basic CRD from community operator looks like the following where YOU have to configure everything by hand.
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: simplest
spec:
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
          http:
    processors:

    exporters:
      logging:

    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: []
          exporters: [logging]

While as Splunk operator CRD looks like the following:
apiVersion: otel.splunk.com/v1alpha1
kind: Agent
metadata:
  name: splunk-otel
  namespace: splunk-otel-operator-system
spec:
  clusterName: <MY_CLUSTER_NAME>
  realm: <SPLUNK_REALM>

As you can see, Splunk operator knows how to configure the collector and instrumentation for Splunk customers and needs just the most important bits of info from you (realm, clustername and token).
Hope this helps.

A different filename was used on purpose to make it easier to differentiate what is being used. It is the same reason why the JARs are released under different names. Also a lot of other Splunk components/code rely on such such naming scheme and forcing one component (operator) to break away from convention would be very hard to justify. While the change itself is simple enough to make in Docker, it can have a cascading effect on a number of other components.
I think the proper solution to this problem is the community operator allowing users to custom images/agents by not hard-coding the agent names/paths. I do realize that might take time be built into the community operator though. I'll see if we can contribute it as a feature to the project. In the meanwhile, could you use self-made custom images derived from official splunk ones?
FROM quay.io/signalfx/splunk-otel-instrumentation-java:latest

RUN mv /splunk-otel-javaagent-all.jar /javaagent.jar

Then built and publish this image and use it with operator.

Looks like the official solution from OpenTelemetry Operator for this problem is also to use a custom image: https://github.com/open-telemetry/opentelemetry-operator/issues/752


History request in Bundle returns 404

Describe the bug
FHIR Version?
R4
Data provider?
SQL Server
To Reproduce
Create a patient resource and copy the ID.
This works:
GET Patient/{patientid}/_history

This doesn't
POST http://localhost:8080
Content-Type: application/json

{"resourceType":"Bundle","type":"batch","entry":[{"request":{"method":"GET","url":"Patient/{patientid}/_history"}}]}

Expected behavior
I would expect a bundle with an entry containing a bundle with the history of the resource
Actual behavior
{
  "resourceType": "Bundle",
  "type": "batch-response",
  "entry": [
    {
      "response": {
        "status": "404",
        "outcome": {
          "resourceType": "OperationOutcome",
          "issue": [
            {
              "severity": "error",
              "code": "not-found",
              "diagnostics": "The route for \"/Task/fdcfb035-6dc3-421a-8675-415fc31f9fc7/_history\" was not found."
            }
          ]
        }
      }
    }
  ]
}

AB#119362

Hi @LodewijkSioen! I just ran a quick test and confirmed I'm seeing this issue on my system.

Quickly looked through the trace, looks like it matches 2 routes: Resource History, the correct route and CompartmentSearch. After that is goes into this method to resolve and ends up returning null. I'm wondering if it isn't evaluating the route parameter constraints either.
https://github.com/microsoft/fhir-server/blob/f9e520c29353b2327c7a51454a84266364ffdb48/src/Microsoft.Health.Fhir.Shared.Api/Features/Resources/Bundle/BundleRouter.cs#L117-L147

@LodewijkSioen  Closing as the issue is addressed


Template Literal handling

Why
This PR implement supports for #64 and #214
Issue #64
Transforms
export enum Foo {
  Bar = 'Bar',
  Baz = 'Baz',
}

export interface Party {
  Time: `${Foo}`;
}

into
export const fooSchema = z.nativeEnum(Foo);
export const partySchema = z.object({
  Time: z.union([z.literal('Bar'), z.literal('Baz')]),
});

Issue #214
Transforms
export type Prefix = "large" | "high";
export type Subfix = "back" | "front";
export type Joint = `${Prefix}-${Subfix}`;

into
export const PrefixSchema = z.union([
  z.literal("large"),
  z.literal("high"),
]);

export const SubfixSchema = z.union([
  z.literal("back"),
  z.literal("front"),
]);

export const JointSchema =  z.union([
  z.literal("large-back"),
  z.literal("large-front"),
  z.literal("high-back"),
  z.literal("high-front"),
]);

Codecov Report
Attention: Patch coverage is 77.77778% with 18 lines in your changes are missing coverage. Please review.

Project coverage is 95.54%. Comparing base (9e52cca) to head (1f6dc99).
Report is 1 commits behind head on main.

Files
Patch %
Lines

src/core/generateZodSchema.ts
73.07%
14 Missing :warning:

src/utils/extractLiteralValue.ts
73.33%
4 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #217      +/-   ##
==========================================
- Coverage   97.26%   95.54%   -1.72%
==========================================
  Files          16       18       +2
  Lines         840      921      +81
  Branches      340      368      +28
==========================================
+ Hits          817      880      +63
- Misses         23       41      +18

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Does it make sense to add tests to cover the unhit branches?
e.g. https://app.codecov.io/gh/fabien0102/ts-to-zod/pull/217/blob/src/core/generateZodSchema.ts#L1010

Does it make sense to add tests to cover the unhit branches? e.g. https://app.codecov.io/gh/fabien0102/ts-to-zod/pull/217/blob/src/core/generateZodSchema.ts#L1010

I don't think it does, other warnings in the lib are not covered either (and I believe those are the only non-covered branches)

LGTM
let's merge!


Support Multiple Gateway API Versions in CI

Gloo Edge Product
Open Source
Gloo Edge Version
v.17.x
Is your feature request related to a problem? Please describe.
Yes, I need to test k8sgw against multiple versions of Gateway API for conformance.
Describe the solution you'd like
Expand the Makefile and GH actions to support multiple Gateway API versions for conformance testing.
Describe alternatives you've considered
No response
Additional Context
No response

i believe this is a dup of https://github.com/k8sgateway/k8sgateway/issues/10186


Use Arrow time in tweet created_at

Codecov Report

Merging #20 (b425ddd) into master (6b17302) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #20   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           33        33
  Lines          463       465    +2
=========================================
+ Hits           463       465    +2

Flag
Coverage Δ

unittests
100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

stweet/model/tweet.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b17302...b425ddd. Read the comment docs.


Support Bio NER using stanza processor

This PR fixes asyml/ForteHealth/issues/19.
Description of changes
I add NER configuration in stanza processor
with the following configuration:
config = {   "processors":{        "tokenize":"default",        "pos":"defualt",        "lemma":"default",        "depparse":"default",        "ner":"i2b2"         },   "lang": "en",   "use_gpu": False, }
You can get biological/ medical/ clinical EntityMention of your input data. (This is BioNER)
Also, you can feel free to change the model in the configuration,
for example, with the following  configuration:
config = {   "processors":{        "tokenize":"default",        "pos":"defualt",        "lemma":"default",        "depparse":"default",        "ner":"ontonotes"         },   "lang": "en",   "use_gpu": False, }
You can get basic EntityMention of your input data. (This is NER)
Possible influences of this PR.
Support basic NER and Bio NER using stanza processor: all you need to do is modify the configuration according to your requirements.
Test Conducted
All the test data are explicitly described in stanfordnlp_processor_test.py, and the test is successful.

I suddenly found a problem with the naming of processors: it should be StanfordNLPProcessor instead of Standfor**d**NLPProcessr. I will add this issue after this PR is merged.

@Piyush13y  I have resolved all the changes you requested and change the styles of the test codes by using ddt.
Is it inevatible to use hardcoding to check pos, lemma, dependency parsing, ner and bio ner?
I noticed that the tests in nltk and icd coding both use this kind of hardcodings.

@Piyush13y I have resolved all the changes you requested and change the styles of the test codes by using ddt.
Is it inevatible to use hardcoding to check pos, lemma, dependency parsing, ner and bio ner?
I noticed that the tests in nltk and icd coding both use this kind of hardcodings.

Unit tests do have the hardcoded component most times, since its generally the processing of a component that we are testing. Hence, we use pre determined results and compare the results with it to know its working in the intended manner. My comment was more about the way you were hardcoding, it wasnt needed to make a list of each token written explicitly when you could leverage split() method itself for this.

Bad news is that build errors come one after another.
Does the workflows run automatically or it will run after an approval from a maintainer? It seems like I always waited more than one day for the results of the workflows.

@Piyush13y Could you plz review the requested changes. I have resolved all. XD


RFC: Certificate-Based Authentication

What problem does this PR solve?

Issue Number: ref #464
Problem Summary:
This proposes a design of supporting certificate-based authentication on TiProxy.
What is changed and how it works:
Add a RFC.
Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[x] No code

Notable changes

[ ] Has configuration change
[ ] Has HTTP API interfaces change
[ ] Has tiproxyctl change
[ ] Other user behavior changes

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.
None

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

:exclamation: No coverage uploaded for pull request base (main@545822f). Click here to learn what that means.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #479   +/-   ##
=======================================
  Coverage        ?   64.34%
=======================================
  Files           ?       61
  Lines           ?     5761
  Branches        ?        0
=======================================
  Hits            ?     3707
  Misses          ?     1768
  Partials        ?      286

Flag
Coverage Δ

unit
64.34% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

/test mysql-test

/test mysql-test


overwriting asdf files

This isn't urgent, I am simply reporting it so it's not forgotten.
When attempting to overwrite an asdf file on Windows I get an error:
In [17]: f1.write_to('foc2sky.asdf')
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-17-1418e770a532> in <module>()
----> 1 f1.write_to('foc2sky.asdf')

C:\Anaconda\envs\gwcs\lib\site-packages\pyasdf-0.0.dev417-py2.7.egg\pyasdf\asdf.
pyc in write_to(self, fd, all_array_storage, all_array_compression, auto_inline,
 pad_blocks)
    586         original_fd = self._fd
    587
--> 588         self._fd = fd = generic_io.get_file(fd, mode='w')
    589
    590         self._pre_write(fd, all_array_storage, all_array_compression,

C:\Anaconda\envs\gwcs\lib\site-packages\pyasdf-0.0.dev417-py2.7.egg\pyasdf\gener
ic_io.pyc in get_file(init, mode, uri)
   1033             realpath = url2pathname(parsed.path)
   1034             return RealFile(
-> 1035                 open(realpath, realmode), mode, close=True,
   1036                 uri=uri)
   1037

IOError: [Errno 22] invalid mode ('wb') or filename: 'foc2sky.asdf'

On Linux the file is silently overwritten. Perhaps add a clobber argument to write_to similar to fits.

I'll see if I can reproduce this on Windows (but will have to reboot into Windows first).  I suspect it may because the file is already open under another handle (which is not allowed on Windows, but is on Unix).
I'm not crazy about adding a clobber option -- I think it's better for these things to behave the way the OS' underlying file routines behave so there's consistent expectations.  I've always thought the clobber in pyfits was a mistake, but I might be in the minority there.

I agree; I wish I could rip it out.  I don't want to deal with the fallout of that though.  People were mad enough when I made it raise a KeyError if you try to delete a keyword that doesn't exist.

I can't seem to reproduce this on Windows 8.  There must be something more to it than just overwriting a file.  Keeping the file handle open in the same process, or another process, and trying to write to it then works as well.  Can you provide a script to reproduce it?

There's something strange happening. I can't always reproduce it either. But it seems like it's happening when there's a numpy array in the file. This gives the error:
rot = models.AffineTransformation2D([[1,2],[3,4]])
f=AsdfFile()
f.tree={'model': rot}
f.write_to('test.asdf')
f=AsdfFile.read('test.asdf')
model=f.tree['model']
f1=AsdfFile()
f1.tree={'model': model}
f1.write_to('test.asdf')

Doing f.close() does not help.

Windows 8.1 here (in case this matters).

Thanks.  My wild guess is that the model is holding a memmapped reference to the file (since the model includes an array, albeit very small).  And on Windows, you can't read from the "old version" of the file while writing to the "new version".  Nothing immediately springs to mind to solve this in general -- deep copying the model before assigning it to the new tree may help (if deep copying of models works in general).
Another thing that springs to mind -- which wouldn't fix this problem in all cases, but would help here -- would be to not memmap but just read in small arrays.  There's really no real need to memory map here.

I haven't really followed this closely, but if you're reading some data from mmap, then trying open another handle to write to the same file, you can't do that on Windows.  You have to write to a temp file and then copy it over the old one.

Just by chance came across this:
https://boltons.readthedocs.org/en/latest/fileutils.html#atomic-file-saving
It might be useful to solve this problem.  It also seems like a nice facility to have -- preventing half-written files from being left on disk when an exception is raised along the way.

Sounds worth trying out.

@nden PR #302 adds a test to attempt to reproduce this issue. Since the test passes on AppVeyor, I'm inclined to close the issue unless it's still a problem.

@drdavella This can be closed, not an issue any more.


Discard duplicate data

Due to HAProxy misconfiguration or lazyness we may end up in situation where the very same process is mapped to different sockets eg.
global
    nbproc             1
    stats   socket     /run/haproxy-1.sock mode 660 level admin process 1
    stats   socket     /run/haproxy-2.sock mode 660 level admin process 2

Haproxy will expose process 1 over socket 1 & 2 and not only socket 1:
bash$ echo 'show info' | socat - UNIX-CONNECT:/run/haproxy-1.sock|fgrep Pid
Pid: 4584
bash$ echo 'show info' | socat - UNIX-CONNECT:/run/haproxy-2.sock|fgrep Pid
Pid: 4584

This will lead to a situation when our statistics double as we aggregate the very same data twice. Haproxystats daemon should discard duplicate data from duplicate sockets which can be identified by Pid (probably the best option). Additionally consistency of sockets / processes can be verified using Process_num & Nbproc, so re-reading of sockets / directory containing sockets might be required if things do not match (we do not collect data from all processes).

This is a misconfiguration for which HAProxy emits a warning on startup but it allows to access a single process from 2 sockets. I looked at possible ways to catch this misconfiguration and do something about it, either ignore the stats and emit a critical log messages or perform some aggregation.
The PID and the stats are saved in 2 different files (PID is in admin1.sock_info file and stats in admin1.sock_stat file) which means we need to parse all files, find possible duplicates and then perform the aggregation. In the example configuration you mentioned it could be easy to do such thing, but if someone has 3 processes (nbproc =3 ) and 8 stats socket configured then what we can do?
I am afraid I have to complicate things in order to catch a misconfiguration from a user, for which HAProxy emits already a warning. Thus, I am going to close this as wont fix


Adding webmap links to an item

Is your feature request related to a problem? Please describe.
Greetings I was doing some tests and trying to create an item that included links with this schema for WMS/WMTS/etc services (https://github.com/stac-extensions/web-map-links):
{
  "stac_version": "1.0.0",
	"collection": "test-collection",
  "stac_extensions": [
    "https://stac-extensions.github.io/web-map-links/v1.2.0/schema.json"
  ],
  "type": "Feature",
  "id": "item",
  "bbox": [
    172.9,
    1.3,
    173,
    1.4
  ],
  "geometry": {
    "type": "Polygon",
    "coordinates": [
      [
        [
          172.9,
          1.3
        ],
        [
          173,
          1.3
        ],
        [
          173,
          1.4
        ],
        [
          172.9,
          1.4
        ],
        [
          172.9,
          1.3
        ]
      ]
    ]
  },
  "properties": {
    "datetime": "2020-12-11T22:38:32Z"
  },
  "attribution": "© 2022 The Example Corp.",
  "links": [
    {
      "href": "https://example.com/examples/item.json",
      "rel": "self"
    },
    {
      "href": "https://maps.example.com/wmts",
      "rel": "wmts",
      "type": "image/png",
      "title": "RGB composite visualized through a WMTS",
      "wmts:layer": [
        "streets",
        "satellite"
      ]
    },
    {
      "href": "https://maps.example.com/wms",
      "rel": "wms",
      "type": "image/png",
      "title": "RGB composite visualized through a WMS",
      "wms:layers": [
        "rgb"
      ],
      "wms:transparent": true
    },
    {
      "href": "https://maps.example.com/xyz/{z}/{x}/{y}.jpg",
      "rel": "xyz",
      "type": "image/jpeg",
      "title": "RGB composite visualized through a XYZ"
    },
    {
      "href": "https://maps.example.com/item/tilejson.json",
      "rel": "tilejson",
      "title": "TileJSON",
      "type": "application/json"
    },
    {
      "href": "https://storage.googleapis.com/open-cogs/planet-stac/cocabamba-peru/3d-geofox.ai/3DTiles/tileset.json",
      "rel": "3d-tiles",
      "title": "3D Tiles",
      "type": "application/json"
    },
    {
      "href": "https://maps.example.com/item/example.pmtiles",
      "rel": "pmtiles",
      "title": "PMTiles",
      "type": "application/vnd.pmtiles",
      "pmtiles:layers": [
        "streets"
      ]
    }
  ],
  "assets": {}
}

Here is the actual item created in the database after a POST /collections/test-collection/items:
{
	"type": "Feature",
	"stac_version": "1.0.0",
	"stac_extensions": [
		"https://stac-extensions.github.io/web-map-links/v1.2.0/schema.json"
	],
	"id": "item",
	"geometry": {
		"type": "Polygon",
		"coordinates": [
			[
				[
					172.9,
					1.3
				],
				[
					173,
					1.3
				],
				[
					173,
					1.4
				],
				[
					172.9,
					1.4
				],
				[
					172.9,
					1.3
				]
			]
		]
	},
	"bbox": [
		172.9,
		1.3,
		173.0,
		1.4
	],
	"properties": {
		"datetime": "2020-12-11T22:38:32Z",
		"created": "2024-04-24T09:54:54.384886Z",
		"updated": "2024-04-24T09:54:54.384886Z"
	},
	"links": [
		{
			"rel": "self",
			"type": "application/geo+json",
			"href": "http://<IP_ADDRESS>:8086/collections/test-collection/items/item"
		},
		{
			"rel": "parent",
			"type": "application/json",
			"href": "http://<IP_ADDRESS>:8086/collections/test-collection"
		},
		{
			"rel": "collection",
			"type": "application/json",
			"href": "http://<IP_ADDRESS>:8086/collections/test-collection"
		},
		{
			"rel": "root",
			"type": "application/json",
			"href": "http://<IP_ADDRESS>:8086/"
		}
	],
	"assets": {},
	"collection": "test-collection"
}

It seems these links are being ignored? I'm a bit lost. Do I need to add a conformance class (don't know how) or does this require addition development ? or is it a bug?
@jonhealy1

I feel like this is a bug but I'm not entirely sure - the best might be to run stac-fastapi-elasticsearch

I feel like this is a bug but I'm not entirely sure - the best might be to run stac-fastapi-elasticsearch

Confirmed. Works just fine in https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch
Output from sfeo:
{
	"type": "Feature",
	"stac_version": "1.0.0",
	"stac_extensions": [
		"https://stac-extensions.github.io/web-map-links/v1.2.0/schema.json"
	],
	"id": "item",
	"geometry": {
		"type": "Polygon",
		"coordinates": [
			[
				[
					172.9,
					1.3
				],
				[
					173,
					1.3
				],
				[
					173,
					1.4
				],
				[
					172.9,
					1.4
				],
				[
					172.9,
					1.3
				]
			]
		]
	},
	"bbox": [
		172.9,
		1.3,
		173.0,
		1.4
	],
	"properties": {
		"datetime": "2020-12-11T22:38:32Z",
		"created": "2024-04-24T16:21:54.865138Z",
		"updated": "2024-04-24T16:21:54.865138Z"
	},
	"links": [
		{
			"href": "https://maps.example.com/wmts",
			"rel": "wmts",
			"type": "image/png",
			"title": "RGB composite visualized through a WMTS",
			"wmts:layer": [
				"streets",
				"satellite"
			]
		},
		{
			"href": "https://maps.example.com/wms",
			"rel": "wms",
			"type": "image/png",
			"title": "RGB composite visualized through a WMS",
			"wms:layers": [
				"rgb"
			],
			"wms:transparent": true
		},
		{
			"href": "https://maps.example.com/xyz/{z}/{x}/{y}.jpg",
			"rel": "xyz",
			"type": "image/jpeg",
			"title": "RGB composite visualized through a XYZ"
		},
		{
			"href": "https://maps.example.com/item/tilejson.json",
			"rel": "tilejson",
			"title": "TileJSON",
			"type": "application/json"
		},
		{
			"href": "https://storage.googleapis.com/open-cogs/planet-stac/cocabamba-peru/3d-geofox.ai/3DTiles/tileset.json",
			"rel": "3d-tiles",
			"title": "3D Tiles",
			"type": "application/json"
		},
		{
			"href": "https://maps.example.com/item/example.pmtiles",
			"rel": "pmtiles",
			"title": "PMTiles",
			"type": "application/vnd.pmtiles",
			"pmtiles:layers": [
				"streets"
			]
		}
	],
	"assets": {},
	"collection": "test-collection"
}

I will have a look. stac-fastapi-mongo uses the core library from sfeos. sfeos is much more developed.

It seems the issue is here:
https://github.com/Healy-Hyperspatial/stac-fastapi-mongo/blob/5bcac32d73dcda4049ee95b4fc493267e4d1ea64/stac_fastapi/mongo/database_logic.py#L686

@jonhealy1 I tested upgradding the stac-fastapi.core to from 2.1.0 to 2.3.0 and it seems to fix the issue although ofc, there's other bugs.... when running tests:

If there are no errors it's good. Just make a pr to upgrade to v2.2.0. I will then relase a new version of stac-fastapi-mongo

I would like to upgrade the stac-fastapi.core to 2.3.0
I've made this fork/branch: https://github.com/pedro-cf/stac-fastapi-mongo/tree/stac-fastapi.core-2.3.0
I've fixed 5 of the 9 tests. Was mostly to this big change inte stac-fastapi.core:
PUT /collections is now PUT /collections/{collection_id}
I've also added this to the basic_auth tests to prevent awkawardness when calling tests outside of the github workflow, like with make test:
    if not os.getenv("BASIC_AUTH"):
        pytest.skip()

I can't seem to understand how to fix these remaining tests:

When running stac_fastapi/tests/api/test_api.py::test_datetime_non_interval:
https://github.com/Healy-Hyperspatial/stac-fastapi-mongo/blob/5bcac32d73dcda4049ee95b4fc493267e4d1ea64/stac_fastapi/tests/api/test_api.py#L399-L417
resp_json has no features and fails with IndexError: list index out of range
When I replicate this test in a real environment I don't run into this error.
I don't really understand the intricisis of conftest.py and ctx and app_client.
Maybe you can help me on understanding how app_client and app_client.post("/search", json=params) works and why it might be acting differently ?

The fourth test is a test we were struggling with that has now been fixed in stac-fastapi but the changes are not in the core library yet. There is an issue here that I made: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/issues/230

It's also weird that they weren't failing before and the same tests pass in sfeos without any changes

I'm starting it now going to see what I can figure out.

@jonhealy1 I think it's something to the with the test_item.json
I tried creating the item with the file itself:
curl --request POST \
  --url http://localhost:8084/collections/test-collection/items \
  --header 'Content-Type: application/json' \
  --data @./stac_fastapi/tests/data/test_item.json

and I tried creating the item by copy pasting the output from resp = await app_client.post("/search", json={}) in the tests...
and when I take what's stored in the database and json diffcheck the two:

This curl is suppost to create the same item in the DB, but the date time isn't being converted to the Mongo date format, it's say the same 2020-02-12T12:30:22+00:00:
curl --request POST --url http://localhost:8084/collections/test-collection/items --header 'Content-Type: application/json' --data '{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[152.15052873427666,-33.82243006904891],[150.1000346138806,-34.257132625788756],[149.5776607193635,-32.514709769700254],[151.6262528041627,-32.08081674221862],[152.15052873427666,-33.82243006904891]]]},"properties":{"datetime":"2020-02-12T12:30:22+00:00"},"id":"test-item","bbox":[149.57574,-34.25796,152.15194,-32.07915],"stac_version":"1.0.0","assets":{"ANG":{"title":"Angle Coefficients File","description":"Collection 2 Level-1 Angle Coefficients File (ANG)","href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ANG.txt","type":"text/plain"},"SR_B1":{"title":"Coastal/Aerosol Band (B1)","description":"Collection 2 Level-2 Coastal/Aerosol Band (B1) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B1.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B1","common_name":"coastal","center_wavelength":0.44,"full_width_half_max":0.02}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B2":{"title":"Blue Band (B2)","description":"Collection 2 Level-2 Blue Band (B2) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B2.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B2","common_name":"blue","center_wavelength":0.48,"full_width_half_max":0.06}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B3":{"title":"Green Band (B3)","description":"Collection 2 Level-2 Green Band (B3) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B3.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B3","common_name":"green","center_wavelength":0.56,"full_width_half_max":0.06}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B4":{"title":"Red Band (B4)","description":"Collection 2 Level-2 Red Band (B4) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B4.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B4","common_name":"red","center_wavelength":0.65,"full_width_half_max":0.04}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B5":{"title":"Near Infrared Band 0.8 (B5)","description":"Collection 2 Level-2 Near Infrared Band 0.8 (B5) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B5.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B5","common_name":"nir08","center_wavelength":0.86,"full_width_half_max":0.03}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B6":{"title":"Short-wave Infrared Band 1.6 (B6)","description":"Collection 2 Level-2 Short-wave Infrared Band 1.6 (B6) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B6.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B6","common_name":"swir16","center_wavelength":1.6,"full_width_half_max":0.08}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"SR_B7":{"title":"Short-wave Infrared Band 2.2 (B7)","description":"Collection 2 Level-2 Short-wave Infrared Band 2.2 (B7) Surface Reflectance","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_SR_B7.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"SR_B7","common_name":"swir22","center_wavelength":2.2,"full_width_half_max":0.2}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"ST_QA":{"title":"Surface Temperature Quality Assessment Band","description":"Landsat Collection 2 Level-2 Surface Temperature Band Surface Temperature Product","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_QA.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"ST_B10":{"title":"Surface Temperature Band (B10)","description":"Landsat Collection 2 Level-2 Surface Temperature Band (B10) Surface Temperature Product","gsd":100,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_B10.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":100,"name":"ST_B10","common_name":"lwir11","center_wavelength":10.9,"full_width_half_max":0.8}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"MTL.txt":{"title":"Product Metadata File","description":"Collection 2 Level-1 Product Metadata File (MTL)","href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_MTL.txt","type":"text/plain"},"MTL.xml":{"title":"Product Metadata File (xml)","description":"Collection 2 Level-1 Product Metadata File (xml)","href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_MTL.xml","type":"application/xml"},"ST_DRAD":{"title":"Downwelled Radiance Band","description":"Landsat Collection 2 Level-2 Downwelled Radiance Band Surface Temperature Product","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_DRAD.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"ST_DRAD","description":"downwelled radiance"}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"ST_EMIS":{"title":"Emissivity Band","description":"Landsat Collection 2 Level-2 Emissivity Band Surface Temperature Product","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_EMIS.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"ST_EMIS","description":"emissivity"}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]},"ST_EMSD":{"title":"Emissivity Standard Deviation Band","description":"Landsat Collection 2 Level-2 Emissivity Standard Deviation Band Surface Temperature Product","gsd":30,"href":"https://landsateuwest.blob.core.windows.net/landsat-c2/level-2/standard/oli-tirs/2021/108/066/LC08_L2SP_108066_20210712_20210720_02_T1/LC08_L2SP_108066_20210712_20210720_02_T1_ST_EMSD.TIF","type":"image/tiff; application=geotiff; profile=cloud-optimized","eo:bands":[{"gsd":30,"name":"ST_EMSD","description":"emissivity standard deviation"}],"proj:shape":[7731,7591],"proj:transform":[30,0,304185,0,-30,-843585]}},"links":[{"href":"http://test-server/collections/test-collection/items/test-item","rel":"self","type":"application/geo+json"},{"href":"http://test-server/collections/test-collection","rel":"parent","type":"application/json"},{"href":"http://test-server/collections/test-collection","rel":"collection","type":"application/json"},{"href":"http://test-server/","rel":"root","type":"application/json"}],"collection":"test-collection"}'

The datetimes are being stored as strings in the database not as ISODate and the /search mechanism seems to be working with strings aswell.
The test_item.json creates an item in the db with the following format:

The stac_fastapi/tests/api/test_api.py::test_datetime_non_intervaltest runs the following first query:
Query:  {'$and': [{'collection': {'$in': ['test-collection']}}, {'properties.datetime': {'$gte': '2020-02-12T12:30:22+00:00'}}, {'properties.datetime': {'$lte': '2020-02-12T12:30:22+00:00'}}]}

As you see the date formats are different, and since the comparison is being done bettween date strings with different formats it doesn't work so the result is 0 features.

I hacked this repo together pretty quickly. The previous releases are here and they are probably generally better. I wanted to show that we could build another backend from the sfeos core library.

I hacked this repo together pretty quickly. The previous releases are here and they are probably generally better. I wanted to show that we could build another backend from the sfeos core library.

But it's ok to focus on this repo and try to improve right ?

How does SFEOs handle different datetime formats and does it store datetimes in a generic format ? I don't know much about elasticsearch...

Do you think the datetimes should be stored as ISODates in MongoDB?

if yes:

Do we just try to parse every field with dateutil.parser perhaps ? or just specific fields (datetime,start_datetime,end_datetime,created,updated)?

Do we enforce ISODate formats only ?

I do like mongo a lot and I have used mongo for clients when I had the choice at my old job. I think the project is worth improving but almost everyone I have talked to in geospatial seems to have an opposite opinion. I was working on stac fastapi mongo first but no one was paying attention then I did elasticsearch and people were like wow - elasticsearch.

I attended a conference and Paul Ramsey, the guy who did A LOT of the geospatial stuff for postgis, asked the creator of stac-fastapi-pgstac how come they didn't build it in mongo. I thought that was pretty cool. The datetime stuff really confuses me. I haven't worked with it a lot. Everything I have done is just: 2020-03-01T00:00:00Z

I didn't know anything about es when I started stac-fatsapi-es. Phil Varner stepped into help out with that part later on behalf of a big company because they were wondering about the performance implications

I still don't know that much about es or mongo ....

I think the big issue is a managed service for elasticsearch is very expensive on Azure/AWS and I'm not even sure if would connect seamlessly withe sfeos-app. Have u tried deploying sfeos on azure f.e.  connected with a Azure Elastic Search managed service?

There is opensearch I guess. I had always used the free tier on mongo atlas but then recently we moved a service to aws documentdb which is compatible. I have never had the opportunity to run stac-fastapi-es in the cloud. I'm pretty sure it would be fine.

mongo atlas has serverless options which aws documentdb doesn't have. I wanted to use it for a non-profit we were working for but our pm thought otherwise.

One of the many things changed in from 2.2.0 to 2.3.0:
POST /search
{
	"collections": [
		"test-collection"
	],
	"datetime": "2020-02-12T12:30:22Z/.."
}

Query in 2.2.0:
{'$and': [{'collection': {'$in': ['test-collection']}}, {'properties.datetime': {'$gte': '2020-02-12T12:30:22Z'}}, {'properties.datetime': {'$lte': '2200-12-01T12:31:12Z'}}]}

QuerY in 2.3.0:
{'$and': [{'collection': {'$in': ['test-collection']}}, {'properties.datetime': {'$gte': '2020-02-12T12:30:22.000Z'}}, {'properties.datetime': {'$lte': None}}]}


Use transaction with Dexie.js

I have the same question
why always 404 in the login page???

hi @az8321550, sorry i didn't see your comment :cry:
You must set your username and repositoryName in the hubpress/config.json of your branch gh-pages :
{
  "meta": {
    "username": "az8321550",
    "repositoryName": "hubpress",
    "branch": "gh-pages"
  },
  "theme": {
    "name": "Casper"
  }
}


The typing effect

I added a typing effect to your title here is one of the screenshot

Please label it has hackoberfest, if you are intrested in merging.


Use public ECR AL2 image as Prowjobs runtime

Changing builder-base Prowjobs to use AL2 from public ECR.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/kind cleanup


If the value is string(from decimal), the MA not work

worked:
{ close: 4976.16, high: 4977.99, low: 4970.12, open: 4972.89, timestamp:<PHONE_NUMBER>000, volume: 204 }

not work:
{ close: '4976.16', high: '4977.99', low: '4970.12', open: '4972.89', timestamp:<PHONE_NUMBER>000, volume: '204' }

demo: https://codepen.io/flowerwrong/pen/zYKezPy

目前要求都是number类型

目前要求都是number类型

Maybe we can add some warning.

Maybe we can add some warning.


add rpms-signature-scan task to pipeline

add rpms-signature-scan task to pipeline

Its just a test for now, I will update the push pipeline as well, and later on update both with the new version of the task.

V.02 of the task was added to both pull-request and push pipelines


Exception: unable to update parent 'Comment' object has no attribute 'args'

When instantiating a domonic.html.comment (or a dom.Comment, same same) object in a Sanic response, I'm getting the exception unable to update parent 'Comment' object has no attribute 'args' (from dom.py line 193) seven to eleven times in my log. I can't reproduce it in a REPL session. I also can't figure out why I get any particular number of exceptions, but they are the same number from page load to page load.
@app.route("/test_comment")
async def test_comment(req):
    return response.html(f"{html(head(), body(comment('foo')))}")

[2021-10-11 12:39:04 +0200] [9197] [INFO] Starting worker [9197]
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
unable to update parent 'Comment' object has no attribute 'args'
[2021-10-11 12:39:15 +0200] [9197] [DEBUG] KeepAlive Timeout. Closing connection.

I can confirm that this issue is gone in 0.6.2.


Pterodacyl isnt sending emails with client login info

Pterodacyl isnt sending emails with client login info is there a template i need or something or am i just being stupid im running panel version 0.7.17

when a client creates the order they dont get an email form pterodactyl regarding login info

Did you set up the mail information in pterodactyl


tts_sync_state not working on swiftmac start-up

I have been debugging why setting mac-default-speech-rate doesn't seem to take effect. I noticed that Emacspeak is sending the rate on server start, but Swiftmac isn't setting it.
Steps to reproduce:

Set the speech rate using C-u C-e d r 0.7
Restart the TTS using C-e C-s. notice that the speech rate gets reset to default, even though all Emacs variables are still as you left them
Here is what Emacspeak sends log-swiftmac:
tts_sync_state all 1 1 0.7
The 0.7 is the rate that Swiftmac should be at which is getting ignored.

OK, so that means that it is not ignorable. It must be implemented.
Easy enough to put back and I will need to go update the docs.
I wonder if it makes sense to make that not use sync state, but to directly send it.

On Apr 17, 2024, at 06:04, Parham Doustdar @.***> wrote:
I have been debugging why setting mac-default-speech-rate doesn't seem to take effect. I noticed that Emacspeak is sending the rate on server start, but Swiftmac isn't setting it.
Steps to reproduce:
• Set the speech rate using C-u C-e d r 0.7
• Restart the TTS using C-e C-s. notice that the speech rate gets reset to default, even though all Emacs variables are still as you left them
Here is what Emacspeak sends log-swiftmac:
tts_sync_state all 1 1 0.7
The 0.7 is the rate that Swiftmac should be at which is getting ignored.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

--
Robert Melton
@.***

I think this clears up that tts_sync_state must be instant as well.  Will put it back.

fixed./


[BUG]Error Installing CompreFace 1.1.0

Describe the bug
Unable to successfully update and use CompreFace 1.1.0
Version of Double Take
Latest Stable Build
Expected behavior
Upgrade or install of CompreFacr addon not to fail
Screenshots
If applicable, add screenshots to help explain your problem.

Hardware

OS: Home Assistant OS OVA on VMWare ESXi 7 VM
Browser (if applicable) [e.g. Chrome, Safari]

Additional context
Everything had been working great with the old version until the new one came out and I tried upgrading. The upgrade failed and left a non functional install of CompreFace. So I removed CompreFace, restored from backup, verified functionality, and tried again. Failed. Removed CompreFace completely and rebooted. Then went to install again and I get the following error

The command '/bin/bash -c apt-get update && apt-get install jq -y && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100

How much free space do you have on your system? That command hasn't changed between versions of CompreFace, so I'm not sure why it would be failing for you now. I'm able to install this version without any issues.
I can look into a few of the solutions here to see if they resolve it for you. Someone in the post said that running docker system prune fixed it for them. Would you mind trying that if you have the ability to do that?

How much free space do you have on your system? That command hasn't changed between versions of CompreFace, so I'm not sure why it would be failing for you now. I'm able to install this version without any issues.
I can look into a few of the solutions here to see if they resolve it for you. Someone in the post said that running docker system prune fixed it for them. Would you mind trying that if you have the ability to do that?

I have a free Jetson Nano and have been trying to find a use for it. Is there a way to run DT and CompreFace on the Nano and have it proxy into Home Assistant? Currently my Frigate server is a Ubuntu Server 20 LTS on an ESXi Sever with the USB controller passed through for the USB Coral TPU. Only thing there that sux is I have to use CPU power for encoding and decoding the streams. But it's on a 40 core Dell PowerEdge R620 so it's doing ok just giving it 8 cores.
My original setup was Frigate on a RPi4 with the TPU, DT in HA as an Add on, and DeepStack running on the Jetson. That worked ok but got a lot of false positives with DeepStack. When I tried CompreFace I was blown away how better it did and also had plug-ins for age, gender, and mask.
So im trying to find the best way to leverage these 3 main hardware pieces together for a Facial Recognition NVR in near real time. I'll take a look at that post and see why it would be doing that.
I don't like messing with HA's docker environment since the supervisor handles most everything. But my guess is that there is a stuck old docker container that might be blocking it. I've been successful in restoring 1.0.0 from backup and it seems to be running, if not a little sluggishly, but every time I try to upgrade ether it fails right out. Or says it's upgraded but won't start the container
Any help you could provide or suggestions would be greatly appreciated. I think im close but from what I've read I'll probably have to build and compile my own image which I've never really done before. I've just started getting used to docker and docker-compose.
Anyway, thanks for the advice and keeping up such a great piece of programming!

I tried an update again after installing HA 2022.11 and got more errors but this time it properly logged in the supervisor log. See below. I've verified connectivity to GitHub and other external resources. All the other updates went through fine. Anything else you can suggest? I'm not worried about saving any data so if there's a way to completely remove it from my system and then install 1.1.0 id be happy to give it a go.
22-11-02 23:59:14 INFO (SyncWorker_8) [supervisor.docker.interface] Updating image c7657554/amd64-addon-compreface:1.0.0 to c7657554/amd64-addon-compreface:1.1.0
22-11-02 23:59:14 INFO (SyncWorker_8) [supervisor.docker.addon] Starting build for c7657554/amd64-addon-compreface:1.1.0
22-11-03 00:02:18 ERROR (MainThread) [supervisor.utils.dbus] D-Bus method org.freedesktop.DBus.Properties.call_get_all not exists!
22-11-03 00:02:18 WARNING (MainThread) [supervisor.dbus.network] Can't process /org/freedesktop/NetworkManager/Devices/55:
22-11-03 00:02:18 ERROR (MainThread) [supervisor.utils.dbus] D-Bus method org.freedesktop.DBus.Properties.call_get_all not exists!
22-11-03 00:02:18 WARNING (MainThread) [supervisor.dbus.network] Can't process /org/freedesktop/NetworkManager/Devices/56:
22-11-03 00:02:18 ERROR (MainThread) [supervisor.utils.dbus] D-Bus method org.freedesktop.DBus.Properties.call_get_all not exists!
22-11-03 00:02:18 WARNING (MainThread) [supervisor.dbus.network] Can't process /org/freedesktop/NetworkManager/Devices/55:
22-11-03 00:02:18 ERROR (SyncWorker_8) [supervisor.docker.addon] Can't build c7657554/amd64-addon-compreface:1.1.0: The command '/bin/bash -c apt-get update && apt-get install jq -y && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
22-11-03 00:02:18 ERROR (SyncWorker_8) [supervisor.docker.addon] Build log:
Step 1/12 : FROM exadel/compreface:1.1.0
 ---> 5a11defc5339
Step 2/12 : ENV PGDATA=/data/database
 ---> Using cache
 ---> d4624b74f4b8
Step 3/12 : RUN apt-get update && apt-get install jq -y && rm -rf /var/lib/apt/lists/*
 ---> Running in 0bc9fe87c98a
Err:1 http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease
  Temporary failure resolving 'apt.postgresql.org'
Err:2 http://deb.debian.org/debian bullseye InRelease
  Temporary failure resolving 'deb.debian.org'
Err:3 http://deb.debian.org/debian-security bullseye-security InRelease
  Temporary failure resolving 'deb.debian.org'
Err:4 http://deb.debian.org/debian bullseye-updates InRelease
  Temporary failure resolving 'deb.debian.org'
Reading package lists...
 They have been ignored, or old ones used instead.

Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package jq

Removing intermediate container 0bc9fe87c98a

Somewhat resolved.
See
https://github.com/home-assistant/core/issues/81433


Flextype Admin Panel: refactor entries creating UI/UX

possible create stage should be part of editor. or after entry creation we should go the editor

or we should just have an  option to setup variants:
redirect to index or redirect to editor


Add documentation for updating tables

Adds documentation for updating PostgreSQL tables with Python. It also adds comments to code and does some reformatting.
Resolves https://github.com/code4sac/learning-blocks/issues/36

Hey @shwndea, this is a work in progress; don't feel the need to review it. I will merge the changes when I'm done.


remove need for displayWith function on mat-autocomplete

Bug, feature request, or proposal:
Proposal
What is the expected behavior?
A simple way to display the selected autocomplete value in an input field.
What is the current behavior?
Currently when you setup an autocomplete and bind a [value] to the , the autocomplete displays the value UNLESS you assign a displayWith function. However, it makes no sense to me to default to showing the [value] since that is intended to be the behind-the-scenes value, not the displayValue. If we simply default to displaying the viewValue somehow, we would no longer need a displayWith function, and that would eliminate numerous other headaches reported in other issues on this forum.
What are the steps to reproduce?
Providing a StackBlitz/Plunker (or similar) is the best way to get the team to see your issue.
Plunker starter (using on @master): https://goo.gl/uDmqyY
StackBlitz starter (using latest npm release): https://goo.gl/wwnhMV
This proposal relies on an understanding of the typical autocomplete implementation e.g. here: https://plnkr.co/edit/W9vdWaGcoG90LKM8hHgI?p=preview
What is the use-case or motivation for changing an existing behavior?
displayWith function has numerous issues reported. For my purpose I see it as a potential blocker to generating dynamic forms in particular, for reasons i'm glad to detail if need be but I'll spare at this time.
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
Is there anything else we should know?
i'm guessing the displayWith function was created in the first place to address a legitimate challenge. Namely, the autocomplete control needs a way to know what the displayValue is.
so in the demo example here:
<mat-form-field>
  <mat-select placeholder="State">
    <mat-option>None</mat-option>
    <mat-option *ngFor="let state of states" [value]="state.code">{{ state.name }}</mat-option>
  </mat-select>
</mat-form-field>

the displayValue is state.name. Perhaps the new syntax that requires no displayValue would be like this:
<mat-form-field>
  <mat-select placeholder="State">
    <mat-option>None</mat-option>
    <mat-option *ngFor="let state of states" [value]="state.code" [dislayValue]="state.name"></mat-option>
  </mat-select>
</mat-form-field>

Such a design would continue to allow flexibility to assign the whole object or a particular field to the [value]..which is important to various use cases out there.
I don't see any downside to eliminating the displayWith function as long as the viewValue is always shown I doubt people would care.

<mat-option *ngFor="let state of states" [value]="state.code" [dislayValue]="state.name"></mat-option>

I disagree with limiting mat-option template to a single string representation (displayValue). It is very powerful and flexible to allow custom templating within mat-option, and I'd opine that feature is more important than the issues presented in https://github.com/angular/material2/issues/4863.

I know I suggested just showing viewValue by default, but that can be troublesome when using icons of images in your mat-option, since viewValue is really just the option's innerText.
I could get behind a mat-autocomplete-trigger directive like mat-select has. It would give you access to the innerText of the selected option, but also support all the current behavior by letting you continue to use your displayWith method.
I could also get behind allowing slightly more context than just value to be added to mat-option. So instead of just binding a value, you can bind something like
<mat-option [value]="myOption.value" [representation]="myOption">

And when doing so, displayWith would be called with representation || value.

I see it as a potential blocker to generating dynamic forms in particular, for reasons i'm glad to detail if need be but I'll spare at this time.

Since dynamic can be used in a lot of different contexts, I'd be interested in what issues you're encountering.

Thanks for the quick response.
I wasn't thinking of those advanced non-string displayValues...and that raises more questions e.g. does displayWith function support those anyway? Or is displayWith only compatible with string displayValues? if yes to the latter then maybe we're okay to support a binding like [viewValue]? Just a thought.
re: the dynamic forms angle...i've been playing with ng-dynamic-forms library and it currently can't support displayWith dynamic creation. Upon reviewing the code it relies on a 'dynamic control component' to generate each control, and a parent form component that dynamically generates multiple controls.
Constraints:

The displayWith function itself has to be in the parent component, since it processes data for the entire form. i.e. at creation-time the individual form controls are blind to form-wide data.
Accordingly, I would need to bind the displayWith function in the parent form after the dynamic form has been created. I don't see any easy hooks to do that for a dynamically generated mat-autocomplete.

..and this is how I ended up wishing displayWith wasn't a necessity, and wishing the displayValue would always show by default. If there's a way to show the selected option (not it's [value] without an explicit displayValue I guess that would be ideal given what you've said.

btw, just wanted to hone in on this part of your response:

I could get behind a mat-autocomplete-trigger directive like mat-select has. It would give you access to the innerText of the selected option, but also support all the current behavior by letting you continue to use your displayWith method.

Displaying innerText is what I also was trying to describe earlier...if innerText displayed by default I can't conceive of ever needing displayWith. Your answer suggests you would still want displayWith capability even after a re-design..curious as to when you think it might still be useful. Thanks.

i've been playing with ng-dynamic-forms library and it currently can't support displayWith dynamic creation

Ehh IMO just because another wrapper library doesn't yet support displayWith does not justify its removal.

...if innerText displayed by default I can't conceive of ever needing displayWith... curious as to when you think it might still be useful.

Suppose you're using the example on the docs site,
<mat-option *ngFor="let state of states" [value]="state.name">

  {{ state.name }} |
  Population: {{state.population}}
</mat-option>

The inner text of the Arkansas option would be " Arkansas | Population: 2.978M". I think it's reasonable that a user would want the trigger to simply read "Arkansas" instead of the full innerText. If they were able to template the trigger to their liking, they could just do
<mat-autocomplete-trigger>{{myDisplayWithFn(select.selected?.value)}}</mat-autocomplete-trigger>

or, as you said, just use the viewValue
<mat-autocomplete-trigger>{{ select.selected?.viewValue }}</mat-autocomplete-trigger>

I identify a few ways this could go to provide a better developer experience,

Remove the ability to template inside mat-option (maybe that's totally valid! but it's also a breaking change and feature reduction)
Allow users to template their own trigger like mat-select
Provide more context to displayWith so that users don't have to cache huge arrays and filter through them each time they want to retrieve the original object.
Maybe add a dataAccessor to mat-option so [value] can be bound to the full object, but the dataAccessor function can be used to set the NgControl? I have no idea if that could work, but it would also solve OP's problems in #4863.

re: this:

Ehh IMO just because another wrapper library doesn't yet support displayWith does not justify its removal.

The problem isn't with this one library as it stands..i've created a fork of it here and have enhanced it to bind to objects, but supporting displayWith dynamically is the missing piece for it to work for my app. Currently my autocomplete fields show a mongodb ObjectId once a value is selected because of this limitation..ugh!
re: continuing to support displayWith even if the default display is the innerText, I get it now..that makes sense that the innerText isn't always the desired display value in the example you gave.
That said, I'd dont have anything to add about how to improve displayWith since I'm hoping not to rely on it for my use case. For those that would rely on it AND want dynamic forms, ideally the new displayWith approach would be dynamic forms friendly i.e. formbuilder should be able to programmatically set the displayWith function, rather than being restricted to define it in markup only.

I'm getting tripped up on implementing what I think is a fairly standard use case for a combobox (autocomplete).
The list data comes from a backend service in a Key-Value pair format. {id:1, name:"One"}) I want to bind id, and show name. I also want to filter when I type.
Is this supported today?

Update: i've got the scenario I was trying to support (matAutoComplete with id as value etc.) working for now as you can see here
Defer to angular team on whether to close out this ticket. i.e. if this solution is sufficient, or if they still want to revise the displayWith directive. My personal opinion after wrestling with all the above discussion on the 2 valid / common scenarios (mine and the one detailed above by @willshowell) and both should be supported easily without having to implement the "solution" in the stackblitz. e.g. a new  directive (used in lieu of displayWith) that you can set to toggle the behavior as desired e.g.
display="value"
OR
display="viewValue" (to show innerText)
Again, a major motivation is that when using dynamic forms, where each control is it's own component, the displayWith function doesn't help much.
So, for example, if I have dynamically built registration form, the displayWith function needs to reference a method in my registration.component.ts file, but instead I'm limited to invoking a displayWith function in the component that builds the input control. This limitation reveals the antipattern / inversion of control issue here, when we start to depend on binding functions to directives, since it's the calling code (registration.component.ts) that usually needs to invoke the bound function. Hopefully that makes sense? I use material2 extensively and displayWith is the only instance I've come across with this anti-pattern.

Any updates about this? Like @kdubious said, it's a fairly common use-case for a functional autocomplete.

Any update on this? I'm running into the same problem. Basically, I am using only values for my model and the displayWith callback is forcing me to make my model look like this: [{ display: 'string', value: 1 }, { display: 'string', value: 1 }] so I can take value.display inside the displayWith callback.

Any update on this? I'm running into the same problem. Basically, I am using only values for my model and the displayWith callback is forcing me to make my model look like this: [{ display: 'string', value: 1 }, { display: 'string', value: 1 }] so I can take value.display inside the displayWith callback.
There is no way to bind a component's context to this function. Neither does it look like MatAutocomplete exposes the string that is displayed as 'triggervalue' to sidestep displayWith and hardcode it by getting a reference to MatAutocomplete and setting it directly when updating the / a model.
Using simple currying like explained here: https://github.com/angular/components/issues/4863#issuecomment-496570865
does not seem to work either, i.e. the displayWith method is executed (several times without changing inputs) but the returned function (value: any) => string is not.

Is there any chance that the vote will be started again for this issue? In our team we decided against this component because of the need to have this extra method which really makes the code ugly since we need to call our pipes on the display value in the component.ts (in the displayWith Function) which feels wrong.  Additionally we still need to apply the same pipe in the template within the MatOption so this is a duplication as weel which needs to kept in sync.

In our team we did the same as you @crainsaw, we decided aganist to use it.
Don't see a bright future for this issue.


refactor: unify pattern loading and variable handling

Stronger separation of concerns between chatter.go and patterns.go
Consolidate pattern loading logic into GetPattern method
Support both file and database patterns through single interface
Maintain API compatibility with Storage interface
Handle variable substitution in one place
Keep backward compatibility for REST API through Get method

The changes enable cleaner pattern handling while maintaining existing interfaces and adding file-based pattern support.

#1144 will probably significantly further refactor this in the future.


ci: fix macos integration tests, remove unnecessary actions

Description

Use Github Runner image macos-13 for Integration tests (Closes #1405)
Remove unnecessary actions

Checklist

[x] The title of my PR starts with a [Conventional Commit] prefix (fix:, feat:, docs:, chore: etc).
[x] I have read the [Contributor Guide] and followed the process outlined for submitting PRs.
[x] I have updated/added tests for ALL new/updated/fixed functionality.
[ ] I have updated/added relevant documentation and added dartdoc comments with ///, where necessary.
[ ] I have updated/added relevant examples in [example].

Breaking Change

[ ] Yes, this is a breaking change.
[x] No, this is not a breaking change.

Related Issues

App tests don't start for some reason on macos-13 image for iOS, but logs are gone in GH CI.

App tests don't start for some reason on macos-13 image for iOS, but logs are gone in GH CI.

Looks like everything passed now?
We should check out the faster macos machines that they have on beta now.


refactor(optimizer): add plan phase to plan root

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?

resolve https://github.com/risingwavelabs/risingwave/issues/16509
Add PlanPhase to the PlanRoot and ensure the PlanRoot would be moved once it is transformed into the final plan, e.g. DistributedBatchPlan, LocalBatchPlan, StreamMaterializedView, StreamSink, StreamIndex and StreamSubscription, etc.

Checklist

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added test labels as necessary. See details.
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[ ] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

Generally LGTM, becuase it prevent some misuse. However, there are still some issues unsolved, such as if someone incorrectly calls the functions that are still using &mut self such as gen_optimized_stream_plan. I am considering whether it would be better to make them private, And I think there is only the planner test needs them as public methods.

Cursor will call gen_batch_local_plan directly right now. For streaming plan maybe we can.


Move Crossref etiquette details to configuration

Crossref etiquette details should be accessible via configuration options instead of needing to modify code.
https://github.com/ietf-ribose/bibxml-service/blob/0d1158f506d2153744a4176f95e3e37559da9a41/sources/doi/__init__.py#L16-L22
Originally posted by @ronaldtse in https://github.com/ietf-ribose/bibxml-service/issues/77#issuecomment-1013906999

I believe almost all of the requisite details are in fact already available from the environment, and those that aren’t (email and site name) will have to be anyway (email for error reporting notifications, site title just to reduce duplication).


Fix Set Outfit command/verb

About the PR

The admin setoutfit command and verb menu option now function in a reasonable manner again.
Specifically, the list of outfits no longer includes all the little fragments of outfits that were added for loadouts, and setting one of the job "gear" outfits no longer makes the character mostly-naked.
Why / Balance

These were broken when loadouts were added to the game, and they're handy for testing and admin stuff.
Fixes #28785
Fixes #27112
Technical details

The list of outfits filters out any StartingGearPrototypes that are referenced by a LoadoutPrototype.
When applying the outfit, if it is referenced as a JobPrototype's StartingGear, that job's RoleLoadout is now looked up and applied to the character if possible. If the target character is controlled by a player, their loadout for that job will be used if they have one. If the command is targeting an Urist instead, the default (species-appropriate) loadout for that job will be used.

Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

finally <3

Somewhat skrunkly but it's a command so doesn't need to be perfect.


Commits from forked repos are showing up in the materialized views

query MyQuery {
  events_daily_to_project(limit: 1, where: {projectId: {_eq: 108}, typeId: {_in: [2, 4]}}, order_by: {bucketDaily: asc}) {
    amount
    bucketDaily
    projectId
    typeId
  }
  event(limit: 1, order_by: {time: asc}, where: {event_type: {id: {_in: [2, 4]}}, toId: {_in: [7786, 7795, 7796, 7797, 7798, 7799, 7800, 7801, 7802, 7803, 7805, 7807]}}) {
    fromId
    toId
    time
    amount
    typeId
  }
}

according to the docs, a simple refresh should fix it

https://docs.timescale.com/use-timescale/latest/continuous-aggregates/drop-data/#drop-raw-data-from-a-hypertable

proposed solution here
https://github.com/opensource-observer/oso/issues/622

I believe this is no longer happening.