Date formatted with %Z results in invalid date when parsed with new Date() in Safari/IE11

I have an application that's parsing date strings with a format that includes %Z like in the example below:
import { utcFormat, utcParse } from 'd3-time-format';

const dateFormat = '%Y-%m-%dT%H:%M:%S%Z';
const dateParser = utcParse(dateFormat);
const dateFormatter = utcFormat(dateFormat);

let dateString = '2017-09-15T06:00:00+00:00';
let dateObject = new Date(dateString);
let parsedDate = dateParser(dateString);

let formattedDateString = dateFormatter(parsedDate);

let newDateObject = new Date(formattedDateString);
let newParsedDate = dateParser(formattedDateString);

The formattedDateString is equal to 2017-09-15T06:00:00+0000 as expected given the return value of the formatZone function (https://github.com/d3/d3-time-format/blob/master/src/locale.js#L523)
function formatUTCZone() {
  return "+0000";
}

The trouble is that in the above code newDateObject has the expected Date value in Chrome/Firefox/Edge, but is a Date which is flagged as an invalid date in Safari or IE11.
I need to pass date strings to a library that then converts them to Date objects, and I'm using utcHour from d3-time to generate some of the date strings. Given those constraints  I was able to work around this problem using the following code instead of just calling dateFormatter(date) directly:
let formatAndFixZone = date => dateFormatter(date).replace('+0000', '+00:00'); // Safari/IE11 fix for default d3 timezone formatting.

TLDR; In Safari and IE11 the Date constructor requires the timezones to be formatted with a : (colon) to be able to parse from date strings.
I get that this is probably a minor concern for d3, since utcParse exists (and local variants as well), but I was curious to know if there was a strong reason for not changing the default formatting to format time zones as +00:00 instead of +0000. This would allow ISO 8601 date strings to be generated with d3-time-format in a way that could be natively parsed in all browsers.

This is the expected behavior when formatting the %Z directive. (Parsing %Z is more flexible: we support ±HH, ±HH:MM, ±HHMM and the literal Z.)
But, it might be possible to extend the %Z directive to include a modifier, similar to how %-d disables the default zero padding. Not entirely sure how that would work since the padding modifiers are applied generically to all directives, and this wouldn’t be a padding modifier per se (unless we want to arbitrarily overload the meaning of the padding modifier in the context of formatting the time zone offset, which is probably inadvisable).
Or, another possibility would be to introduce another directive, like %z, for ±HH:MM.
Or, we could just change the behavior of %Z. I think that would probably also be inadvisable for backwards-compatibility reasons, though perhaps it could be done with a major version bump.

Thanks for taking the time to look into this somewhat obscure issue! :-)
I had indeed noticed PR #4 which made the date parsing looser for timezone formatting
You have a good point that changing the behavior of %Z could be inadvisable for backward compatibility reasons.
Using the padding modifiers (0, _, -) is an interesting option, but it is indeed hard to see how to map those sensibly to the 3 timezone formats (±HH, ±HHMM, ±HH:MM) even if the number of modifiers to formats does conveniently line up.
There's also the fact that there isn't currently a way to choose whether Z should be used whenever possible in place of any occurrence of ±00, ±0000 or ±00:00...
I'd be inclined to look at some type of syntax for the format string like %2Z or %4Z or %4:Z, and maybe use something like the lower case z to parallel that, but with greedy attempts to use Z for the timezone whenever the offset from UTC is 0...
I'm not sure how difficult it would be to implement parameterized variants of %Z like this (and of course the padding modifiers would need to be handled).
I suppose that if people see some value in it then I could try to find the time to make a PR.

Hi everyone,
is it possible that you can help me?
I have a function like this:
var parseDate = d3.time.format("%Y-%m-%dT%H:%M:%S.%L%Z").parse;
But when i call:
parseDate("2018-03-20T21:02:10.981+01:00")
ist always return null. But why?
When i change  the format to "%Y-%m-%dT%H:%M:%S.%L+01:00" it works, but i have timestamps with +01:00 and +02:00 so that wouldnt work out for me.
Can you please say me why %Z is not parsing the +01:00 in my example?

@Sionicmon You are using an old version of D3. The relaxed %Z parser was added in D3 4.0 / d3-time-format 1.0. Here is an example:
https://beta.observablehq.com/d/8734448fe49bfb0e

@mbostock Thanks for your help!

I believe this is covered by #6.


[Enhancement Proposal]: Aws Eks support docs update

Enhancement Title
Update Docs for aws Eks support
Summary of the Enhancement
It refers to https://github.com/ksctl/ksctl/pull/393
Motivation and Goals
Docs update
Detailed Design
N/A
Alternatives
No response
Scope of changes

[ ] Cli
[ ] UI
[ ] Docs

Agreement

[X] I agree to engage in constructive discussion and follow the Contribution Guidelines.

Check this docs https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html#sts-regions-manage-tokens
check the ksctl policies and other aspects


Implement a first basic pipeline string parser for the currently existing pipeline operators

This PR adds the ability to parse & execute the currently implemented pipeline operators in the command line, via vast export <pipeline>.
Fixes tenzir/issues#6
TODO:

[x] unit tests
[x] integration tests
[x] syntax cleanup; we can maybe "globalize" e.g. extractor rules
[x] if slated for 3.0 release: changelog item saying this is experimental & one-liner in docu
[ ] if slated for 3.1 release: changelog item & docu

Thanks for the good test coverage. I think the only code related issue is Dominik's suggestion about the return value from the option_set parse method.

@Dakostu would you mind squashing your commits into a smaller number of logical units? The PR consists of 78 commits now and many of them are small fixups. I understand it is already approved, but doing such a final pass will make it easier to navigate history should it become necessary.

@Dakostu would you mind squashing your commits into a smaller number of logical units? The PR consists of 78 commits now and many of them are small fixups. I understand it is already approved, but doing such a final pass will make it easier to navigate history should it become necessary.

I squashed the number down to 38.


ITR: point to new url

Update URL

Now deployed at https://snoopycrimecop.github.io/idr.openmicroscopy.org/itr.html for review

@jburel are we ready to go with this for the upcoming deployment?

I will push a commit to use https


No sound

Hello, I'm not sure why but I can't get sound to work. I've tried MP3 and WAV on both the official release and a release built from source. I'm testing on a Windows 10 and a Windows Server 2016 machine. The Windows 10 test is using a local user; the Windows 2016 is a domain user. In both cases I have a valid SSL certificate and am connecting via HTTPS. I've tested connecting to both from a Windows 10 machine using Firefox and a Windows 10 VM using Firefox and Chrome.
There aren't any errors in the logs or browser console/network calls. I added some console logging to the audiowebsocket.js file and verified that init() and open() are successful while message(), error(), close(), and receive() aren't called during a connection. I've also verified that Myrtille.Services is sending the appropriate settings to FreeRDP in the RemoteSessionProcess.cs StartProcess() method.
I'm not quite sure in what direction to dig next while troubleshooting this, hoping to get some guidance.

Hi, did you enabled audio into the remote session? also, because this is server to client audio redirection, the rdp client (FreeRDP) must be able to play it; which means, the machine running Myrtille must have an audio card, either physical or virtual (if it's a VM).
It doesn't really make sense in fact, because FreeRDP is not supposed to play the audio stream on the Myrtille gateway (the stream is just forwarded and decoded/played by the browser), but that's how it works in FreeRDP, probably because it relies on the Windows audio API.
For the same reason and because I wasn't able to change the Docker VM settings, I couldn't get the audio redirection or printing with Myrtille running as a container (Windows containers, at least). If anyone has an idea about this, I am listening!

I did some additional digging and can provide the following additional info:

All machines I am testing are VMs. Some are in MS Azure and others are on VMWare. I have tested once from my local physical computer just to be safe (no dice).
The issue occurs both when attempting to connect 'through' the Myrtille gateway to a target machine or directly to the Myrtille gateway server (server: localhost).
The "Remote Audio" device shows up while connected in any of the configurations I tested (win10 -> win10, server->win10, etc.).
I confirmed that sound works when connected via mstsc.exe (Microsoft's built-in RDP client).
In one configuration the Myrtille gateway OS did not have the Windows Audio service running. I rectified this and ensured that audio was redirected when connecting directly via mstsc.exe. However, the issue persists.
I haven't tried running inside of a container but can confirm that printing is working without issue for me.

Thanks!

Can you try to add the (VMWare or Hyper-V) guest tools to the Myrtille VM?

A Myrtille user provided me with the solution. That's what I thought, the VM running Myrtille must have a virtual sound card for wfreerdp to be able to process the audio stream. I could not find the equivalent for Hyper-V (I will re-check later), but please find below the settings for VMWare:
Here are the parameters to apply to the .vmx file:
sound.present = "true"
sound.allowGuestConnectionControl = "false"
sound.virtualDev = "hdaudio"
sound.fileName = "-1"
sound.autodetect = "true"
sound.pciSlotNumber = "34"

Hi @camjcorley, did this helped ?

Hey cedrozor, sorry for the delay. Unfortunately I had to shutdown the VMWare host I was using for testing and am only testing in Azure at this point. However, I do have an update.
I connected directly via the wfreerdp.exe that's packaged with the Myrtille release build and had sound without an issue:
.\wfreerdp.exe /v:<IP_ADDRESS> /sound /u:cjadmin /p:[omitted]
I'm playing around with the initialization of the process arguments now in the Myrtille.Services.RemoteSessionProcess.cs file now but am not having much luck. I think the next stop would be the Myrtille fork of wfreerdp to see what's going on but unfortunately I don't have much experience with C++.

Another update. I've spent this whole time thinking to myself that I could have sworn I tested sound when I first came across Myrtille and that I would've noticed if there was an issue. I just did some testing and confirmed that sound worked without issue on version 2.7.1 with default settings (just install, no configuration). After that release I have this issue.

I was able to resolve the issue with sound by extracting the following files from the 2.7.1 release binaries and copying over an installation of 2.8.1:

freerdp-client2.dll
wfreerdp.exe

I left the following files intact:

freerdp2.dll
winpr2.dll
winpr-tools2.dll

I'm uncertain what the implications of this change are but am going to check out your FreeRDP fork to see if I can tell the commit it started after.

Hi @camjcorley, thanks for your investigation! So it looks like a regression into FreeRDP :/ it's surprising that it requires a sound card when it was not the case in the past. I wasn't sure, but this confirms that I remember having Myrtille in a VM with a sound that worked.
There have been many changes in FreeRDP between Myrtille 2.7.1. and 2.8.0, simply replacing certain DLLs is hazardous. But this is interesting, because it restricts the search between the common rdp client code and the windows client.
I will resync the FreeRDP fork in the next version of Myrtille and see if the problem persists. If you can take a look too, any help is greatly appreciated! :)

Happy to help! I'm not terribly familiar with C++ (or compiling the code for that matter). I was finally able to get a build on the 2.7.0 and 2.8.0 releases of your fork of FreeRDP and their functionality matched my results so far. Sound on 2.8.0 and no sound on 2.7.0.
My next plan was to start checking out verified commits between 2.7.0 and 2.8.0 to see if I can find where the issue starts. However, when I build one (Merge pull request f89d1d4 for example) it doesn't work when plugged into my installation of Myrtille. Is there a step I'm missing that occurs between the verified commits and releases? Thanks!

Did you used cmake to prepare the build projects ? you also need openssl (for example Win32 OpenSSL v1.1.1g). Then you can enable the FreeRDP logs into Myrtille.Services.exe.config or even extended logs as explained here : https://github.com/cedrozor/myrtille/issues/156#issuecomment-505554971.
But the best would be to debug through Visual Studio C++ because there are some arguments which are not passed through command line, such as the target server and user credentials. This is for security reasons, in order to avoid being able to retrieve these informations from the process command line.
I set tags on the FreeRDP fork when I release an new version of Myrtille, so you can focus on commits between 2.7.0 and 2.8.0 (and you can see there were a lot !). I will also have a look as soon as possible.
Thanks again !

Hi @camjcorley , did you found something relevant about this regression?

Hey @cedrozor, other than further confirming my previous findings I have been unable to determine which commit caused this issue to first appear. I have also been going through a few code sprints at work recently and have not had as much time as I was hoping to dig into this.

@cedrozor
Hi, can I ask if this issue is resolved please, we are running 2.8.1 of Myrtille.
My Myrtille setup running from a Windows 10 Home machine works fine with audio, but on our production 2019 server we get no audio in the browser. However the audio is active and the audio works if us manually using wfreeRDP.exe directly.

Hi, this problem only happens when Myrtille is installed on a VM which doesn't have a (virtual) sound card. The current workaround is to add one, as explained before in this thread.
I didn't had the time lately to resync FreeRDP with the latest code, maybe this issue was fixed since. I will try to re-prioritize this and resync FreeRDP in the next version of Myrtille.
Which DLLs state 2.8.1 after you upgraded to 2.9.0? this may be the MSI engine which see no change beside the version number, and should not prevent Myrtille to work, but this is not the expected behavior.

On the other hand, I noticed a regression on the audio. After a limited time (between 40 secs and 1mn), the audio no longer works. This may be due to the fact that the audio websocket only receives data and does not send data (no microphone support with Myrtille) and the browser would therefore take the initiative to close the websocket, which it considers inactive.
If so, it may be a recent update to the websocket protocol. In any case this does not affect the display, whose websocket uses a "keep-alive" with a frequency of 10 seconds. The solution in this case would be to also have such a "keep-alive" on the audio websocket. I would look at this for a future version of Myrtille.

Hey @cedrozor, thanks for following up on this issue! I haven't had a chance to test out the latest version and will do so next week. I'll also try playing around with some alternate virtual sound card options. By the description, @clusteruk is experiencing the same situation as myself.

Hi, the virtual sound card addition solved my problem. I was confused because it had sound available through wfreeRdp so I assumed it was available. After checking with the VM hosting team it was clear no Virtual sound card was applied and mine was just redirected sound which the Myrtille system does not use.
Thanks @cedrozor for the quick resolution.
Steve

@clusteruk that's what threw me off as well. Can you share which virtual sound card solution you went with?

@ccameron-ntap We have our own hosting house and they just set a default one, nothing special but I would need to email them to ask what VM software and sound card.

Thanks @clusteruk! In the meantime, @cedrozor do you have any recommendations for this?

@camjcorley this depends on the virtualization software used to run the Myrtille VM:

VMWare: as detailed upper in this thread, see http://virtualvillage.cloud/?p=546
Hyper-V: https://www.quora.com/Why-doesnt-Hyper-V-support-audio
VirtualBox: https://www.youtube.com/watch?v=tjWKnDVQOuA
ProxMox: https://forum.proxmox.com/threads/sound-on-windows-guest-vm.34419/

In a future version of Myrtille, I will try to remove the need for a sound card on the Myrtille gateway. This is a bit tricky because the RDP audio output virtual channel is created only if the RDP client (wfreerdp.exe) runs on a machine with a sound card (either virtual, physical or RDP redirected). What Myrtille does actually is send the remote audio output to the audio websocket instead of the sound card.
@clusteruk the same thing goes for remote audio input in the other way

Thanks @cedrozor !


v1.14 Backports 2024-03-06

[x] #30928 (@giorio94)
[x] #30916 (@giorio94)

:information_source: Skipped the renovate-related commit, as not relevant in stable branches.

[x]  #31198 (@giorio94)

Once this PR is merged, a GitHub action will update the labels of these PRs:
 30928 30916 31198

/test-backport-1.14

Need to backport https://github.com/cilium/cilium/pull/31198 as well to make sure that we use the variables from the target branch also in pull_request/push workflows.

/test-backport-1.14

Conformance K8s Kind and Conformance K8s Upstream Network failures are expected, because they use the set-env-variables checked out from the target branch, which doesn't contain the additional variables. This will be fixed after merging this PR.

@giorio94 Looks like the version number is missing from this (executed) line in conformance k8s kind:
  curl -L https://dl.k8s.io//kubernetes-test-linux-amd64.tar.gz -o ${TMP_DIR}/kubernetes-test-linux-amd64.tar.gz

Hence tar is complaining that its stdin is not gzip.

@giorio94 Looks like the version number is missing from this (executed) line in conformance k8s kind:
  curl -L https://dl.k8s.io//kubernetes-test-linux-amd64.tar.gz -o ${TMP_DIR}/kubernetes-test-linux-amd64.tar.gz

Hence tar is complaining that its stdin is not gzip.

Yes, this is expected because the workflow checks out the action (set-env-variables in this case) from the target branch, which doesn't yet include the appropriate changes. It will be fixed after merging this PR. This applies to all workflows run on pull_request target, while the ones triggered by Ariane work correctly because it retrieves the actions from the current branch (being this branch from cilium/cilium, and not a fork).

Marking as blocked to avoid merging while releases are in progress.

Releases are out, unblocking.


Change native compilation to use VC++ 2015 instead of VC++ 2013.

For you @pranavkm!

Would this work for the native code in XRE? cc @davidfowl

cc @ChengTian

@pranavkm , I think you mentioned that we will have this in Beta3 release. Why is this PR targeting dev branch?

Sorry, this was for the rc branch. My bad.

XRE isn't using the standard goal to compile native code. XRE/makfile.shade has hard-coded path to MSBuild 14: https://github.com/aspnet/XRE/blob/dev/makefile.shade#L52

We use both now


⚠ CIOOS Pacific ERDDAP dataset DFO_MEDS_BUOYS has degraded performance

In 31a0b98, CIOOS Pacific ERDDAP dataset DFO_MEDS_BUOYS (https://data.cioospacific.ca/erddap/tabledap/DFO_MEDS_BUOYS.htmlTable?STN_ID&time>=now-1week&distinct()) experienced degraded performance:

HTTP code: 503
Response time: 19913 ms

Resolved: CIOOS Pacific ERDDAP dataset DFO_MEDS_BUOYS performance has improved in 2addfd6.


Proactive Alexa ChangeReport messages

Description:
Implement Alexa's proactive ChangeReport API messages. This allows the alexa component to notify the skill that an entity has changed.
Requires that the user provides the skill's client_id and client_secret.
Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#7452
Example entry for configuration.yaml (if applicable):
alexa:
  smart_home:
    endpoint: https://api.amazonalexa.com/v3/events
    client_id: !secret alexa_client_id
    client_secret: !secret alexa_client_secret

Checklist:

[x] The code change is tested and works locally.
[ ] Local tests pass with tox. Your PR cannot be merged unless tests pass

If user exposed functionality or configuration variables are added/changed:

[x] Documentation added/updated in home-assistant.io

Thanks @balloob I'll get back to this ASAP. Do you have any tips or another component I could look at? How can I test the component if it sends a message to an endpoint?

Pretty easy actually 👍 Since you're using aiohttp to make requests, you can just stub it out. Something like this:
async def test_report_state(hass, aioclient_mock):
    aioclient_mock.post('https://url-we-are-going-to-hit', json={ 'data': 'we will send'})
    await do_something(hass)
    assert len(aioclient_mock.mock_calls) == 1
    call = aioclient_mock.mock_calls

Search other tests for aioclient_mock to find more examples

Coveralls complains, means we need more tests:

Receiving AcceptGrant message is not tested
Hardly anything from alexa/auth.py is tested

Coveralls complains, means we need more tests:

Receiving AcceptGrant message is not tested
Hardly anything from alexa/auth.py is tested

Ok. Added one for AcceptGrant (let me know if I missed anything). I'll try to add more for auth.

The coveralls check is a pretty good guidance here. You don't need full 100%, but critical code paths should be tested, or else they will end up throwing exceptions and users complain. For example, what happens when our get access token function throws an exception ?

I've added some more tests now. Do we still need more?

Nope! I'll do one last pass tomorrow

Damn it. I forgot because of all the Logitech stuff. I actually did it partially, but then realized I shouldn't return the refresh token but use that to fetch the access token 🤷♂  I also forgot to do that last pass I promised to do 5 days ago 🤦♂
Right now I'm busy with Christmas, but I will make sure that this will get merged before next beta, which is Jan 3. I'll also work on the Alexa tokens

@balloob Will this still be in the next beta?

Yes! Just got back from by break, slowly waking up too, this is literally the first PR I will look at.


Update syntax for using client

Update syntax for using client since https://syntax.abaplint.org/?filter=client#/abap/expression/SQLClient is currently used only for delete and update.
However, it's also relevant for select, insert, and modify.
In addition to USING CLIENT clnt, there are a few more options:
 | { USING { CLIENT clnt }
          | { CLIENTS IN @client_range_tab }
          | { CLIENTS IN T000 }
          | { ALL CLIENTS } } ...

Releases:
USING CLIENT as of 740sp5
USING CLIENTS IN and USING ALL CLIENTS as of 754
Ref:
https://help.sap.com/doc/abapdocu_latest_index_htm/latest/en-US/index.htm?file=abapselect_client.htm
https://help.sap.com/doc/abapdocu_latest_index_htm/latest/en-US/index.htm?file=abapiumd_client.htm

@mbtools try again, 2.91.5

👍


Document how to add additional packages to the environment, such as pandoc

I use jupyterWith through this shell.nix:
let
  jupyter = import (builtins.fetchGit {
    url = https://github.com/tweag/jupyterWith;
    rev = "";
  }) { };

  iPython = jupyter.kernels.iPythonWith {
    name = "numpy";
    packages = p: with p; [
      numpy
      bokeh
    ];
  };

  jupyterEnvironment = jupyter.jupyterlabWith {
    kernels = [ iPython ];
    directory = jupyter.mkDirectoryWith {
      extensions = [
        "jupyterlab_bokeh"
      ];
    };
  };

in
  jupyterEnvironment.env

and activate the environment with nix-shell --option sandbox false --command "jupyter lab".
When I try to export a notebook to a PDF, my browser shows an error that pandoc is not installed.
I can't figure out how to get pandoc into my jupyterWith environment. This seems like a common task, perhaps suitable for documenting in the readme?

There is currently no default way, but it is simple enough to allow JupyterWith to do that, I will prepare a PR.

I merged a PR that adds the functionality you wanted, look at the commit for an example on how to use it: https://github.com/tweag/jupyterWith/pull/55

Thanks @guaraqe, but when I try e29ce461a2445118fdc791126533d60899a75937 I get
error: 'jupyterlabWith' at /nix/store/7s6wjdfq94cw858bmwrlmyqmfgs8wgzs-source/default.nix:22:20 called with unexpected argument 'extraPackages', at /home/marcus/shell.nix:20:24

Should this: https://github.com/tweag/jupyterWith/blob/e29ce461a2445118fdc791126533d60899a75937/default.nix#L22
actually be something like this?
jupyterlabWith = { directory ? defaultDirectory, kernels ? defaultKernels, extraPackages ? (p: [ ]) }:

The version that would work is:
let
  jupyter = import (builtins.fetchGit {
    url = https://github.com/tweag/jupyterWith;
    rev = "e29ce461a2445118fdc791126533d60899a75937";
  }) { extraPackages = p: [p.pandoc] };

but I agree that what you expected is more natural. It is a little more modular too. I will adapt it.

The PR has been merged, I realized the documentation said what you tried. Thank you for the catch!


How to disable completion inside a command line window?

Issue
I have a mapping keymap("n", ":", ":<C-f>i", opts) which allows me to enter command line window instead of just regular command line. However when I enter this mode nvim-cmp continues to work, interrupting wildmenu completion for :h and other command, which is annoying. So I want to disable completion using the following (or any other working) command :lua require('cmp').setup.buffer { enabled = false } for the command line window but I got no clue how.
Potential solution
I tried solving it with autocommand, but since my vimscript knowledge is exactly 0, it doesn't work.
augroup disable_completion_in_cmd_line
  autocmd!
  autocmd BufWinEnter * if &buftype == 'nofile' | lua require('cmp').setup { enabled = false }" | endif
  autocmd BufWinLeave * if &buftype == 'nofile' | lua require('cmp').setup { enabled = true }" | endif
augroup END

Additional info
My Neovim config can be found here.

Solution
It can be done with autocommands:
autocmd CmdWinEnter * lua require('cmp').setup({enabled = false})
autocmd CmdWinLeave * lua require('cmp').setup({enabled = true})

Props to this comment on Reddit.


Bronze Dataframe

It works! (apparently)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


How to uninstall gaurd

Hi
I mistakenly create another usres guard, now I want to uninstall that users guard, How can I do that?

Remove the folders and files manually. There was no command to remove it.


update elevation profile example to display results in a chart

FYI - I was testing C3 as a way to draw an elevation profile for another project, and used your elevation profile example as a way of generating some data, so I thought I'd share that back w/ you.
http://jsfiddle.net/tomwayson/6zc07mr9/
You need to set the include the M values param when you make the request to the GP service, and then you parse the M and Z values out of the response to use as inputs to the chart. If we end up going this route and implementing it, I could make a PR to your example if you're interested.

this is really cool!   you're almost reading my mind because i forked cedar yesterday with the intention of getting more comfortable mashing up charts and maps.
PR away, but i'm happy to help too as i think it would make a really solid addition.

Cool. As you can tell that fiddle is missing the code to parse the feature in the response. If we go forward with this, then I'll have something I can PR (if you haven't already come up w/ something).

closed via c3a9b28cab8ce77f565aa452329e367df965dcbd


Capitalize beginning letters

was just beginning to bug me

Pulling this into develop. It will propagate through.


Add a new branch for the flang variant

/cc @inducer
Would you mind creating a new branch called "flang" so that I can file a PR against it? We're currently blocked on adding flang to the feedstocks.

Not sure I understand the full context here. What would this allow you to do that you couldn't by cloning this repo and creating a pull request?

I can only file a pull request against branches that exist. @isuruf wants me to put flang in a separate branch.

See this comment: https://github.com/conda-forge/clangdev-feedstock/pull/19#issuecomment-341870635

Shall I file against master?

https://github.com/conda-forge/clangdev-feedstock/tree/flang


Make NotFound public

Hi!
It seems like the loading cache has a NotFound error, but it's not possible to use errors.Is because NotFound is inside internal. Could you make this public so that it would be possible to do errors.Is(err, theine.ErrNotFound) to identify when a record is not found?

@Fire-Dragon-DoL
The NotFound error is designed for internal use within the NVM cache. Specifically, it’s used in the GetWithSecondary method in internal/store/go:
	if errors.As(err, &notFound) {
		return value, false, nil
	}

In this method, the NotFound error is intentionally ignored and is not propagated to the public API. The reason the method has an error parameter is due to the nature of the NVM cache, which operates on disk. This can lead to various types of errors, and simply returning a boolean would not suffice to capture these errors.
Regarding loading cahce, if the NVM cache is not enabled, any error encountered in the Get API will be the error returned by your loading function.

@Fire-Dragon-DoL
The NotFound error is designed for internal use within the NVM cache. Specifically, it’s used in the GetWithSecondary method in internal/store/go:
	if errors.As(err, &notFound) {
		return value, false, nil
	}

In this method, the NotFound error is intentionally ignored and is not propagated to the public API. The reason the method has an error parameter is due to the nature of the NVM cache, which operates on disk. This can lead to various types of errors, and simply returning a boolean would not suffice to capture these errors.
Regarding loading cahce, if the NVM cache is not enabled, any error encountered in the Get API will be the error returned by your loading function.

I see I misunderstood the purpose.
So in case of loadable cache, the value is always expected to be present, because fetched from the loader function in case of cache miss?

the value is always expected to be present, because fetched from the loader function in case of cache miss

Correct


logs: don't create logs in threads of blacklisted channels

Previously, if channel A was blacklisted, the logs command could still be used in subthreads of channel A (though not channel A itself.)
Thanks to wroclaw on Discord for reporting this bug.

fix "seperator" in comments as well : )

in logs/plugin_bot.go... return message "This channel is blacklisted from creating message logs..." needs maybe some fine-tuning, because thread needs to be clarified.... "This channel/thread's parent..." or something like that.


Strange window behavior inside flash frame

Thanks to this patched flash player plugin, I can access archives of a daily newspaper using Waterfox plus "libflashplayer.so" in "flash_player_patched_npapi_linux.x86_64.tar.gz".
However, a strange window behavior (a black area covering the lower half of the viewing pane) reduces the viewing height badly and also prevents viewing of a good top part, as seen in the left screenshot below.
The same image can be seen properly in Puffin mobile web browser featuring builtin flash plugin as seen in the right.

Hey, can you send links to the SWFs for testing?

Hey! Thanks for providing these login credentials.
Unfortunately I can't reproduce the issue. I've tried the following configurations:

Waterfox on Manjaro Linux (KDE)
Basilisk Browser on Manjaro Linux (KDE)
Basilisk Browser on Windows 10
Waterfox on Windows 10

They all render correctly:

I'm not sure if this is a bug we can fix in the first place, it'd have to be something Adobe would fix.
What display manager / window system are you using? What distro with which desktop environment? Might be a bug with the desktop environment.

Sorry, I forgot to provide some system information as well.
I'm using Waterfox G3.2.1 (64-bit) based on Gecko 78.10.0, KDE neon 5.21.5 with X11 display manager.
I have the same problem in a Live Session of KDE neon.

These are the steps I've followed:

Install browser plugin package with the command sudo apt install browser-plugin-freshplayer-pepperflash (actually I get the same results with or without this - probably it's for Chromium-based browsers only).
Download https://github.com/darktohka/clean-flash-builds/releases/download/v1.7/flash_player_patched_npapi_linux.x86_64.tar.gz
Copy libflashplayer.so to /usr/lib/mozilla/plugins/ (installing the standalone flash player also makes no difference as I remember)
Download https://cdn.waterfox.net/releases/linux64/installer/waterfox-G3.2.1.en-US.linux-x86_64.tar.bz2
Extract contents to /opt
Start /opt/waterfox/waterfox
I also installed Manajaro KDE on another laptop, but I couldn't even get the plugin showing up following the steps above (unfortunately I couldn't get anywhere after enabling AUR and successfully refreshing community.db and community.files). It seems Arch Linux is still too complicated for me although I regard myself a bit above average level Linux users. ;-)
I understand if you don't consider this an issue to be attended by you...

Hey there!
I downloaded KDE neon 5.21.5 and installed it on my machine (using your steps 2 to 6). Only difference is that I chowned the /opt/waterfox folder to my user before running it.
Unfortunately, I do not see the strange black area that you do.
Might this be a video driver issue?

I also performed two tests on another laptop with hybrid Intel/AMD graphics card with KDE neon Live Session.
I was hopeful with the command env DRI_PRIME=1 "/home/neon/.local/opt/waterfox/waterfox" but it was the same again.
The plugin actually works except this particular frame/panel configuration.
 ;-)
Thanks for your efforts!

This looks very much like a display scaling issue to me. If you have display scaling enabled in KDE Neon's settings, you could try turning it off, if this is still an issue.

@n0samu Thanks, but no, I haven't touched display scaling, and this happens to me when testing with KDE neon live session with all default settings.
After @darktohka reported that this problem couldn't be reproduced on a fresh install of KDE neon a year ago, it seems this is a weird problem related my particular hardware/software configuration, so I should better close it.


WithPermissionsFilteredChildren stores the children as state, making it impossible to control which children of a list are rendered

First I would like to say that I really appreciate the work you have done with this great framework.
I have an issue with WithPermissionsFilteredChildren. Apparently this component stores the children as state and by that it controls which children are rendered. This makes it is impossible for a component wrapping a e.g List to control the children.
Although there may be reasons of which I'm not aware, I don't feel that a framework should interfere with regular expectations of how React children work. Are there any particular reasons for storing the children as state?
I have made an example of how I would expect it to work in this fiddle: https://jsfiddle.net/activist/L19pgec6/
If you try to do the same (with the respectively named components) in admin-on-rest the second child/paragraph will render even though the wrapping component removes it.
Any thoughts on this? Would it be possible to rewrite this part of the code? Or if the children must be stored as state, could a callback possibly be provided to update the children state?
Also see https://stackoverflow.com/questions/46606712/is-there-a-simple-way-of-implementing-a-column-picker-for-a-list for relevant example.
Environment

Admin-on-rest version: 1.3.1
React version: 15.4.2

Have you tried the function as a child pattern here ? See [restricting-access-to-fields-and-inputs(]https://marmelab.com/admin-on-rest/Authorization.html#restricting-access-to-fields-and-inputs) in the documentation. In the 2.0.0 version, you won't have to use an authClient for this to work. In the mean time, just implement a dummy one if needed

In any case, please provide a small repository reproducing the issue

I tried to use the function as child pattern, but it didn't really make any difference (or at least I was unable to make it work). My theory is that the children is never updated after initial render since children are being set as state in WIthPermissionsFilteredChildren's componentDidMount which only runs once. I cannot find any place where they are "updated" after that.
I added a fork of the admin-on-rest-demo here: https://github.com/activist/admin-on-rest-demo
Check out the "Orders" section where I have wrapped the original Command-list with a "ColumnPickableList" which in turns manipulates which children should be rendered.

@activits I don't understand what you're trying to do, can you elaborate?

@fzaninotto I have a table with so many columns so that horizontal scrolling occurs. My usecase is that I want to dynamically control which columns in a List are displayed based on some condition. For my scenario this condition is a set of checkboxes (one per column) that when unchecked, the column(s) is dynamically removed/hidden from the table.
But with the current implementation of List (and WithPermissionsFilteredChildren) this is impossible as the List children are stored as state in WithPermissionsFilteredChildren in componentDidMount.  In addition to the above fork I have made a small fiddle here where you can see how it should work (or at least how I would expect it to work using regular React components). For the sake of the example the components in the JSFiddle have the same names as the AOR respective components used in the above fork.  Let me know if you need more information.

Understood. So I guess the fix is to add a componentWillReceiveProps method to WithPermissionsFilteredChildren.

Yes, that would probably work if the "initializeChildren" function is called again or the state is updated with the new children..

I figured out a workaround to this issue which works bypassing the List and instead adding the columnpicker as a wrapper around the Datagrid. That introduces some challenges with respect to placement of the columnpicker, but at least it keeps the "children as state" in List.js out of scope and controlling which children are rendered is possible..

@activist Can you share an example of that?  We're trying to do something similar with a Column Selector.  We have a components object and loop through them in DataGrid.  If the key is in the reducer values, it will render the component.  This worked in 1.2.X but fails in 1.3.X.
const component = {
  id: (props) => <TextField source="id" />
}

<DataGrid>
  { this.props.values.map(value => components[value](this.props)) }
</DataGrid>

The workaround I have only tested with the current master, so I actually cannot confirm that it works with e.g 1.3.x yet. I will try to make time for it this evening.
Regarding the approach it is slightly different than yours. It is basically what I cooked together for this bug: https://github.com/activist/admin-on-rest-demo/blob/master/src/columnpicker/columnpickablelist.js
Just note that as stated above the workaround for my case seems to be to let the ColumnpickableList wrap the Datagrid directly so that the render method would be like this (i.e List is commented out by intent):
...

    removeHiddenColumns(children) {
        return React.Children.map(children, (child) => {
            if (!child.props.source) {
                return child;
            }
            const column = this.props.columnsDisplayed.find((columnDisplayed) => {
                return columnDisplayed.source === child.props.source;
            });

            if (this.props.columnsDisplayed.length === 0 || (column && column.checked)) {
                return child;
            }
            return null;
        });
    }

    render() {
        const { children, ...rest } = this.props;
        const childrenToRender = this.removeHiddenColumns(children);

        return (
            <div className="columnpickable-list">
                <ColumnPicker columns={this.props.columnsDisplayed} onCheckboxChanged={this.props.handleCheckboxChanged} />
                { /* <List {...rest}> */ }
                    <Datagrid>
                        {childrenToRender}
                    </Datagrid>
                { /* </List> */ }
            </div>
        );
    }

@danejorgensen I'm not sure if it is exactly what you need, but I have added a proof-of-concept here: https://github.com/activist/admin-on-rest-demo/tree/datagrid_columnpicker
It works with admin.on-rest 1.3.2. You can see the columnpicker when running the demo and open the 'Orders' page.

@activist thanks, that looks great.  I'm going to try to get your version working with permissions as well.  We have some columns that only an admin should see in DataGrid and the ColumnPicker.

I think we have the same problem :(
cf. https://stackoverflow.com/questions/46979716/components-seems-to-be-not-re-rendered

Hi,
I am facing an issue, wich seems to be linked to the same root cause.
I want update a custom component embbed inside a <Show> component, but this is never updated.
The component is rerendering,but the props is updated.
Originally my custom component was inside a <SimpleShowLayout>, but the problem seems s to be at any level below Show


Is this possible? -> ADFS and "Receives an email that indicates that access is only available using Outlook app"

Just wondering if there is some way this is possible, or if for this clever pseudo email pointing the user to install Outlook is only possible when the Basic authenication is taking place in Exchange Online directly (i.e. no ADFS).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a1d67eec-b53c-4bbc-b160-b3fb8fca5ff4
Version Independent ID: b943cd70-2c1b-5b9b-8ed7-0b0331da3982
Content: Azure Active Directory app-based conditional access
Content Source: articles/active-directory/active-directory-conditional-access-mam.md
Service: active-directory
Product: unspecified
GitHub Login: @MarkusVi
Microsoft Alias: markvi

@JeremyTBradshaw Thanks for your feedback! We will investigate and update as appropriate.

@JeremyTBradshaw I have assigned your concern to content author for further investigation.

Hi ,
The quarantine email, triggered by conditional access policy that directs the user to install Outlook, is just available in Exchange Online. The CA policy is not used by an on-prem instance of Exchange, which is what I think you are asking. If I misunderstood that let me know.
Thanks,
Caleb

#please-close

@JeremyTBradshaw We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


How to open binary position as quick as possible

After i tried this API several times, i found that i must start the websocket connection first with api.start(), then after that i can execute buy_v2 command.
Delay from api.start() is about 10 seconds in my case. I want to open binary position as soon as my signal is firing.
so, i want to know how i can quickly execute buy_v2 command, maybe without always calling api.start() everytime, or maybe by keeping my session alive?
How i can do that?
thanks

cool, thanks for your response.
you're doing an amazing job!

Hi. I have failed to remove the 10 second delay when buying, I changed the 10 sec timeout in the buy methods to 1 sec but there is still a 10 second delay to buy each option, so if i am buying 6 at the same time the last one will be bought 60 seconds later which messes up the strike price significantly and causes losses, Can you tell me how you removed it? Thanks


Allow '.' and '.yaml' for conf file name

This PR has two changes

Enable conf file to prefix with ..
Allow .yaml extension also.

The reasons of adding the changes are:

For 1., I want to use hidden file for config files at the root of repository. And I feel it might be default also.
For 2., .yaml is also widespread file extension for YAML. So only supporting .yml might be confusing.

From result of drone.io,

[info] Timeout request received, killing process

I'm not sure this failure is related to this PR.

Thank you for your review and additional tweaks :+1:


Adafruit Ultimate GPS Issues

Hello,
I'm new to the Johnny-Five libraries, let alone the Raspi-IO library suite. I'm also trying to get the Adafruit Ultimate GPS HAT to work with my Raspberry Pi 2 (similar to issue #47 ). The code:
var five = require("johnny-five");
var Raspi = require("raspi-io");
var board = new five.Board({
  io: new Raspi()
});

board.on("ready", function() {
  var gps = new five.GPS({
    breakout: "adafruitUltimateGPS",
    pins: { tx: "P1-8", rx: "P1-10" }
  });

  // If latitude, longitude, course or speed change log it
  gps.on("change", function() {
    console.log("position");
    console.log("  latitude   : ", this.latitude);
    console.log("  longitude  : ", this.longitude);
    console.log("--------------------------------------");
  });
});

And the output:
pi@raspberrypi ~/nodejs $ sudo node gps_test.js
1471744440488 Device(s) RaspberryPi-IO
1471744440598 Connected RaspberryPi-IO
1471744440628 Repl Initialized
>> /home/pi/nodejs/node_modules/johnny-five/lib/board.js:1008
  var requirements = controller.requirements && controller.requirements.value;
                               ^

TypeError: Cannot read property 'requirements' of undefined
    at GPS.Board.Controller (/home/pi/nodejs/node_modules/johnny-five/lib/board.js:1008:32)
    at new GPS (/home/pi/nodejs/node_modules/johnny-five/lib/gps.js:189:22)
    at Board.<anonymous> (/home/pi/nodejs/gps_test.js:8:13)
    at emitNone (events.js:85:20)
    at Board.emit (events.js:179:7)
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickDomainCallback (internal/process/next_tick.js:122:9)

Not sure what this means. Any idea what the issue may be?
Thanks in advance!
P.S.: I was confused by contributor rwaldron because my abbreviation of first initial/last name is the same (My first name is Ross).

Sorry, wasn't referring to the right breakout. Checking the source confirmed that.
But now I get a new error, something about an invalid pin:
1471786664946 Device(s) RaspberryPi-IO
1471786665059 Connected RaspberryPi-IO
1471786665089 Repl Initialized
>> /home/pi/nodejs/node_modules/raspi-io/lib/index.js:366
        throw new Error('Pin "' + pin + '" does not support mode "' + mode + '"');
        ^

Error: Pin "P1-8" does not support mode "undefined"
    at Raspi.value (/home/pi/nodejs/node_modules/raspi-io/lib/index.js:366:15)
    at Raspi.pinMode (/home/pi/nodejs/node_modules/raspi-io/lib/index.js:348:21)
    at GPS.<anonymous> (/home/pi/nodejs/node_modules/johnny-five/lib/gps.js:269:15)
    at Array.forEach (native)
    at GPS.initialize (/home/pi/nodejs/node_modules/johnny-five/lib/gps.js:267:16)
    at new GPS (/home/pi/nodejs/node_modules/johnny-five/lib/gps.js:251:10)
    at Board.<anonymous> (/home/pi/nodejs/gps_test.js:8:13)
    at emitNone (events.js:85:20)
    at Board.emit (events.js:179:7)
    at _combinedTickCallback (internal/process/next_tick.js:67:7)

Any idea what it could be? Thanks!

Again, more fiddling, and it was magically fixed. Current code:
var raspi = require("raspi-io");
var five = require("johnny-five");
var board = new five.Board({
  io: new raspi()
});

board.on("ready", function() {
  var gps = new five.GPS({
    breakout: "ADAFRUIT_ULTIMATE_GPS",
    pins: ['P1-8', 'P1-10']
  });

  // If latitude, longitude, course or speed change log it
  gps.on("change", function() {
    console.log("position");
    console.log("  latitude   : ", this.latitude);
    console.log("  longitude  : ", this.longitude);
    console.log("--------------------------------------");
  });
});

For future reference, if you're using the Adafruit Ultimate GPS with your Raspberry Pi, make sure your pins object in GPS initialization is an array (TX first, RX second). The Johnny-Five example with the rx/tx object didn't pan out for me.
Closing issue.


Implement builder pattern

To make SDK easier to use builder pattern was implemented in the file Builder.ts. From an implementation point of view, the "Builder with a twist" design pattern also known as "Step builder" was specifically chosen because of the large number of mandatory fields. In short this pattern uses the common builder pattern, but adds a twist by using interface to force which value to set next and thereby solving the problem with many mandatory values.
Builder pattern supports following use cases:

Transfer tokens from parachain to parachain
Transfer tokens from relaychain to parachain
Transfer tokens from parachain to relaychain
Open a channel
Close a channel

To use Builder API, developer first imports a Builder function as follows:
import { Builder } from '@paraspell/sdk'
Builder function accepts one parameter of type ApiPromise. Then, other functions can be chained depending on the use case. In the case of initiating a token transfer the Builder class decides which scenario to use depending on the from() and to() methods. Depending on the scenario different mandatory fields are required. Every step of the build process is properly typed and therefore invalid object cannot be created. After providing all required parameters the build() method executes the transaction or open/closes a channel. Mandatory fields must be specified in the same order as they are shown in the example otherwise typescript error will be generated.
Developer experience
When developing, the developer is guided by the typescript and thus knows which parameter can be added as next. This increases the developer experience and makes SDK easier to use.

Other changes:

Unit tests for each use case are written in the file Builder.test.ts. unit tests are designed in such a way that they check the parameters of the target function, which each case of using the builder pattern calls when build() method is executed. This is achieved by the spy function from vitest.

Examples of all use cases
Transfer tokens from parachain to parachain
Both from and to parameters are provided, thus the parachain to parachain scenario will be used.
  Builder(api)
      .from('Acala')
      .to('Quartz')
      .currency('ACA')
      .currencyId(currencyId)
      .amount(amount)
      .address(address)
      .build()

Transfer tokens from relaychain to parachain
Only from parameter is provided, thus the relaychain to parachain scenario will be used.
  Builder(api)
      .to('Acala')
      .amount(amount)
      .address(address)
      .build()

Transfer tokens from parachain to relaychain
Only to parameter is provided, thus the parachain to relaychain scenario will be used.
    Builder(api)
      .from('Acala')
      .currency('ACA')
      .currencyId(currencyId)
      .amount(amount)
      .address(address)
      .build()

Open a channel
When opening a new channel the operation has to be specified by calling openChannel() method and then providing maxSize and maxMessageSize parameters.
    Builder(api)
      .from('Acala')
      .to('Quartz')
      .openChannel()
      .maxSize(maxSize)
      .maxMessageSize(maxMsgSize)
      .build()

Close a channel
When opening a new channel the operation has to be specified by calling closeChannel() method and then providing inbound and outbound parameters.
    Builder(api)
      .from('Acala')
      .closeChannel()
      .inbound(inbound)
      .outbound(outbound)
      .build()

Nice job!
Wondering why you need both currency and currencyId?

@vikiival some nodes require asset symbols and some asset ids. This way user can enter ones that their node supports. However this is good point and I am gling to create new issue covering this.

@vikiival some nodes require asset symbols and some asset ids. This way user can enter ones that their node supports. However this is good point and I am going to create new issue covering this.

@vikiival some nodes require asset symbols and some asset ids. This way user can enter ones that their node supports. However this is good point and I am going to create new issue covering this.


Split test.Helper across integration tests and gps

What does this do / why do we need it?
We've noticed that usages of internal/test has snuck into gps's tests because it has some useful helper functions. We want to keep gps separate from dep, not referencing "upwards" into dep.
This allows gps to take advantage of helper methods without bringing in the supporting functions for the integration tests.
What should your reviewer look out for in this PR?
Do you need help or clarification on anything?
internal/gps references internal/fs which references internal/test. There isn't a circular reference but I'm not sure if this is a deal breaker? Are we okay with this one exception?

yeah, the fs reference is the one thing we can accept

so, just thinking - if we do this, we end up having to make the testing interface here part of the public API, once gps itself moves out of internal, right?
that's also icky :(
maybe it's better just to accept this one, additional way that gps reaches out.

Phew okay, I've made codeclimate happy again. Ready for review. 😀

I was thinking that once gps moves out of internal, it should get its own internal and stuff like fs and test will go there, no?

the plan is to keep it in dep, for sure.
yeah, i think that split might be better. let's do internal/test and internal/test/integration. then we can just bite the bullet, and say it's OK for gps to import internal/fs and internal/test, and that's it. 👍

I've split the test package into internal/test and internal/test/integration, and filled in the documentation the integration package.

cool! 👍


Fixes #45848 Long file paths for window.title": "${activeEditorLong}"

This PR fixes issue#45848  and allows users with the setting window.title": "${activeEditorLong}" to view their file long paths in the title bar.
The title bar will also display a relative path for users in linux as per screenshot below.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: Owen-Mak sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@Owen-Mak sorry but I cannot accept that change. The proper fix is to follow what https://github.com/Microsoft/vscode/issues/45848#issuecomment-374516847 suggests.


[rtext] Font glyph images height could be bigger than font size

Font atlas generation assumes that font glyph images height can not be bigger  than font size (assumed as max height) but it seems that's not the case, it should be reviewed.
More detailed info here: https://github.com/raysan5/raylib/issues/2991#issuecomment-1975093390

Indeed if we think of characters that like "g" or "y" or "ç" and so on, they all have dangling content below the baseline. In fact font software is meant to deal with that. If where only about the graphics a simple svg editor would be sufficient enough, but its the baseline and hints, kerning (despite not that interesting for raylib), referenced letters that makes it a font.
LineHeight is already addressed quite flexible by keeping it as global and apply where needed. When a font is loaded the image size of the letters seem already correct. But stitching them together to an Atlas assumed they all are the same height not larger than lineheight leading to choped-off visual.
Can create a simple test_font ttf or otf using the hanging graphics for ya.
Fonts can also be higher above their baseline + lineheight.
Because of that there should be some relative coordinate to which offsetY and respective its origin (0) are relative to.

@designerfuzzi Please, could you provide a font to test this issue, I've been trying to find one for about an hour and I can't find any...

It seems this issue is very difficult to reproduce, closing it. A warning log message was added in case it happens.


Out of memory error on picture page with no category specified on URL

Hi all,
I have two large piwigo installs, one with around 19 thousand images and one with 166 thousand, both running piwigo 2.9.4 and the latest bootstrap_darkroom.
I have found that when requesting an image page without a category in the URL, i.e. "picture.php?/19552" rather than "picture.php?/19552/category/17" I end up with an error similar to as follows:
"Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /config/www/gallery/themes/bootstrap_darkroom/include/themecontroller.php on line 232"
Note this is intermittent on the 19k gallery.
This is the getAllThumbnailsInCategory function which, I'm guessing, attempts to load all the images in the gallery when a category is not specified and runs out of space.
I've also seen the following on the 166k gallery.
"Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /var/www/clients/client1/web16/web/photos/include/dblayer/functions_mysqli.inc.php on line 226"
I believe this to be a symptom of the same issue.
Looking at the function's use, it's added as an event handler in the init function in the same file:
add_event_handler('loc_end_picture', array($this, 'getAllThumbnailsInCategory'));
// also needed on index.tpl for compatibility with GThumb+/GDThumb
add_event_handler('loc_end_index', array($this, 'getAllThumbnailsInCategory'));

I'm guessing (again) that loc_end_picture is where this particular instance of the function is being invoked, and I could remove that add, but I've no idea why the thumbnails are being loaded and what it could break in the process.
Any advice would be appreciated.

This is to be expected, because without category context ALL images are queried for $thumbnails - and this is a pretty heavy operation. $thumbnails is used for PhotoSwipe and Slick carousel.
You could:
a) turn off PhotoSwipe and Slick carousel. It's optional.
b) raise PHP allowed memory until it's enough to run your gallery. I run mine at 512MB, there is nothing wrong about doing this, it just takes that much. If it takes 1024MB, well then that's what it takes.
c) rewrite the whole thing using the web service API and Ajax.

Hi tkuther,
I have Photoswipe and Slick carousel turned off for the moment, it doesn't effect the issue.  I also have  512MB memory limit on the 166k gallery and I'm still hitting the limit and it takes forever to do it and blocks other requests to the gallery (the mysql query appears to block other queries to the image table)

Err, indeed, the query gets executed even if bot is disabled.
Could you please try this patch:
diff --git a/include/themecontroller.php b/include/themecontroller.php
index 8e5b44c..03a5731 100644
--- a/include/themecontroller.php
+++ b/include/themecontroller.php
@@ -31,9 +31,11 @@ class ThemeController {
         add_event_handler('loc_end_picture', array($this, 'registerPictureTemplates'));
         add_event_handler('loc_begin_index_thumbnails', array($this, 'returnPageStart'));

-        add_event_handler('loc_end_picture', array($this, 'getAllThumbnailsInCategory'));
-        // also needed on index.tpl for compatibility with GThumb+/GDThumb
-        add_event_handler('loc_end_index', array($this, 'getAllThumbnailsInCategory'));
+        if ($this->config->slick_enabled === true || $this->config->photoswipe === true) {
+            add_event_handler('loc_end_picture', array($this, 'getAllThumbnailsInCategory'));
+           // also needed on index.tpl for compatibility with GThumb+/GDThumb
+           add_event_handler('loc_end_index', array($this, 'getAllThumbnailsInCategory'));
+        }
     }

     public function assignConfig() {

Hi tkuther,
Yes, that solves the problem when Photoswipe and Slick carousel are disabled, thank you.

I have no errors or out-of-memory - but using a link like this https://www.dhk-galerie.de/dhkfotos/picture.php?/5831 lets piwigo extremly slowing down. The problem for me is the carousel. I have deactivated the carousel and piwigo loads at normal time.
But: I am using piwigo since months and have no problems before. The problems began at the last weeks. The carousel loads the whole gallery, all the pictures - and that slows down the whole site.

I have no errors or out-of-memory - but using a link like this https://www.dhk-galerie.de/dhkfotos/picture.php?/5831 lets piwigo extremly slowing down. The problem for me is the carousel. I have deactivated the carousel and piwigo loads at normal time.
But: I am using piwigo since months and have no problems before. The problems began at the last weeks. The carousel loads the whole gallery, all the pictures - and that slows down the whole site.

It's caused by https://github.com/tkuther/piwigo-bootstrap-darkroom/commit/517ada7694d33fc79f436b01569eb574a2424357 which fixed a valid bug (reported in https://github.com/tkuther/piwigo-bootstrap-darkroom/issues/181#issuecomment-468967203), but has some rather serious side effects. Flat view button can be disabled in Admin -> config -> display -> untick "show all images in all categories", which lowers the risk of being hit.

So this is gonna ever be fixed?
I normally use Modus theme but 2 weaks ago I change to Bootstrap Darkroom. Today I discovered this bug. A webpage that normally took like 0.5 seconds to load, take 50 seconds to load with this theme.

I first believed that is something wrong with my webserver. Then I saw other portions of my site load normally. Then tried to open a picture that have the category ID in the link and worked perfectly. Then switched to Modus theme. Modus theme work perfect with both category ID and without it. So the problem is the theme not the webserver or Piwigo. Made a google search and found this 2 year old report.
I have pictures links that have posted in forums and social networks. The reason links are posted like this
https://website.com/picture.php?/12474 and not like this https://website.com/picture.php?/12474/category/223 is because first type of link is more reliable on the long term. For example the second type of link will be broken if in 1 year I decide to change the structure of the pictures/albums and move that picture ID in another album ID. So you need to stay with that album structure forever.
The fact picture pages cannot load without a category ID make this (great) theme unusable.

We still have the problem with Bootstrap Darkroom 2.5.1.

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in include/functions_url.inc.php on line 175

The only solution right now is to disable Photoswipe and SlickCarousel.
We could automatically deactivate (with no persistence) these features if there are more than X photos in the current set.


AC-6: Add a Role

AC-6: Add a Role
User Story
AS A business owner
I WANT to be able to view and manage the departments, roles, and employees in my company
SO THAT I can organize and plan my business
Acceptance Criteria
GIVEN a command-line application that accepts user input
WHEN I choose to add a role
THEN I am prompted to enter the name, salary, and department for the role and that role is added to the database.

To allow the user to select a department from the existing departments when adding a role:

Retrieve the list of departments from the database.
Present this list to the user as choices in the prompt.
Use the selected department's name to determine the department ID for the role.

Retrieve Departments:
Fetch all departments from the database using department.findAll().
Format Choices:
Map the departments to an array of objects where the name is the department name shown to the user and value is the department ID.
Prompt User:
Use inquirer.prompt with the list type for the department selection. The user will see the department names, but the value will be the corresponding department ID.
Create Role:
Use the selected departmentId to create the new role with the appropriate department ID.


feat: Auth routes3

[x] Added Get Route for Likes
[x] Added Get Route for Likes
[ ] Added Get Route for All Users

Codecov Report

Merging #53 (c6a216b) into dev (37faa7c) will decrease coverage by 32.08%.
The diff coverage is 53.40%.

@@             Coverage Diff             @@
##              dev      #53       +/-   ##
===========================================
- Coverage   85.71%   53.63%   -32.09%
===========================================
  Files           1        8        +7
  Lines           7      179      +172
  Branches        0       22       +22
===========================================
+ Hits            6       96       +90
- Misses          1       74       +73
- Partials        0        9        +9

Impacted Files
Coverage Δ

src/middleware/bearer.js
15.38% <15.38%> (ø)

src/routes/user.js
21.56% <21.56%> (ø)

src/handlers/not-found.js
25.00% <25.00%> (ø)

src/models/user.js
34.37% <34.37%> (ø)

src/models/index.js
81.81% <81.81%> (ø)

src/handlers/error.js
83.33% <83.33%> (ø)

src/routes/public.js
90.69% <90.69%> (ø)

src/server.js
94.73% <100.00%> (+9.02%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 896032c...c6a216b. Read the comment docs.


password is not meeting the standard

when register user, password is not meeting the standard, must be at least 8 characters and include at least one upper and lower case letters.

added password validator in user schema


oragono.io/joined

When a client connects to an always-on Oragono server (or just a bouncer like ZNC), they get a bunch of JOIN messages - one for every channel they're already connected to. Clients typically display this as a separate "you just joined XYZ channel" line in their scrollback, but really it's "you just reconnected to the server" and not really a you-joined-the-channel thing.
If you have this cap active, instead of being sent a JOIN messages you'll either just have your nickname in RPL_NAMREPLY to indicate that you're already joined to the channel, or maybe get a JOINED verb instead. idk which one feels better.
suggested by @kylef

ZNC also allows you to 'detach' a channel from a client, that allows you to have channels only in specific clients or to leave it to save bandwidth. So there may also be "PART" in that case is not impacting the presence/state of channel membership (https://wiki.znc.in/Detaching). There are also modules to automatically reattach based on mentions or keywords used.
Not sure if we want to have this extension for both JOINs and PARTs or not, but sharing that additional case (ATTACH/DETACH).

How's something along these lines look?
oragono.io/joined Extension

This capability allows servers to express to clients that they are already joined to channels,
(as opposed to having just joined now). This is useful for persistent connections, for example
those provided by bouncers or special persistence features built into the server.

When this capability is requested, JOIN messages may be replaced with JOINED messages. The
JOINED message indicates that the client is already joined to the channel.

    :nick!user@host JOINED #channelname accountname +|- :Real Name

This message shows that the client nick!user@host has joined the channel #channelname, with the
realname/gecos "Real Name". If they are logged in, 'accountname' will be their account name, and
otherwise this parameter will be '*'. The third parameter represents whether or not this channel
is 'detached' (connected on the server but not actively shown to the client), in standard use
this will be '+', and if the channel has been 'detached' then this will be '-'.

## Examples

Client 'dan' being shown they are joined to a channel.

<EMAIL_ADDRESS>JOINED #ircv3 dan + :Hello there, I like making IRC software!

Client 'dan' being shown they are joined to a channel (but it's detached).

<EMAIL_ADDRESS>JOINED #tacos dan - :Hello there, I like making IRC software!

**Versioning Considerations**
If any other information needs to be conveyed with JOINED messages in the future, we can always
add tags to that message (those tags will be defined in and added to this specification).


Bug 1876852: ceph: fix to support TLS enabled rgw-endpoint

Backport PR to support TLS encrypted rgw endpoint
Signed-off-by: Arun Kumar Mohan<EMAIL_ADDRESS>(cherry picked from commit 3de87ccd3534f5b9fe0f1a3958b071cf8adb839a)

Description of your changes:
Which issue is resolved by this Pull Request:
Resolves #
Checklist:

[ ] Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
[ ] Skip Tests for Docs: Add the flag for skipping the build if this is only a documentation change. See here for the flag.
[ ] Skip Unrelated Tests: Add a flag to run tests for a specific storage provider. See test options.
[ ] Reviewed the developer guide on Submitting a Pull Request
[ ] Documentation has been updated, if necessary.
[ ] Unit tests have been added, if necessary.
[ ] Integration tests have been added, if necessary.
[ ] Pending release notes updated with breaking and/or notable changes, if necessary.
[ ] Upgrade from previous release is tested and upgrade user guide is updated, if necessary.
[ ] Code generation (make codegen) has been run to update object specifications, if necessary.

cc: @leseb
Please take a look


Older generation VMs support

Hi,
Regarding older generation VMs, is there a support end date for these types of machines?
I'm considering using an A0\Basic_A0 machine due to better pricing fit and want to verify its lifecycle support will not change my decision
Thanks,
Assaf

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 616242df-d3da-e9b3-73b0-fba7ae7a28a7
Version Independent ID: 33587769-6f9d-af7c-3fc1-08676adc860b
Content: Azure VM sizes - previous generations - Azure Virtual Machines
Content Source: articles/virtual-machines/sizes-previous-gen.md
Service: virtual-machines
Sub-service: sizes
GitHub Login: @mimckitt
Microsoft Alias: mimckitt

@assaf-grth, Thanks for the question! We are taking a look into this and will get back to you soon.

@mimckitt Can you please add your comments on this.

@assaf-grth
It is not recommended using Av1 VMs as its old generation.
Please check Av2 VMs, as they come with better price/performance.
I checked with internal team on end of support for the older generation models. Currently there is no timeline decided on this. Once this is decided it will be available on public documentation.
Hope this information is helpful.

Price differences between lower end of both generation are high - 13$ - 30$
I require a VM with half of the offered RAM, could there be a plan to include such offering in the future?

@assaf-grth I would request you to provide this feature request as feedback here.
And regarding details on the older gen the information is available on the public documentation.
Also, product team wanted some more information on your workload considering inclination towards these Av1 VMs.
Can you please share more details.

Hey @assaf-grth, the key difference between Av1 and Av2 is the performance and HDD vs SSD storage offering. Av1 VMs today come under the "Previous Generation VM" category, their design is ~8 years old. Av2 on the other hand comes with a better price-to-performace. Hope this answers your concerns.

My requirement is to have a wordpress site at the minimal cost available
At AWS there is an offer of 5$ (https://aws.amazon.com/lightsail/pricing/) which is 6 times under the minimum price in Azure (A1 v2)

Thanks for sharing this information. Given that the workload is smaller, have you considered using B series VMs ? B series will give you better pricing in this usecase.

@assaf-grth Any update on the issue?
As suggested by @rishabv90 please check the B series VM's for your workloads.

@assaf-grth
Hope the provided information is helpful.
We will now close this issue. If there are further questions regarding this, please tag me in a comment. I will reopen it and we will continue the discussion.

Hi @VikasPullagura-MSFT,
I checked the B series and since we need an operating website without known activity hours in advance, there is a risk of not having enough credits and the site being unresponsive
Is using B series VMs for wordpress websites an official recommendation?

@rishabv90 Any comments on this.

@rishabv90 Any comments on this.

Yes, given the price constraints,  B series is the official recommendation as it's our entry-level offering.

Yes, given the price constraints,  B series is the official recommendation as it's our entry-level offering.


Major documentation update

Added attribution page
Updated user guide to include details about CLIs and Payload Templates
Updated Configuration docs to include details about Slack and Secrets Manager
Also updated the GitHubSyncWorkerShip to upload objects with the bucket-owner-full-control canned-acl
Upgraded to Python 3.10
Some small Python fixes too

Docs has been pushed out. This fixes #19


EmbeddedBotHandler: Make sure the interface is in sync with ExternalBotHandler.

We strive to keep the ExternalBotHandler interface in zulip/python-zulip-api in sync with EmbeddedBotHandler in zulip/zulip. Over time, it looks like the two have drifted apart and ExternalBotHandler now supports a method bot_handler.update_message.
We should do a sweep and implement the missing methods in EmbeddedBotHandler.
We should also discuss strategies to prevent this from happening again, if possible.
ExternalBotHandler: https://github.com/zulip/python-zulip-api/blob/d8157d6b80203ab4e1cf31359b3ad939be4c8e5d/zulip_bots/zulip_bots/lib.py#L106

@PIG208 FYI -- is this resolved by your recent PR?  If so I think this issue is a duplicate and am going to close it under that assumption.


[ALLUXIO-3179] Fix docker empty dir

https://alluxio.atlassian.net/browse/ALLUXIO-3179

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/19411/
Test PASSed.

Merged build finished. Test PASSed.


Remove chrono, use time directly, due to RUSTSEC-2020-0159

Motivation
There is an open security issues for chrono that has been open since 2020:

https://rustsec.org/advisories/RUSTSEC-2020-0159

Per the discussion on the chrono issue about the vulnerability, the recommended approach is to replace chrono with time 0.3:
https://github.com/chronotope/chrono/issues/602
In tiny-http the dependency is nicely encapsulated, so no users of tiny-http should be affected by this change.
Changes

Replace the chrono dependency with time 0.3.

Tests
Existing tests pass, no new tests have been added.
The formatting can be compared on the rust playground with:
use time::{format_description, OffsetDateTime};
use chrono::{DateTime, Utc};

fn main() {
    {
      let format: Vec<format_description::FormatItem<'static>> = format_description::parse("%a, %e %b %Y %H:%M:%S GMT").unwrap();
      let date = OffsetDateTime::now_utc();
      println!("date:   {}", date.format(&format).unwrap());
    }
    {
        let date = Utc::now();
        println!("chrono: {}", date.format("%a, %e %b %Y %H:%M:%S GMT").to_string());
    }
}

This does currently NOT match, so this must be resolved.

The format string required to construct an RFC5322 date-time string appears to be:
[weekday repr:short], [day padding:space] [month repr:short] [year repr:full] [hour repr:24 padding:zero]:[minute padding:zero]:[second padding:zero] GMT

See a playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=192c3d87438cbe90eff5ed86162502cc
Rather than express the time zone as an offset using the formatter we have to include the string GMT as required by RFC 7231.
I think it should be possible to construct the format specification once rather than on every call to to_string, I'll pull your branch and see if it can be done either in a const context or with a static at the very least.

@bitdivine I've incorporated your commit into a branch: https://github.com/tiny-http/tiny-http/compare/b-nuke-chrono
Can you take a quick look and make sure it matches what you expected a solution to look like?


[T3A2][T15-A2] Oguz Yuksek

@yuksek  Some comments added. Please close the PR after reading comments.


update(docs): more detail about container runtime populated k8s fields

Signed-off-by: Vicente J. Jiménez Miras<EMAIL_ADDRESS>
What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind user-interface

/kind content

/kind translation

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area blog

/area documentation

/area videos

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: vjjmiras
Once this PR has been reviewed and has the lgtm label, please assign leogr for approval by writing /assign @leogr in a comment. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Thanks a bunch 🙏 ! Great concise changes!

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jasondellaluce, vjjmiras
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: c8ec1bcc4075c3461fc0df3cef0dfda5764e5975


Files comprising spotdata routine.

Includes unit tests for neighbour finding and data extraction routines. Pylint-ed (except where it disagrees with pep8). Pep8-ed.
Instructions on the ticket outline how to run this code to produce output. See me with any queries.
There are just over 3000 lines of code and comments in this commit, so we will need to break this down. However, the routines are all fairly inter dependent, so I have created the pull request in a complete form so it can be run.
Reference issue if exists
Testing:

[OK ] Ran tests and they passed OK (if environment variables defined as required).
[OK ] Added new tests for the new feature(s)

I think the first thing that should be reviewed is the new plugins and the unit tests for these. These should be independent of anything else. Should you make a separate pr, just containing the plugins and unit tests?


Do you want to have the enhancement of existing game ? 😀 Describe yourself..
The score for both the players is covering some part of the court, furthermore, the score below the court looks ugly.

Describe the solution you'd like
I will do the following:

Remove the scores on top
Make the scores below, as standalone numbers, for better visual

Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @VijaySamant4368 , You can't have another issue before completing the previous one 😀
you already have the following 1 open issues 👀 ! :
#3327

Hey @VijaySamant4368 ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇

Hey @VijaySamant4368 !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

/assign


finals aren't handled properly

I think the current final handling is more of a workaround than a real solution, leading to issues like this:
class Main {
	final foo:Int = 1;
	static function main() {}
}

becomes
class Main {final foo: Int = 1;
	static function main() {}}

There's two issues there: a space is inserted after the : and there's no line breaks.

shouldn't this be final var foo?

No. final replaces var.

ohhhh nooo.... that's unfortunate, especially for checkstyle

How so? :)

everything that looks for vars now also has to look for finals

Don't forget that final is also a valid modifier for methods, so you need to make sure not to confuse the two. :D


Move docker-multinode to this project

Ref: kubernetes/kubernetes#21646
@mikedanese @fgrzadkowski @vishh @cheld @janetkuo @justinsb

SGTM

CC @zreigz, @baticanu

@mikedanese Yes, in a few days I'll have one.

@luxas Isn't kube-deploy community maintained? But the table of solutions says Docker multinode is supported by the project?

@jimmyjones2 It was Brendan's project, but he's not actively working on it anymore, so now the community maintains it.
But anyway, the line between community/k8s supported and this project isn't so sharp I think, we just want good k8s deployment code to kube-deploy

Kubernetes is maintained by the community :) I agree that this is a good spot for it. It's at least better than cluster/.

Kubernetes is maintained by the community :)

Yes, of course :smile:

The PR is merged, I'm closing it


[1.16.0] CIDRGroupRef is breaking sanitization and not applying correctly

Is there an existing issue for this?

[X] I have searched the existing issues

Version
higher than v1.16.0 and lower than v1.17.0
What happened?
We are utilizing CiliumCIDRGroups.
When we create a CNP or CCNP (it happens for both), the status changes to an error message
status:
  conditions:
  - lastTransitionTime: "2024-07-26T13:24:29Z"
    message: 'Unable to parse CIDRRule "": netip.ParsePrefix(""): no ''/'''
    status: "False"
    type: Valid

I searched the code and found out that sanitize is the problem here and I think that the cidrGroupRef is not translated correctly into a list of IPs before it is sanitized.
This should be tested somewhere IMO, but this is a different topic.
cilium-dbg policy get is showing the correctly translated IPs for any C(C)NP though. I would expect that the agent also applies the correct policy if it knows about the translated list of IPs, but this seems to not be the case. It seems to not apply the policy at all.
Setting the default deny rule to allow all ingress traffic by default (rendering the CCNP effectively useless) works as it should, even though the status still shows the CCNP to be invalid.
Can you give us a short explanation, what we could be doing wrong, if you think this is on us? Or please investigate in the code, how to fix this problem.
Remark: It is not just an appearance problem, but the firewall is not applied as it should be.
How can we reproduce the issue?

Apply the following in a cluster:

apiVersion: cilium.io/v2alpha1
kind: CiliumCIDRGroup
metadata:
  name: test-cidrgroup
spec:
  externalCIDRs:
  - <IP_ADDRESS>/32
---
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: test-fw
spec:
  nodeSelector: {}
  ingress:
  - fromCIDRSet:
    - cidrGroupRef: test-cidrgroup

Check the status of the test-fw CCNP -> It is marked as invalid.
Check the cilium-dbg output -> It correctly translates the CCG to a list of IPs.
Check the firewall access via telnet for example -> It is not applying correctly.
If you set the spec of the CCNP to the following:

spec:
  ingress:
  - fromCIDRSet:
    - cidr: <IP_ADDRESS>/32

it applies and the status is correctly set.
Cilium Version
Client: 1.16.0 82999990 2024-07-23T22:22:14-07:00 go version go1.22.5 linux/amd64
Daemon: 1.16.0 82999990 2024-07-23T22:22:14-07:00 go version go1.22.5 linux/amd64
Kernel Version
Linux $node 6.5.0-45-generic #45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Jul 15 16:40:02 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
Kubernetes Version
v1.30.3
Regression
It worked on 1.15.6.
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Cilium Users Document

[ ] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi @jonasbadstuebner, I took a brief look at this, here's my findings:
The error you're seeing in the status of the CCNP stems from the newly introduced policy validation which happens centrally, in the operator. It's unclear whether a (C)CNP referencing a non-existing/not yet existing CiliumCIDRGroup should be considered valid, but in any case it seems that the code does not take into account the resolving of the reference, which appears to be a bug. (cc @squeed as the author of that feature)
However, it's unclear to me as of now why the policy would not take effect - the plumbing of the policy is separate from the validation and status, hence I'm confused as to why it wouldn't work. The logic to import policy with group references seems to correctly ensure that if the policy is imported before the group exists, it is reimportet once the group it references is added - if you are adding the policy first, do you see two log messages Imported CiliumNetworkPolicy with ciliumNetworkPolicyName=test-fw in your reproduction? That would indicate that the policy was re-imported correctly once the CCG was added.

Hey @bimmlerd, thank you for your reply.
The fix that @chaunceyjiang is providing in #34076 should fix the first issue.
We migrated to 1.16.0 and reading the changelog and migration guide did not help us in figuring out why the main CCNP we apply did not work as before. Traffic between Nodes is blocked. I will try to investigate a little more and let you know about my findings.
I will try to apply the CCNP without referencing the CCGs, to take them out of the equation.
It is possible, that some other change happened, that we are unaware of, that makes our CCNP not apply as it should.
Because when I enable default allow (/disable default deny) on ingress, it works. Even with the CCNP referencing the CCGs. But this also means, I have effectively no firewall, so it is not an option.
I think I saw the "imported CiliumNetworkPolicy" message. But I will double check.

Please reopen.
This is not yet completed. Only the sanitization should work after the next release, but the Policy still does not apply as expected.

@jonasbadstuebner I couldn't repro the policy not applying, that's why I asked specifically for details in that part:

When you are add the policy first and then the group, do you see two log messages Imported CiliumNetworkPolicy with ciliumNetworkPolicyName=test-fw in your reproduction? That would indicate that the policy was re-imported correctly once the CCG was added.

This happens for me, and it would help with debugging to understand whether this does not happen for you, or whether the repro needs to be different.

I can see the Policy imported via API, recalculating... followed by a Imported CiliumNetworkPolicy.
In the cilium logs I see
Policy verdict log: flow 0x7dae9598 local EP ID 219, remote ID remote-node, proto 6, ingress, action deny, auth: disabled, match none, [ff78:abd1:2561:1:abcd::12]:33498 -> [ff78:abd1:2561:1:abcd::13]:4240 tcp SYN

even though I had
  - fromCIDRSet:
    - cidr: <IP_ADDRESS>/0
    - cidr: ::0/0
    toPorts:
    - ports:
      - port: "4240"
        protocol: TCP

applied.

@jonasbadstuebner I'm a bit confused; it now looks like you're specifying CIDRs directly, rather than by cidrGroupRef. The issue as described is about a bug with validation and cidrGroupRef. Is something else broken?
It also looks like you're trying to grant access to pods via CIDRs, which won't work. CIDRs can't select in-cluster entities.

Should I open a new issue? And cross-link this one?

@jonasbadstuebner yes, please open a new issue. The hubble flow you posted says the peer traffic is from the remote-node identity, which should be included in the cluster entity. And, of course, it never hurts to check that the policy matches your understanding in the editor: https://editor.networkpolicy.io

I will open a new issue for this then. And include all the information I think could be relevant.


Seeing the code-input rendering data in two different modes

I want to use the syntax highlighter for my editor, and  see the content only with syntax highlighted and in editable mode. But currently I see it in two different modes like below.
Is there any possibility to do it that way?

I am using this in VueJS and this is the code I have right now.
<template>
  <div ref="htmlEditor" class="html-editor">
    <textarea
      ref="editor"
      :value="value"
      @input="handleInput"
      is="code-input"
      data-language="html"
      :data-readonly="disabled"
      spellcheck="false"
    ></textarea>
  </div>
</template>

<script>
import Prism from 'prismjs';

export default {
  props: {
    value: { type: String, default: '' },
    language: { type: String, default: 'html' },
    disabled: Boolean,
  },

  methods: {
    handleInput(event) {
      this.$emit('input', event.target.value);
    },

    initializeEditor() {
      const textarea = this.$refs.editor;
      textarea.setAttribute('is', 'code-input');
      textarea.setAttribute('data-language', this.language);

      // Register Prism for syntax highlighting if needed
      if (window.codeInput) {
        window.codeInput.registerTemplate(
          'syntax-highlighted',
          window.codeInput.templates.prism(Prism, [])
        );
        window.codeInput.setDefaultTemplate('syntax-highlighted');
      }
    },
  },

  mounted() {
    this.initializeEditor();
  },
};
</script>

<style>
/* Hide the non-editable preview content */
.code-input pre[aria-hidden="true"] {
  display: none !important;
}

/* Additional styling */
.html-editor {
  width: 100%;
  position: relative;
}

.html-editor textarea {
  font-size: 15px;
  min-height: 200px;
  width: 100%;
  padding: 8px;
  border: none;
  resize: none;
}

.token.tag { font-weight: bold; }
.token.attr-name { color: #111; }
.token.attr-value { color: #0066cc; }

.html-editor textarea:focus {
  outline: none;
  border-color: #0066cc;
}
</style>

Thanks for asking here, @Bowrna !
General Problems
There are several problems in this code (perhaps from my unclear documentation) that are making it not work. Please note that I know very little about Vue except that it is very similar to vanilla HTML/CSS/JavaScript so you may have problems in Vue code I haven't identified:

You are not using code-input in the correct way: you are using a <textarea is="code-input"> element while the code-input library expects a <code-input> element, and does not expect any elements to be inside this element initially (just the initial text value). You can use the input event and value property of the <code-input> element instead of that of a textarea, and if you really need to access the <textarea> once code-input has created it you can use (code-input element).getElementByTagName("textarea").
I'm also unsure of the purpose of the template element: code-input doesn't require it (maybe it's a Vue thing) and code-input.registerTemplate("syntax-highlighted", codeInput.templates.Prism(Prism, []) will set up all <code-input template="syntax-highlighted"> elements, with template being an HTML attribute not an element.
The data-language attribute should be language without the "data-" part.
Your textarea and token CSS will also be unnecessary when you import code-input and Prism.JS' CSS. You must import code-input's CSS for the editor to  work, but Prism.JS' CSS doesn't need to be imported if you style its tokens yourself.

For more information, you should look at the demo code (vanilla HTML/CSS/JS), which strangely doesn't seem to have Prism highlighting work now - the more complicated but similar demo does. If needed, read through the README documentation and follow all of the steps that apply to you, especially the "Using the component" section. My labelling of the CSS-Tricks article will not be helpful since it explains the behind the scenes functioning but not the library usage - I have added a clarification to the documentation.
Changing language
You can use (code-input element).setAttribute("language", "HTML"), (code-input element).setAttribute("language", "Markdown"), and (code-input element).setAttribute("language", "plaintext"), and it should hopefully work.
Please let me know if you have any further questions / it still doesn't work.

Thanks for asking here, @Bowrna !
General Problems
There are several problems in this code (perhaps from my unclear documentation) that are making it not work. Please note that I know very little about Vue except that it is very similar to vanilla HTML/CSS/JavaScript so you may have problems in Vue code I haven't identified:

You are not using code-input in the correct way: you are using a <textarea is="code-input"> element while the code-input library expects a <code-input> element, and does not expect any elements to be inside this element initially (just the initial text value). You can use the input event and value property of the <code-input> element instead of that of a textarea, and if you really need to access the <textarea> once code-input has created it you can use (code-input element).getElementByTagName("textarea").
I'm also unsure of the purpose of the template element: code-input doesn't require it (maybe it's a Vue thing) and code-input.registerTemplate("syntax-highlighted", codeInput.templates.Prism(Prism, []) will set up all <code-input template="syntax-highlighted"> elements, with template being an HTML attribute not an element.
The data-language attribute should be language without the "data-" part.
Your textarea and token CSS will also be unnecessary when you import code-input and Prism.JS' CSS. You must import code-input's CSS for the editor to  work, but Prism.JS' CSS doesn't need to be imported if you style its tokens yourself.

For more information, you should look at the demo code (vanilla HTML/CSS/JS), which strangely doesn't seem to have Prism highlighting work now - the more complicated but similar demo does. If needed, read through the README documentation and follow all of the steps that apply to you, especially the "Using the component" section. My labelling of the CSS-Tricks article will not be helpful since it explains the behind the scenes functioning but not the library usage - I have added a clarification to the documentation.
Changing language
You can use (code-input element).setAttribute("language", "HTML"), (code-input element).setAttribute("language", "Markdown"), and (code-input element).setAttribute("language", "plaintext"), and it should hopefully work.
Please let me know if you have any further questions / it still doesn't work.

Anyone who knows Vue better, please feel free to help!