Serilog.Sink.Elasticsearch, Could not create SSL/TLS secure channel

I am using Serilog and Serilog.Sink.Elasticsearch in an ASP web API project, there is an issue with the connection to the ELK service that has been configured on an azure Kubernetes cluster, I get the error below when I deploy my code on the windows server 2012 R2, but it works on my desktop correctly, and SSL has been issued by LetsEncrypt and expiration time is fine
packages:
<package id="Microsoft.AspNet.WebApi" version="5.2.3" targetFramework="net452" />
<package id="Serilog" version="2.10.0" targetFramework="net472" />
<package id="Elasticsearch.Net" version="7.17.0" targetFramework="net472" />
<package id="Serilog.Sinks.Elasticsearch" version="8.4.1" targetFramework="net472" />

error:

2022-04-06T11:49:33.7509141Z Caught exception while preforming bulk operation to Elasticsearch: Elasticsearch.Net.ElasticsearchClientException: The request was aborted: Could not create SSL/TLS secure channel.. Call: Status code unknown from: POST /_bulk ---> System.Net.WebException: The request was aborted: Could not create SSL/TLS secure channel.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elasticsearch.Net.HttpWebRequestConnection.d__5`1.MoveNext()
--- End of inner exception stack trace ---
at Elasticsearch.Net.Transport`1.HandleElasticsearchClientException(RequestData data, Exception clientException, IElasticsearchResponse response)
at Elasticsearch.Net.Transport1.FinalizeResponse[TResponse](RequestData requestData, IRequestPipeline pipeline, List1 seenExceptions, TResponse response)
at Elasticsearch.Net.Transport1.d__151.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Serilog.Sinks.Elasticsearch.ElasticsearchSink.d__2.MoveNext()

I've added these lines to Application_Start but it doesn't work me:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls | System.Net.SecurityProtocolType.Tls11 | System.Net.SecurityProtocolType.Tls12;
            ServicePointManager.Expect100Continue = true;

Any thoughts?

Hi, @Mehranh. Serilog.Sink.Elasticsearch is a third-party package that we (Elastic) do not directly support. Any issues with that library need to be raised in its repository as a starting point.
That said, this appears to be a TLS error from the low-level client so I will try to help. It appears that the client and server are unable to negotiate TLS. Your server is quite old and may not have the appropriate TLS version(s) enabled (see this article I found for TLS 1.2). It also does not support TLS 1.3 and newer cypher suites. You will need to begin by verifying that the server running your client application and Elasticsearch support the same TLS requirements.
What is your Elasticsearch server version?
Which version of the .NET Framework runtime is installed on your server?

Hi again @Mehranh. Just checking in to see if this is still and issue for you and whether you can provide more details to assist further?

Hi ,
I resolved the issue by configuring the ingress to support the legacy TTLS , you can use this ConfigMap :
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
data:
  ssl-ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA"
  ssl-protocols: "TLSv1 TLSv1.1 TLSv1.2 TLSv1.3"

for more information please follow this link


Add special column handling of NWIS site

Site 01434498 has odd column names. We have been handling these sites manually in the get_daily_nwis_data function.
colnames(site_data)
[1] "agency_cd" "site_no" "Date" "Value_Max"
[5] "Value_Max_cd" "..4.._Value_Max" "..4.._Value_Max_cd" "..2.._Value_Max"
[9] "..2.._Value_Max_cd" "..3.._Value_Max" "..3.._Value_Max_cd" "..4.._Value"
[13] "..4.._Value_cd" "..2.._Value" "..2.._Value_cd" "..3.._Value"
[17] "..3.._Value_cd" "Value" "Value_cd"

I see that this site has special handling applied in get_inst_nwis_data.R. So perhaps they've just started reporting the daily values for that site too?

That's probably what happened! I'll take care of adding the special handling to the daily data.


appimage: switch to gst-plugins-base 1.0

Motivation for this change
gst-plugins-base 0.10 is depricated and marked as broken as of https://github.com/NixOS/nixpkgs/commit/d034a0039d8937aca2f64d1d3911e8b71f416137.
Not sure if we should delete GStreamer 0.10 as well, as version 1.0 is already in the package. (https://github.com/NixOS/nixpkgs/issues/39975)
fixes: #93174
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"

     19 packages built:
     Sylk appimage-run deltachat-electron devdocs-desktop irccloud joplin-desktop ledger-live-desktop
     marktext minetime notable obsidian runwayml ssb-patchwork standardnotes station tusk unityhub
     wootility zulip

[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 93398 1

19 packages built:
- Sylk
- appimage-run
- deltachat-electron
- devdocs-desktop
- irccloud
- joplin-desktop
- ledger-live-desktop
- marktext
- minetime
- notable
- obsidian
- runwayml
- ssb-patchwork
- standardnotes
- station
- tusk
- unityhub
- wootility
- zulip

Result of nixpkgs-review pr 93398 1

19 packages built:
- Sylk
- appimage-run
- deltachat-electron
- devdocs-desktop
- irccloud
- joplin-desktop
- ledger-live-desktop
- marktext
- minetime
- notable
- obsidian
- runwayml
- ssb-patchwork
- standardnotes
- station
- tusk
- unityhub
- wootility
- zulip


Various bug fixes

Issue #, if available:
Description of changes:

Various bug fixes (for example, in kernel computation code)
Simplifications and small extensions

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Job PR-849-2 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-849/2/index.html

Job PR-849-2 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-849/2/index.html

Thanks @sxjscience ! I will add some docstrings. I am currently working on a larger topic, for which some of these changes here are needed, but they are in general bug fixes and simplifications.

Thanks @sxjscience ! I will add some docstrings. I am currently working on a larger topic, for which some of these changes here are needed, but they are in general bug fixes and simplifications.

Job PR-849-3 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-849/3/index.html

Job PR-849-3 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-849/3/index.html


Dimension mismatch when using --prefer_dedicated_fmaps

I'm getting a dimension mismatch error when I try to use --prefer_dedicated_fmaps.
Here is my call to qsiprep:
singularity run --cleanenv /scratch/users/ethanroy/singularity_images/qsiprep-0.18.1.sif \
/scratch/groups/jyeatman/ethan/input_test_fmap_reorg \
/scratch/groups/jyeatman/ethan/input_test_fmap_reorg/derivatives participant --participant-label sub-LD4138ses3 \
--prefer_dedicated_fmaps \
-w /scratch/groups/jyeatman/ethan/work \
--output-resolution 1.5

And here is the stack trace:
RuntimeError: Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node fmap2ref_apply.

Cmdline:
	antsApplyTransforms --default-value 0 --dimensionality 3 --float 1 --input /scratch/groups/jyeatman/ethan/work/qsiprep_wf/single_subject_LD4138ses3_wf/dwi_preproc_acq_60_wf/hmc_sdc_wf/sdc_wf/fmap_wf/applymsk/sub-LD4138ses3_acq-60_fieldmap_ras_rad_unwrapped_hz_filt_demean_maths_masked.nii.gz --interpolation BSpline --output sub-LD4138ses3_acq-60_fieldmap_ras_rad_unwrapped_hz_filt_demean_maths_masked_trans.nii.gz --reference-image /scratch/groups/jyeatman/ethan/work/qsiprep_wf/single_subject_LD4138ses3_wf/dwi_preproc_acq_60_wf/hmc_sdc_wf/b0_ref_to_lps/topup_imain_LPS.nii.gz --transform /scratch/groups/jyeatman/ethan/work/qsiprep_wf/single_subject_LD4138ses3_wf/dwi_preproc_acq_60_wf/hmc_sdc_wf/sdc_wf/sdc_unwarp_wf/fmap2ref_reg/transformComposite.h5
Stdout:

Stderr:

Traceback:
	Traceback (most recent call last):
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 398, in run
	    runtime = self._post_run_hook(runtime)
	  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/niworkflows/interfaces/registration.py", line 106, in _post_run_hook
	    return super(ANTSApplyTransformsRPT, self)._post_run_hook(runtime)
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/mixins/reporting.py", line 50, in _post_run_hook
	    self._generate_report()
	  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/niworkflows/interfaces/report_base.py", line 61, in _generate_report
	    cuts = cuts_from_bbox(mask_nii, cuts=n_cuts)
	  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/niworkflows/viz/utils.py", line 215, in cuts_from_bbox
	    ras_coords.append(apply_affine(mask_nii.affine, cross).tolist())
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nibabel/affines.py", line 97, in apply_affine
	    pts = pts @ rzs.T + trans[None, :]
	ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 3 is different from 4)

The prefer-dedicated-fmaps method is deprecated. Just curious, why are you preferring dedicated fmaps instead of using data from dwi series?

Do the two dwi series have different PhaseEncodingDirections specified in their sidecars?

Yes based on the sidecars it looks like the PhaseEncodingDirections are different across the two series

Strange update: I just tried running the same call to qsiprep as above (minus the prefer-dedicated-fmaps flag) but using an older version of qsiprep (qsiprep-0.16.1) and things ran smoothly and included the expected distortion correction in the output...I think for the time being I can proceed with the older version of qsiprep but thought I'd flag that for some reason the distortion correction is working with the older version but not the new one.

Would you be able to share this dataset with me?

I just heard back from our collaborators and unfortunately I don't think I can share the dataset with you based on our data usage agreement.

Hi @earoy and @mattcieslak,
I encountered this same error on some older dataset while using version 0.19.1. Matt I will reach out to you to share the data.

Any updates on this issue?
I am getting the same error with the latest version of qsiprep (0.19.1 docker). I checked the affine matrices of my input dwi and all my fmap (2 phase and 2 magnitude) images with nibabel and they match perfectly.
I also ran the same preprocessing on an older version of qsiprep (0.16.1) based on earoy's comment. It completed without error, but there seems to be no difference between the images pre and post SDC visually based on the subject's .html report. The fieldmap it generated looks good though (resembles the fieldmap generated by FSL).


SMA to collect CPU metrics (Edinburgh Roadmap Item)

the System Management Agent and associated system management API will add the ability to track service CPU usage and metrics.

Note that if https://bitbucket.org/bertimus9/systemstat is to be used for tracking memory usage, we will need to request additional permissions for the snap in order for the snap to be able to read /proc/meminfo. This will probably be added to system-observe, which we already use, but that's not totally clear right now.


[1.9.4] Inventory Panel Issue(s)

Inventory Panel now support JEI's "+" integration however it doesn't detect if you actually have the items, would it be possible to integrate that too?

[CLOSED] Inventory Panel now supports Storage Drawers, thank you!

Inventory Panel does not keep the sorting preference in mind:  https://youtu.be/ZCcOS7_l73c

Inventory Panel behaves badly when shift-clicking "+" in JEI: https://youtu.be/BGNW0Jjzmu0 (may be linked to #3421)

Inventory Panel will not refill some recipes, even if it has the items and they stack: https://youtu.be/XtPrxf1VsK8

Withdrawing a stack of items from the Inventory Panel will cause the rest of the items to disappear, being rescanned? And you will have to wait until it's finished to pick-up the next stack, making large withdraws annoying:  https://youtu.be/hT8VQ13S8rY (may be linked to #3421)

Updated with 3 new issues (4, 5 and 6).

Are there actually multiple stacks of diamonds and obsidian available?

Yes


Modify KVM driver to create/use created network overlay

Allow the creation of the network using static MAC address and a fixed IP address.
So at this moment we assume the VM will allows uses the same MAC address and the expected IP address is <IP_ADDRESS>.

Note, this might conflict with the way we expect the Pre-flight code to 'create' the network, but we might have to move this code to a temporary shared part of the library and 're-use' this inside the driver, until we can figure out a ncie way to have the pre-flight/setup code to trigger tthis as part of the driver itself (future enhancement)

Driver is now using crc as default network, but it is created by the pre-flight check.

Fixed by #63


Continue work in  Dashcam Viewer support #155

Tasks:

[x] 1 - Implement latest bootstrap 5.3
[x] 2 - Add ability to view screen records
[x] 3 - Improve design
[x] 4 - Tests

Thanks for the PR! 🎉


Added link to Webservices guide to Summary

@Sander3003
Added link in Summary to Webservices page.

All is well.

Looks good to me!


!!!suggestion!!! Support Nullable

How about support nullable?
If set .csproj nullable:
<PropertyGroup>
	<Nullable>enable</Nullable>
</PropertyGroup>

Generate the follwing code:
public partial class CourseMerge
{
    public string? Title { get; set; }
    public int? Credits { get; set; }
    public ICollection<EnrollmentMerge> Enrollments { get; set; }
}

@chaowlert  Thank you very much！
请问你是中国人吗？


[FEATURE REQ] Setup proper test framework

We need to settle on a test framework.
Possible options could be:

Cleanup and use what we have
Leverage the IOT test framework
Use the community favorite framework

DevOps supports getting results from CTest, which should make test failure information available in the pipeline for free: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml

https://github.com/Azure/azure-sdk-for-c/pull/343


feat(testing-mongoose): update mongodb-memory-server version

Information

Type
Breaking change

Feature
No

Our direct interaction with the package is minimal and the things we do interact with are not impacted by any breaking changes as far as I could see.

Todos

[ ] Tests
[ ] Coverage
[ ] Example
[ ] Documentation

Hello @abenerd
updating the mongodb-memory-server version seems to broke the CI. There is something I can do to help you ?

Yeah, I saw that, I had some issues running the tests locally, as the mongoose tests also failed for me on production, so I had issues differentiating between the errors. And later did also not have the bandwidth to continue working on this, so this stalled on my side.

Hello @abenerd
I finally migrated all tests on TestContainer using @tsed/testcontainers-mongo. I encourage you to use it ;)


ClusterwideNetworkPolicy causes intermittent connection problems to k8s API

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
We're running Cilium with egress/ingress CiliumClusterwideNetworkPolicy/CiliumNetworkPolicy to ensure all traffic must be explicitly allowed:
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: deny-all
spec:
  description: Deny all traffic by default.
  endpointSelector: {}
  ingress:
    - {}
  egress:
    - {}

---
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: deny-all-host
spec:
  description: Deny all traffic by default.
  nodeSelector: {}
  ingress:
...
  egress:
 ...
    - toEntities:
        - kube-apiserver
 ...

This works fine - until it doesn't :)
From time to time, worker nodes are getting NotReady and stay there until manual interaction. Investigating deeper it turned out that the worker nodes aren't able to reach the Kubernetes API and thus kubelet/Cilium isn't functional anymore.
Kubelet Log:
root@fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v:~# journalctl -u kubelet --since '1 hour ago' --lines 2
May 24 06:49:18 fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v kubelet[718]: E0524 06:49:18.561511     718 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"node-local-dns-6lprl.17d1e77cc4ae555c", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"kube-system", Name:"node-local-dns-6lprl", UID:"d8ea403f-dace-478e-b6cc-2ee7dcad3df1", APIVersion:"v1", ResourceVersion:"35076729", FieldPath:""}, Reason:"FailedMount", Message:"MountVolume.SetUp failed for volume \"kube-api-access-wrqkj\" : failed to fetch token: Post \"https://27sdd5kjmq.dus2-1.REDACTED.org:30153/api/v1/namespaces/kube-system/serviceaccounts/node-local-dns/token\": dial tcp 109.68.xxx.yyy:30153: connect: connection timed out", Source:v1.EventSource{Component:"kubelet", Host:"fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v"}, FirstTimestamp:time.Date(2024, time.May, 22, 19, 50, 56, 66590044, time.Local), LastTimestamp:time.Date(2024, time.May, 23, 10, 13, 10, 402818080, time.Local), Count:70, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v"}': 'Patch "https://27sdd5kjmq.dus2-1.REDACTED.org:30153/api/v1/namespaces/kube-system/events/node-local-dns-6lprl.17d1e77cc4ae555c": dial tcp 109.68.xxx.yyy:30153: connect: connection timed out'(may retry after sleeping)
May 24 06:49:20 fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v kubelet[718]: E0524 06:49:20.993363     718 webhook.go:154] Failed to make webhook authenticator request: Post "https://27sdd5kjmq.dus2-1.REDACTED.org:30153/apis/authentication.k8s.io/v1/tokenreviews": dial tcp 109.68.xxx.yyy:30153: connect: connection timed out

Cilium Agent Log:
root@fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v:~# nerdctl --namespace k8s.io logs $(nerdctl --namespace k8s.io ps -a| grep cilium-agent | tail -n 1 | awk '{print $1}') --tail 10
level=info msg="Establishing connection to apiserver" host="https://27sdd5kjmq.dus2-1.REDACTED.org:30153" subsys=k8s-client
level=info msg="Establishing connection to apiserver" host="https://27sdd5kjmq.dus2-1.REDACTED.org:30153" subsys=k8s-client
level=info msg="Establishing connection to apiserver" host="https://27sdd5kjmq.dus2-1.REDACTED.org:30153" subsys=k8s-client
level=error msg="Unable to contact k8s api-server" error="Get \"https://27sdd5kjmq.dus2-1.REDACTED.org:30153/api/v1/namespaces/kube-system\": dial tcp 109.68.xxx.yyy:30153: connect: connection timed out" ipAddr="https://27sdd5kjmq.dus2-1.REDACTED.org:30153" subsys=k8s-client
level=error msg="Start hook failed" error="Get \"https://27sdd5kjmq.dus2-1.REDACTED.org:30153/api/v1/namespaces/kube-system\": dial tcp 109.68.xxx.yyy:30153: connect: connection timed out" function="client.(*compositeClientset).onStart" subsys=hive
level=info msg=Stopping subsys=hive
level=info msg="Stop hook executed" duration="482.18µs" function="metrics.NewRegistry.func2 (pkg/metrics/registry.go:96)" subsys=hive
level=info msg="Stopped gops server" address="<IP_ADDRESS>:9890" subsys=gops
level=info msg="Stop hook executed" duration="578.108µs" function="gops.registerGopsHooks.func2 (pkg/gops/cell.go:50)" subsys=hive
level=fatal msg="failed to start: Get \"https://27sdd5kjmq.dus2-1.REDACTED.org:30153/api/v1/namespaces/kube-system\": dial tcp 109.68.xxx.yyy:30153: connect: connection timed out" subsys=daemon

With the help of nerdctl I was able to manually start the Cilium Agent container:
root@fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v:~# nerdctl --namespace k8s.io start --attach $(nerdctl --namespace k8s.io ps -a| grep cilium-agent | tail -n 1 | awk '{print $1}')
...

This allowed me to interact with Cilium directly, but this wasn't much fruitful because the Cilium (and Monitor/...) socket isn't there yet - probably because the Kubernetes API still can't be reach:
root@fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v:/tmp# nerdctl --namespace k8s.io exec -it $(nerdctl --namespace k8s.io ps --filter name=k8s://kube-system/cilium-mm6hw/cilium-agent --latest -q) cilium identity list
Error: Cannot get identities for given labels []. err: Get "http://localhost/v1/identity": dial unix /var/run/cilium/cilium.sock: connect: no such file or directory

FATA[0000] exec failed with exit code 1

But it gave some insights and showed that there isn't a matching Egress rule anymore for the IP of the Kubernetes API:
root@fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v:/tmp# nerdctl --namespace k8s.io exec -it $(nerdctl --namespace k8s.io ps --filter name=k8s://kube-system/cilium-mm6hw/cilium-agent --latest -q) cilium bpf ipcache list | grep 109.68.xxx.yyy
109.68.xxx.yyy/32     identity=16777325 encryptkey=0 tunnelendpoint=<IP_ADDRESS>
root@fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-snt6v:/tmp# nerdctl --namespace k8s.io exec -it $(nerdctl --namespace k8s.io ps --filter name=k8s://kube-system/cilium-mm6hw/cilium-agent --latest -q) cilium bpf policy list | grep 16777325
Was impossible to retrieve label ID 16777325: Get "http://localhost/v1/identity/16777325": dial unix /var/run/cilium/cilium.sock: connect: no such file or directory
Allow    Ingress     16777325                      22/TCP       NONE         disabled    0             0          24

My guess (and I don't have much experience with Cilium/BPF or networking in general 🙈) what happening is: Cilium looses its mapping from kube-apiserver to the IP and isn't able to recover from that.
Temporary workaround for us is:
root@fb75189d-8291-4b73-baeb-6010916aecbd-7855768f7f-j9jbk:~# iptables -D CILIUM_OUTPUT -m comment --comment "cilium: host->any mark as from host" -j MARK --set-xmark 0xc00/0xf00

This deletes the iptables jump to the bpf firewalling (?) and thus allows access to the Kubernetes API again. This allows Cilium to fully restart and bootstrap itself. Afterwards everythings seems to be back to normal.
Cilium Version
1.15.2
Kernel Version
5.15.0-100-generic
Kubernetes Version
1.28.7
Regression
Not sure when it started, but we're seeing it for a couple of months and mostly doing update relatively fast after release.
Sysdump
cilium-bugtool-20240524-131108.522+0000-UTC-3443698366.zip
Relevant log output
2024-05-22 21:36:02.358	level=error msg=k8sError error="github.com/cilium/cilium/pkg/k8s/informer/informer.go:46: Failed to watch *v2.CiliumNode: failed to list *v2.CiliumNode: Get \"https://27sdd5kjmq.dus2-1.REDACTED.org:30153/apis/cilium.io/v2/ciliumnodes?resourceVersion=58814000\": dial tcp 109.68.xxx.yyy:30153: connect: connection timed out" subsys=k8s
2024-05-22 21:36:02.358	level=warning msg="github.com/cilium/cilium/pkg/k8s/informer/informer.go:46: failed to list *v2.CiliumNode: Get \"https://27sdd5kjmq.dus2-1.REDACTED.org:30153/apis/cilium.io/v2/ciliumnodes?resourceVersion=58814000\": dial tcp 109.68.xxx.yyy:30153: connect: connection timed out" subsys=klog
2024-05-22 21:35:59.313	level=info msg="Envoy: Version 99c1c8f42c8de70fc8f6dd594f4a425cd38b6688/1.27.3/Distribution/RELEASE/BoringSSL" subsys=envoy-manager
2024-05-22 21:35:56.440	level=error msg=k8sError error="github.com/cilium/cilium/pkg/k8s/resource/resource.go:808: Failed to watch *v2.CiliumEndpoint: failed to list *v2.CiliumEndpoint: Get \"https://27sdd5kjmq.dus2-1.REDACTED.org:30153/apis/cilium.io/v2/ciliumendpoints?resourceVersion=58813701\": read tcp <IP_ADDRESS>:59304->109.68.xxx.yyy:30153: read: connection timed out" subsys=k8s
2024-05-22 21:35:56.440	level=warning msg="github.com/cilium/cilium/pkg/k8s/resource/resource.go:808: failed to list *v2.CiliumEndpoint: Get \"https://27sdd5kjmq.dus2-1.REDACTED.org:30153/apis/cilium.io/v2/ciliumendpoints?resourceVersion=58813701\": read tcp <IP_ADDRESS>:59304->109.68.xxx.yyy:30153: read: connection timed out" subsys=klog
2024-05-22 21:35:56.439	level=error msg=k8sError error="github.com/cilium/cilium/pkg/k8s/informer/informer.go:46: Failed to watch *v2.CiliumNode: failed to list *v2.CiliumNode: unexpected error when reading response body. Please retry. Original error: read tcp <IP_ADDRESS>:59304->109.68.xxx.yyy:30153: read: connection timed out" subsys=k8s
2024-05-22 21:35:56.438	level=warning msg="github.com/cilium/cilium/pkg/k8s/informer/informer.go:46: failed to list *v2.CiliumNode: unexpected error when reading response body. Please retry. Original error: read tcp <IP_ADDRESS>:59304->109.68.xxx.yyy:30153: read: connection timed out" subsys=klog
2024-05-22 21:35:56.438	level=error msg="Unexpected error when reading response body: read tcp <IP_ADDRESS>:59304->109.68.xxx.yyy:30153: read: connection timed out" subsys=klog
2024-05-22 21:35:56.437	Unexpected error when reading response body: read tcp <IP_ADDRESS>:59304->109.68.xxx.yyy:30153: read: connection timed out
2024-05-22 21:35:55.689	level=info msg="FQDN garbage collector work deleted 3 name entries: logs-msvc.REDACTED2.org.,kubernetes.default.svc.cluster.local.,google.com." controller=dns-garbage-collector-job subsys=fqdn
2024-05-22 21:35:48.264	level=info msg="Imported CiliumNetworkPolicy" ciliumNetworkPolicyName=deny-all-host k8sApiVersion=cilium.io/v2 k8sNamespace= subsys=k8s-watcher
2024-05-22 21:35:48.264	level=info msg="Policy imported via API, recalculating..."
We have SSH access to broken nodes, logs in Loki and lots of metrics. If you need anything in particular to investigate this deeper, please let me know!
Also: It feels related to this issue -> https://github.com/cilium/cilium/issues/24502
Cilium Users Document

[X] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thanks for the report. Not having a sysdump is going to make it harder for us to diagnose unfortunately. Can you reproduce this if you upgrade to the latest point release?

Hey @lmb, thanks for you answer! As mentioned, I've got quite few sysdumps. I just can't share them publicly. Should I share a link with you by mail?
We will attempt the latest point release update soon, but have little hope since this issue persistent for quite some time now.

OK. I've attached one. Please let me know when you don't need it anymore. I will deleted it afterwards.

This is a scary chicken-and-egg problem; the agent needs to be able to determine which IPs belong to the apiserver, and to do that, it needs access to the apiserver.
I would suggest enabling matching nodes via CIDR and then allowing access to the set of possible apiserver IPs.

@squeed Thanks for you answer! I think that would be an option for when the pods are restarting, but I'd like to understand why those nodes loose the kube-apiserver identity in the first place. Those pods only restart, at least by my interpretation, because Cilium looses the kube-apiserver <-> CIDR mapping.

That is indeed interesting. We should probably log enough information to catch this in debug mode. If you like, you can enable debug mode globally in your cluster (set debug: true in your helm values) or on a per-node basis using per-node configuration overrides.

This is a core functionality from Kubernetes. It removes the endpoint from the kubernetes on apiserver shutdown -> https://github.com/kubernetes/kubernetes/blob/fad52aedfcc14061cf20370be061789b4f3d97d9/pkg/controlplane/controller/kubernetesservice/controller.go#L130

Indeed. That said, it seems that Cilium is doing the "right thing" -- or, at least what it has been told. Access is restricted to IPs know to be apiserver IPs, and that service is empty.
I could imagine a world in which we have some hysteresis for the apiserver to prevent these sorts of issues, but those are only papering over the real problem: it is too easy a Host Firewall policy to lose access to the apiserver on upgrades or failovers.
We should document this limitation.

Thank you for clarifying! If I interpret your answer correct, this means: Don't use kube-apiserver (and other entities?) in NetworkPolicies because they might not be stable 100% of the time - at least in the current implementation state. Is that correct?
I'd currently consider dropping all kube-apiserver rules in favor of toFQDNs. This should work without policy-cidr-match-mode=nodes because we run the apiserver outside of the cluster, right?
+1 for documenting this and/or implementing something like a cache TTL for the entity <> CIDR mapping.

Don't use kube-apiserver (and other entities?) in NetworkPolicies because they might not be stable 100% of the time

No, the kube-apiserver entity works well. The bigger issue is: be careful when using Host Firewall. Specifically, how the kube-apiserver is defined must not rely on the kube-apiserver always being up.

I'd currently consider dropping all kube-apiserver rules in favor of toFQDNs. This should work without policy-cidr-match-mode=nodes because we run the apiserver outside of the cluster, right?

ToFQDNs relies on layer 7 interception, which does not work for host firewall. You really have to allow access to the apiserver by 100% static means right now.


Track cbor-node Upstream

We are currently using a patched cbor library, due to an issue wrt. date strings.
We should therefore track upstream until PR 145 is merged.

As mentioned on the issue, you can use this instead of a patched library:
cbor.decodeFirstSync(input, {
  tags: {
    0: x => x
  }
})

Works perfectly, thanks!
1f95791d4a8781b0ae0ea703078d283eb546494a uses upstream cbor-node again.


Invisible actionbar when comic opened from search

Hi
When I sesrch for a comic, open it and want to share it, there is no actionbar. But the menu can still be opened.

I already noticed that bug yesterday, will be fixed in the upcoming update.

Just pushed the update to Google Play and F-Droid, so the bug should be fixed.


Should we add LearnIT Girl to the list?

Learn IT Girl: https://www.learnitgirl.com/

No. This is not a Summer/Season of Code program. The aim of SOCs are not to help people learn programming. SoC programs typically encourage and help increase open source participation and help students get project experience through a few months of open source programming.

@djokester Thanks for clarifying the point.


InferQuantizedStreamingFCLayer error

I'm trying to implement Tincy-YOLO with Brevitas and FINN. When I get to streamline and related transformations, I get the following error with InferQuantizedStreamingFCLayer:
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-7-c0ac61a29cf1> in <module>
      7
      8 model = model.transform(to_hls.InferBinaryStreamingFCLayer(mem_mode))
----> 9 model = model.transform(to_hls.InferQuantizedStreamingFCLayer(mem_mode))
     10
     11 # TopK to LabelSelect

/workspace/finn-base/src/finn/core/modelwrapper.py in transform(self, transformation, make_deepcopy, cleanup, fix_float64)
    137         model_was_changed = True
    138         while model_was_changed:
--> 139             (transformed_model, model_was_changed) = transformation.apply(
    140                 transformed_model
    141             )

/workspace/finn/src/finn/transformation/fpgadataflow/convert_to_hls_layers.py in apply(self, model)
    648                             scale == 1.0 or bipolar_ok
    649                         ), "out_scale = 1.0 or bipolar output needed for conversion."
--> 650                         assert (not odt.signed()) or (
    651                             actval < 0
    652                         ), "Signed output requres actval < 0"

AssertionError: Signed output requres actval < 0

I think that the error origin might be Thresholding layers don't get "out_bias" and "out_scale" attributes when exported with Brevitas from QuantReLU activation layers. If you look at the code  of the transformation, those attributes are expected when the consumer node of MatMul is a Thresholding node:
                        scale = getCustomOp(consumer).get_nodeattr("out_scale")
                        actval = getCustomOp(consumer).get_nodeattr("out_bias")
                        assert (
                            int(actval) == actval
                        ), "out_bias must be integer for HLS conversion."
                        actval = int(actval)
                        odt_is_bipolar = odt == DataType.BIPOLAR
                        bipolar_ok = (
                            odt_is_bipolar and (scale == 2.0) and (actval == -1)
                        )
                        assert (
                            scale == 1.0 or bipolar_ok
                        ), "out_scale = 1.0 or bipolar output needed for conversion."
                        assert (not odt.signed()) or (
                            actval < 0
                        ), "Signed output requres actval < 0"

Anyway, Yaman suggested that this shouldn't be a problem. So here  I leave my ONNX model and the notebook that I'm using to work with it. I must mention that I'm always working on dev branches both with Brevitas and FINN. If you need any further detail I'll be happy to help.
Thank you so much in advance.

Hi @rubenkop,
Thank you for reporting the issue! And also thank you a lot for including the ONNX model and notebook.
Unfortunately, It took me quite a while to get around to take a deeper look at it. As far as I can tell there are two things happening here:

There is a transformation missing at the beginning of the streamlining block, which is the absorb.AbsorbSignBiasIntoMultiThreshold()  transformation. This transformation would normally set the out_bias attributes accordingly.
However, the Tincy-YOLO ONNX, which was included in your issue seems to be missing some adder nodes, which the absorb.AbsorbSignBiasIntoMultiThreshold()  transformation normally expects after the MultiThreshold nodes.

I'm not entirely sure why they are missing in this case, could you maybe share your network definition, which you used in Brevitas? That might give some insights.

I am closing this issue due to inactivity. Please feel free to reopen or create a new issue, if you have more questions!


 Is there a size guide for the weight of Yolo Detection you are using? Thank you

Hey @manb911 , that's a good catch. Since our camera is not intended to be used in a "webcam" setup, the vast majority of training images contained faces at a certain distance, as you would see on most real camera scenarios.
So, even though there's no specific threshold for maximum face size, the detector won't work well in such cases. Unfortunately there's no quick fix that I can think of for that problem, other than retraining the model. Or, somehow resizing the model input, if you don't care that it will perform worse for distant faces.


Limit branch names on GitHub PR's to 255 chars

Is there an existing issue for this?

[X] I have searched the existing issues

Code improvement description
Our internal error logs are showing a PR branch failed to be created because the branch name it was trying to create was 299 characters.
According to the reporter, the limit for GitHub branch names is 255 (244 really after refs/heads).
We should be able to leverage the great work @TomNaessens did in https://github.com/dependabot/dependabot-core/pull/5338 to to enforce the proper character limit on GitHub.

Looks like the limit may be enforced on bytes, not characters: https://stackoverflow.com/a/66514473/770425
I googled but couldn't find any public docs published by GitHub documenting the limit.
Since #5338 hasn't made it to a public release yet we may want to teak the option slightly to enforce bytes limits rather than characters... or we could keep it sane for humans and just enforce a slightly smaller character limit such that it's unlikely to ever go over the bytes limit.


Closes #1493 - Fix Conjugated Gradient

Pull Request Description
What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)
Closes #1493
What are the significant changes in functionality due to this change request?
A debug statement was still in the code. This was not allowing the line-search to be activated.
The Conjugated Gradient basically was not doing what the method was supposed to do.

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[x] 1. Review all computer code.
[x] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[x] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[ ] 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
[x] 5. If significant functionality is added, there must  be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in  XML block, the node <internalParallel> to True.
[x] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[x] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[x] 8. If an analytic test is changed/added is the the analytic documentation updated/added?
[x] 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

Job Mingw Test on 51dcdfa : invalidated by @alfoa

Job Mingw Test on 51dcdfa : invalidated by @joshua-cogliati-inl
restarted civet

@JiaZhou-PU I wrote the criteria in the previous comment. If we are not clear about something, we can always ask.

I'm seeing the following test failures:

SyntheticHistory.ARMA fails on CentOS 8, Fedora 31/32, Ubuntu 16/18, but passes on Mac, Falcon, and Windows. Do we have any idea if something changed in the libraries on linux (and why Falcon doesn't care)? I doubt this PR directly affects that.
CentOS 7 and OpenSuse Leap seem unresponsive.

These both seem to be common to multiple PRs right now. The difference in SyntheticHistory.ARMA is in a relatively small (but not quite insignificant) value, so perhaps the tolerance could be increased, but I hate doing that without knowing what libraries changed ...

2. CentOS 7 and OpenSuse Leap seem unresponsive.

I think CentOS 7 is fixed. Our OpenSuse Leap system is having some weird DNS issue that neither Derek nor I have had time to try and figure out and fix.

Job Test CentOS 7 on 51dcdfa : invalidated by @joshua-cogliati-inl
restarted civet

Job Test qsubs on 12746c8 : invalidated by @alfoa
?

Job Test CentOS 7 on e22815c : invalidated by @joshua-cogliati-inl
Failed with scripts/establish_conda_env.sh: line 136: /home/civet/civet/build_0/raven/plugins/plugin_directory.xml: Permission denied

Strange, the permissions on /home/civet/civet/build_0/raven/plugins/plugin_directory.xml seem fine:
 ls -l /home/civet/civet/build_0/raven/plugins/plugin_directory.xml
-rw-r-----. 1 civet civet 170 Jul  6 09:05 /home/civet/civet/build_0/raven/plugins/plugin_directory.xml

@aalfonsi Could you address the commends in this week so that we can include these changes in new release?

addressed @wangcj05 @PaulTalbot-INL


router: factor out platform-dependent underlay conn flags

The use of the platform-dependent flag syscall.MSG_WAITFORONE prevents building the router on platforms other than Linux. This change isolates this dependency. Behavior on non-Linux platforms will be less efficient but still semantically correct because ReadBatch will only read a single message on those platforms anyway.
Together with the previous PR #4499 and the dispatcher removal PR #4344 this set of changes will, e.g., enable running local development topologies on platforms other than Linux.

This change is


[Bug]: sbb-dialog-content partly covered (sbb.ch Mobile timetable settings)

Preflight Checklist

[X] I have read the Contributing Guidelines for this project.
[X] I agree to follow the Code of Conduct that this project adheres to.
[X] I have searched the issue tracker for an issue that matches the one I want to file, without success.

Bug type
Functionality
It affects the following packages
Components
Which version of Lyne Components are you using
No response
Operating system
iOS
Browser / Browser version
Safari
Input mode
Touch
Additional settings
No response
What happened?
On the mobile version sbb.ch, it's not possible to activate the toggle "Alternative connections" in the timetable settings as this is partly covered by the dialog action buttons.
Steps to reproduce the issue

Visit sbb.ch on a mobile device
Open the timetable settings
Try to activate the toggle at the bottom ("Alternative connections")

Relevant log output
No response

Closed by #2637, please update lyne

Included in Lyne 0.51.0


理解问题

你好，我查看了代码。看到配置文件中,比如yolov3中，total_epoch和cyclic_times都是24，意思是每个epoch都按照iter进行一次余弦退火，然后把这24个epoch求平均吗

Part of your understanding is right. You need to manually average those 12 or 24 checkpoints. See step 3 of Getting Started.


How to transmit data in python?

As i understand it is only possible to encode data, but not to send it. I need to receive them in grandMA2

This happends


FqMerger - too many open files

Limit number of concurrently open files in the merging step of read simulation.
Error in job 42 while creating output file simulation.fq.
RuleException:
OSError in line 36 of /home/karel/anaconda3/lib/python3.4/site-packages/rnftools/mishmash/mishmash.snake:
[Errno 24] Too many open files: 'simulation/951/_final_reads.fq'
  File "/home/karel/anaconda3/lib/python3.4/site-packages/rnftools/mishmash/mishmash.snake", line 36, in __None
  File "/home/karel/anaconda3/lib/python3.4/site-packages/rnftools/mishmash/Sample.py", line 81, in create_fq
  File "/home/karel/anaconda3/lib/python3.4/site-packages/rnftools/rnfformat/FqMerger.py", line 37, in __init__
  File "/home/karel/anaconda3/lib/python3.4/site-packages/rnftools/rnfformat/FqMerger.py", line 37, in <listcomp>

How to get to know the limit:
import resource

# the soft limit imposed by the current configuration
# the hard limit imposed by the operating system.
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
print 'Soft limit is ', soft

# For the following line to run, you need to execute the Python script as root.
resource.setrlimit(resource.RLIMIT_NOFILE, (3000, hard))

On my Linux machine it still does not work:
...
Error in job 42 while creating output files test_metagenomic_pe.1.fq, test_metagenomic_pe.2.fq.
   RuleException:
   OSError in line 36 of /home/karel/anaconda3/lib/python3.4/site-packages/rnftools/mishmash/mishmash.snake:
   [Errno 24] Too many open files: 'test_metagenomic_pe.2.fq'
   Removing output files of failed job 42 since they might be corrupted:
   test_metagenomic_pe.1.fq
   Will exit after finishing currently running jobs.
   Exiting because a job execution failed. Look above for error message


Update futures dependency

Why only futures? What is with other dependencies?

@bkchr Haven't spotted any changes in them
what 0.3.4 has is

Fixed missing Drop for UnboundedReceiver (#2064)`

which we use a lot

Please merge master to fix the check_warnings job.

Note that according to https://github.com/rust-lang/futures-rs/pull/2064 this was only a bug in futures 0.3.3, and it was correct back in futures 0.3.2.


Interfaces Example

Create example using interfaces.

Do we have to provide just an example for an interface and it being used in a contract ?


Add option to continuously transform visuals in odometry plugin

Hello,
I made a new pull request, which is essentially the same as #1631, which was merged but then reverted as it broke the visualization of other users. But now the changes are less invasive and do not affect old visualizations as long the option is unchecked (the default). The new option triggers the same behavior as the activated frame_locked flag in the Marker Message.

This PR makes sense when you want to view the history of measurements as a time-independent path instead of time-specific measurements. In this case, the default behavior is not very useful, because it does not allow you to compare trajectories published in different frames:

With this PR, the history is re-transformed with the latest transform, allowing the comparison of trajectories.

In principal, I support merging this PR, but like mentioned in the similar #1687, I would suggest changing the naming of the option to something like "Ignore timestamp" or "Use most recent timestamp" to be consistent with the Map Display ("Use timestamp", does the opposite).
Also, in my opinion, the changes regarding ref poses for high values should be split into a separate PR.
Best regards,
Martin

This PR makes sense when you want to view the history of measurements as a time-independent path instead of time-specific measurements. In this case, the default behavior is not very useful, because it does not allow you to compare trajectories published in different frames:

I am happy to read that you see the advantages of this option for odometry display.

In principal, I support merging this PR, but like mentioned in the similar #1687, I would suggest changing the naming of the option to something like "Ignore timestamp" or "Use most recent timestamp" to be consistent with the Map Display ("Use timestamp", does the opposite).

I can change the naming in order to be consistent. @rhaschke, what do you think?

Also, in my opinion, the changes regarding ref poses for high values should be split into a separate PR.

I already split it into a separate commit. Why would you split it into a separate PR? Are you not fully convinced about this feature, yet? @rhaschke, do you also think it is useful to split it into a separate PR?


Segmentation Fault when make run-example-c-project-verilator

While running the simulation, I encountered an issue that I haven't been able to resolve. Specifically, when I run  make run-example-c-project-verilator (or the other tests), they all result in the segmentation error (image below).

Which version of Verilator are you running?
I believe they changed something relatively recently that caused this; I suggest applying this patch: https://github.com/CKeilbar/ck-cva5/commit/33a2546033779d681f84ec31732168c19562d96f


Missing tagged release for v3.2.0

I see this release in NPM, but no git tag exists

https://github.com/stefanpenner/ember-moment/releases/tag/3.2.0 done thanks


[Bug]: Rust crate does not compile with client and tls features only

Issue description
iota-sdk does not compile with client and tls feature. Adding the "wallet" feature is a "fix".
Version
Rust crate, 1.0.0
Expected behaviour
Reproduction example succeeds without errors.
Actual behaviour
Reproduction example fails.
Can the issue reliably be reproduced?
Yes
Steps to reproduce the issue

cargo new --lib test && cd test
cargo add<EMAIL_ADDRESS>--no-default-features --features client,tls
cargo check

Errors
    Checking iota-sdk v1.0.0
error[E0433]: failed to resolve: could not find `wallet` in the crate root
   --> ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/iota-sdk-1.0.0/src/client/node_manager/syncing.rs:132:82
    |
132 |     pub async fn update_node_manager(&self, node_manager: NodeManager) -> crate::wallet::Result<()> {
    |                                                                                ^^^^^^ could not find `wallet` in the crate root

error[E0282]: type annotations needed
   --> ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/iota-sdk-1.0.0/src/client/node_manager/syncing.rs:154:9
    |
154 |         Ok(())
    |         ^^ cannot infer type of the type parameter `E` declared on the enum `Result`
    |
help: consider specifying the generic arguments
    |
154 |         Ok::<(), E>(())
    |           +++++++++

Some errors have detailed explanations: E0282, E0433.
For more information about an error, try `rustc --explain E0282`.
error: could not compile `iota-sdk` (lib) due to 2 previous errors

Duplicate declaration

[X] I have searched the issues tracker this issue and there is none

@PhilippGackstatter Thanks for the report, can you see if https://github.com/iotaledger/iota-sdk/pull/898 fixes it please?

Closed by https://github.com/iotaledger/iota-sdk/pull/898


Class diagram

Description
Class diagram generated from all sources.
Fixes #83
Type of change
Please delete options that are not relevant.

Documentation update

Testing steps
N/A

Codecov Report

Merging #84 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #84   +/-   ##
=======================================
  Coverage   47.10%   47.10%
=======================================
  Files           6        6
  Lines         121      121
=======================================
  Hits           57       57
  Misses         63       63
  Partials        1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cbd2fbe...50dc1e5. Read the comment docs.


CUDA 11.8

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Installation seems to go well, the cudatoolkit libraries are present in the right places. The checks fail due to the CUDA driver not being available in the Docker image:
Finding driver from candidates: libcuda.so, libcuda.so.1, /usr/lib/libcuda.so, /usr/lib/libcuda.so.1, /usr/lib64/libcuda.so, /usr/lib64/libcuda.so.1...
Using loader <class 'ctypes.CDLL'>
	trying to load driver...	ERROR: failed to open driver:
CUDA driver library cannot be found.
If you are sure that a CUDA driver is installed,
try setting environment variable NUMBA_CUDA_DRIVER
with the file path of the CUDA driver shared library.

Finding nvvm from Conda environment
	located at $PREFIX/lib/libnvvm.so.4.0.0
	trying to open library...	ok
Finding cudart from Conda environment
	located at $PREFIX/lib/libcudart.so.11.8.89
	trying to open library...	ok
Finding cudadevrt from Conda environment
	located at $PREFIX/lib/libcudadevrt.a
Finding libdevice from Conda environment
	trying to open library...	ok

Got linux to go through by pointing numba to the location of the stub libcuda.so:
os.environ["NUMBA_CUDA_DRIVER"] = os.environ["PREFIX"] + "/lib/stubs/libcuda.so"

Not sure how to do that windows-aware

Searching online I see the Windows equivalent is "nvcuda.dll". Anyone has any idea if the toolkit installs a stub for this or something similar? Its not in the few places I looked for and I do not have a windows machine at hand -.-


Using avrgirl in Chrome app

Hello,
I am trying to integrate avrgirl into a Chrome app, but as avrgirl uses localStorage, the app wont load. Is there a way of fixing this issue without having to manually remove the calls to window.localStorage from the source?

I realise that this is coming in from a dependency. (a number of them, actually). I will investigate further.
I've managed to get it working by commenting out the lines dealing with local storage, but this is not a viable option in the long run.
On the positive side though it looks like I can build on this very awesome piece of software in order to deliver block-based Arduino programming to the school system where I live. Fingers crossed :)

hi @joachimhs! Thanks for raising this issue. Given that you're looking to integrate avrgirl for Arduino support, have you checked out this example Chrome app repo for steps on getting all the dependencies working as expected within a Chrome app? Let me know if it works or not or if something doesn't make sense. Thanks, and stoked you're considering avrgirl for your code block app!

Thank you for the link. I will take a closer look at it.
I've used webpack and gulp to handle the dependencies. I've also gone a slightly different route, as I already have the block-based web application up and running (Blockuino.no). So the workflow will be

Create the code via drag-and-drop (blockly) (or in the future also write it with text)
Click the "upload to Arduino" button. This will then send the source code to the Blockuino backend where it will get compiled using PlatformIO, before returning the .hex file to the web-app
The web-app will then message the Chrome app with the hex file, which will upload it to the Arduino.

I am hoping to support the Uno, Mini and Mega, as well as some of the ATTiny45/85 based micro controllers as a start.
At some point I will also probably release an English version as well. The main goal is to make Arduino programming accessible for kids down to 4th grade.

Makes sense. The key feature that browserify does is shimming all dependencies for ones that will work in the browser. Webpack doesn't have full shim support for every native Node API, which you can see here: https://github.com/webpack/node-libs-browser/blob/master/index.js. Regardless, it seems you got it working by eliminating localStorage calls. Could you let me know which deps use local storage? Thanks!

https://www.dropbox.com/s/m17if8j9xkip5ia/Screenshot 2017-02-15 19.18.44.png?dl=0
I also had to make a small change to avrgirl. This is due to the fact that my hex file-content is stored in a string (which avrgirl assumed was a filename). I brute-force-changed it for now, but I will make it work with both files and strings before submitting a PR

Just an FYI you can convert the string to an array buffer in the browser and avrgirl will accept that with no complaints. The example of this is in the example repo I linked :)

You are correct, of course :) I will try to use browserify instead of webpack and see if that solves the dependency issue with local storage.

I don't mean to be spamming you. I have committed a very early working version of the Chrome app. It now uses Browserify to bundle the JavaScript. It do still show the error message that localStorage is not available, but for some reason it also seems to be working fine :)
It's currently only available in a separate branch of Blockuino :)
https://github.com/joachimhs/blockuino/tree/blockuino_pi_beta/blockuino-chome-app

So, I've made some progress here. I've published a very early version of the plugin to the Chrome App Store, so the app is available for Chromebooks. Sadly the Serial-api is not brought along to Chrome Extensions, which means that I will be building something else for win, mac and linux machines, probably with the same stack, only using Electron to build a native application to handle the Serial-communication.
The Chrome app is available here in case you are interested: https://chrome.google.com/webstore/detail/blockuino-hex-uploader-av/ekbpmcpbckbdpbjhdchoniihflnmabie
I've documented this in a very short (and very Norwegian) video as well :) https://kodegenet.no/figurer/Video.MOV

Hi @joachimhs!
Thanks for the update. I'm glad you were able to progress with your uploader! Again - Blockuino is really cool. Let me know if I can help any further, but for now I'll close this issue as I believe it's resolved.


Adding auto filling README.md

This PR adds a table to the README.md file with a list of networks that have been added to the chains.json file.
It also adds a GitHub action workflow that updates the README.md file when a push to master occurs.
How it's looks like:
https://github.com/stepanLav/nova-utils

Also I found that our Telegram link does not work. Fixed it:


PyCon TW 2019 final tag and maintenance branch

pycontw-2019 branch and 2019-final tag is created.


Crash in useSelectableCollection when ref not attached to dom node in first render

🐛 Bug Report
When the ref passed to useSelectableCollection is not yet attached to a DOM node in first render - and if autoFocus is set to truthy - a crash with TypeError: element is undefined happens in this useEffect, which triggers focusSafely => focusWithoutScrolling which then tries to call focus() on null.
🤔 Expected Behavior
No errors.
😯 Current Behavior
see above
💁 Possible Solution
🔦 Context
I ran into this issue when moving useListBox up a level in the useSelect docs example, i.e. when the listbox ref is only attached when state.isOpen becomes true.
💻 Code Sample
CSB was acting up, so i put a repro here: https://github.com/stefanprobst/react-aria-issue/blob/main/components/Select.js
EDIT: here's a codesandbox based on the useSelect docs example: https://codesandbox.io/s/react-aria-listbox-ref-r6elr?file=/src/App.js
🌍 Your Environment

Software
Version(s)

react-spectrum
latest

Browser
Firefox 85

Operating System
Ubuntu 20.04

Yep, I see the issue in your project. Looks like we can just add a check to that if for the ref.current https://github.com/adobe/react-spectrum/pull/1629 I've opened a PR for it so we can see if there's anything else needed

Hey, so I was talking to a coworker, and we noticed that with the fix, even though it doesn't crash, we can't select any items from the picker. Any idea what's going on there? I'll try to check tomorrow as well

hmm that's surpsrising - i'll try to dig a bit later today.

updated issue description with CSB based on useSelect docs example.

would it make sense to use a callback ref so ref changes can trigger a render?

Sorry for the delay, I was pulled away to some other work.
I don't think we can use a callback ref, at least, not as described here https://stackoverflow.com/questions/55838351/how-do-we-know-when-a-react-ref-current-value-has-changed
The ref is passed in and not through, and we can't really require everything using this to move over to a state kept one.
My coworker actually noted a great way to deal with this (thanks @LFDanLu) if you pull the ListBoxPopup out of the Select as we have in our docs, then this will work as expected. Since there isn't a good way for us to know when to try to do the autofocus other than after mounting, I think this is the best way to handle it. useSelectableCollection has no idea about modals and we shouldn't be passing something like "isOpen" to it or "isMounted" (which is what the callback ref amounts to)
I don't think we should continue with the PR I created to try this out. I think by not failing, it'll hide the problem that my coworker pointed out.
I'll leave this open in case you have some other ideas, but feel free to close if not.


Request: Allow for offsetting 2D elements drawing positions relative to their actual positions

Hey,
I've been looking for a way to modify where sprites are drawn relative to their position, similar to the .origin() feature allowing to change where they are rotated. At first I thought that .shift() is the solution, but It seems that shift doesnt actually shift any internal values, but actually "hard" shifts the position.
This feature might not seem like an absolute must if you design your game around the engines way of doing things, but it could for sure be a useful feature for a lot of devs who are used to this. Many game developers are used to having an entities position in the center, as this often works best with how games logic is programmed.
Simple example: Have a top down game where you control a spaceship. When you shoot, you want to spawn the projectiles at the center of the ship. Currently this would mean you have to compute the center yourself by taking the ships size as a vector and halve it, add it to the original position.. etc.
This will be especially useful for games that get entity information from external sources: for example a multiplayer game. The server is almost certainly going to store player positions at their center and not care about how the visual representation on the client will be. As a result right now you have to do a pretty dirty hack in order to get the sprites drawing correctly, as you can see in a multiplayer prototype that im working on, using craftyjs as a client: https://github.com/dekdevy/spaceroad
My proposal for this would be simple; Add a function to the 2D element which takes similar inputs as  .origin(). Allow for input such as "center", "top left" etc, for easy use - or alternatively allow for input as fixed values.

I think it's really really impractical to actually change what ._x and _y give you while having the drawing position stay the same. Why not just have a ._centerX and ._centerY?

@Sythe2o0 Well this seems to have sparked quite the fundamental discussion, so allow me to explain where im coming from. You have a good point that introducing such a feature might be a cause for confusion.
So let me explain why storing entity positions at their center is a practice that can be beneficial. The whole idea stems from the fact that coupling game visuals to game logic isnt very practical.
Lets say you have a game with monsters. Monsters of different sizes. The monsters can drop items. These items can have a lot of different sizes. They can drop huge swords and really really small gold coins.
Now, if you want to drop items at the monsters positions, how do you do this? The most obvious, performant, and uncluttered approach would be to drop the item at the monster position: item.x = monster.x; this is only possible if the entities positions are all centered by default, right? I assume you would handle this by doing something along the lines of: item.x = monster.x + (monster.w0.5 - item.w0.5); (please dont tell me im right)
To answer your question regarding which engines do this: Pretty much all of them. Unity , Unreal etc.
It all boils down to the fact that in most game engines, entity position and entity visuals are mostly something completely unrelated. In unity for example you add a 2d component to the entitiy. the 2d component has  the entity as a logical anchor and moves relative to the entity position, which allows to shift and offset the entity visuals however you please, without affecting the logics of the game (this is really important for a lot of stuff.. lets say you want to squash a ball in a tennis game whenever you hit it. you would do this by changing the scale of the ball, and not touch the actual position at all. The scale would be happening relative to the center so it wouldnt make sense, visually. If the balls position was stored at its upper left corner, you would have to temporarily shift its position relative to the scaling in order to make it look correct - except you wouldnt be able to do that without fiddling deep inside the balls movement code and that would just be horrible because you are now coupling visuals to game logic.  )
I realized that this is a pretty fundamental thing and its probably not gonna be possible to implement just like that, because in craftyjs it seems that an entities position is owned by the 2d component rather than some kind of root entity component if I understand correctly.

No, if I need to access an entity's center at all regularly I define a centerX value for it.
Both Unreal and Unity are 3d Engines. I've worked with Pygame and Game maker as 2D engines, and positions haven't ever been centered as far as I can recall. Phaser, the only big javascript engine off the top of my head, also uses top-left as default, although they do offer an 'anchor' which is equivalent to your suggestion.

The display transforms are currently the same as the logical anchor's transform. Like you guys have figured out, you could have the logical anchor as the center in a local coordinate system, where collision bounding rectangles and all display stuff is actually attached to it by using offsets from the center. The transforms (position, rotation) of the logical anchor then have to be applied to the transforms of display and collision stuff to get the final transforms in the world coordinate system for collision detection system and drawing layers. (Note the performance impact).
Logic separation of components and systems is an ongoing process (#578). If we could introduc

I have been thinking about this for quite some time now and have managed to come up with a proposal. If you're interested keep on reading, I'd be happy about any comments that anyone has about this proposal (also cc @starwed )

It'd be a pretty fundamental change, so I have considered the ideas as noted in the comments by dekdevy and Sythe2o0.
Any change that is introduced should minimize the impact on:

backwards compatibility
performance
api design bloat

Here's what I propose:

Move w & h from 2D to the Renderable component.
Reason: Width and height have little to no meaning when talking about a logical anchor (a spatial point in space), they are most closely related to stuff with dimensions like images & sprites. They could be set & changed automatically for Image & Sprite components, in addition to manually changing them for e.g. the Color component.

Introduce scaleX & scaleY to 2D and retain the x, y & rotation properties.
Reason: The x & y  properties serve as aliases to the translation. The rotation property matches the rotation transform syntactically. Additionally, we would still like to scale things, hence the default scale of 1 can be altered to scale the Collision hitbox, the Renderable component, as well as any _children in the scene graph.

Change Collision & Crafty.map logic to consider the newly introduced scale properties, rather than the display related w & h properties. Viewport would still use them, as some viewport's methods depend on display stuff (like display boundaries of all entities).
Reason: These functionalities are display agnostic. Collision already works with point coordinates, it should be simple to adapt Crafty.map to work with top-left corner & bottom-right corner coordinates too (see also starwed's comment on that).

Augment origin to also set the origin of the Renderable component and the Collision hitbox.
Reason: This would serve to specify the offsets from the origin where these components are "attached", as well as around which point they are rotated around. It'd default to the current "top-left" origin. Setting it to "center" would make a Sprite start at -w/2 and end at w/2.

What are the implications of implementing this proposal?

(+) Solves this feature request.

The enhanced origin allows to specify this offsets, as suggested by dekdevy.

(+) Doesn't introduce many additional API functions

Future code will almost look the same as before. (Unlike the monstrosity I proposed in previous comment).

(+) Logic code becomes even more decoupled from display code.

Enables possible further optimizations (like asm.js).
Easier integration of 3rd party libraries into custom Crafty components that fill the roles of a 3rd party collision system (e.g. Box2D, PhysicsJS) or a render system (e.g. pixi.js).

(?) Code has good backwards compatibility.

Changing w & h won't affect collision hitboxes any more. Slight changes to Crafty.map signatures. Collision's "automatically"-computed hitbox will require a tad more explicit user interaction.
Existing code that sets origin to center would be shifted w/2 to the left and thus cause incorrect display of this entity.
Rest of code should work fine.

(-) To be determined impact on performance.

The origin offset and scale have to be considered when calculating final drawing positions.
This is still more performant than having separate (logical anchor) parent and (renderable / collidable) children that simulate local and world coordinate systems, as described in my previous comment.

I like your proposal and I am super thankful for all the thoughts that you put into this. I dont have much to add. It seems to address all the problems that I had! Just curious exactly how you would do this internally. One tiny thing to keep in mind is that if you change the width of the Renderable or the Sprite itself, that this would also have to update the internal offset values automatically. Lets say you set .origin('center) and then change the sprite of a different size, if the origin would be wrong if its not using some kind of dynamic calculation internally.
Also, great point with Box2D. If I remember correctly, by default Box2D will also use center-based origins when creating rectangle or sphere fixtures.

Lets say you set .origin('center') and then change the sprite of a different size, the origin would be wrong if its not using some kind of dynamic calculation internally.

Good observation! Hmm, I wonder if origin offsets should be specified as relative coordinates in addition (or instead of) absolute coordinates, ranging from 0.0 to 1.0 like UV coordinates. A value of 0.5 would center the sprite, no matter its dimensions. That complicates things for custom polygon hitboxes though, as these coordinates would be relative to the polygon's minimum bounding rectangle.

Necro-ing, so graphics drawing will always be top-left and origin is ONLY for rotation?

Necro-ing, so graphics drawing will always be top-left and origin is ONLY for rotation?

I'd rather say that the rectangle will always be the base primitive crafty uses for it's built in components.  Rendering to Canvas, DOM, and WebGL all ultimately require knowing the top-left position and the width/height, so making the underlying properties anything else would probably cause a performance hit.
That doesn't mean we can't add convenience methods that make it easier to work in other idioms.  The simplest approach would be to define a "Centered" component that provided new setter/getters for the underlying _x and _y properties in terms of the center of the entity.
e.g. setters for x position and width could take into account the origin.

this._x = value - this._origin._x,
this._w = value; this._x = (this._origin._x - value/2);

This has come up often enough that I'll take a stab at implementing it after the 0.8 release.  (Though if anyone else is interested in working on that, go ahead!)

Opened PR #1089 that provides some related functionality.


Async mail sender

Fix #138

Coverage remained the same at 53.069% when pulling 8c47c3d6b8446740cc6846a28617ea6ea13b069f on RLejolivet:async-mailsender into 8ca02f67cf3c76a20edcedcc50fb424c620b022e on blossom-project:master.

Shouldn't we reference a specific pooled Executor for the async ? With a parametrizable number of threads and queue size ?

No default properties: I wanted to leave org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor defaults by default, without having to duplicate them.


Improve Test Coverage

@LasNikas do you want to take a look as well before this is merged?

@LasNikas Reminder! This waits for over a week now.


SDK pulls in development level dependencies for production

This SDK is used by several other IBM cloud SDKs, in my case the @ibm-cloud/secrets-manager SDK. My application is a consumer of the @ibm-cloud/secrets-manager and I would like to package it (and its dependencies) as part of my app.
I notice that this causes a significant amount dependencies to be pulled in that are not required for production and that should also not appear in the dependencies of a production level application, e.g. the expect package. This package is meant to write unit tests and should only be a development level dependency. It alone adds around 200KB to the overall bundle size, because it in turn depends on jest packages which should not exist in a production environment either (see<EMAIL_ADDRESS>The reason for this dependency is the fact that this core SDK exports utility functions for testing purposes as part of its regular index (https://github.com/IBM/node-sdk-core/blob/main/index.ts#L21)
Since this SDK is also only available as a commonjs package, there is also no way to use tree shaking to get rid of these undesirable and unused dependencies.
I suggest to move the test functions to a sub-package with its own package.json and its own dependencies. This way consuming SDKs can still make use of the exported test helpers but they would not interfer with the production usage of the package.
Besides the undesirable bundle size this also effects legal clearance when we at IBM depend on the SDK, because test level packages typically do not have to be cleared with the same diligence as production level packages. With the current design, we are forced to clear test packages.

@Carsten-Leue Thank you for the issue.  The SDK squad will look into it.

This is e.g. the size of the expect dependency:

I suggest to move the test functions to a sub-package with its own package.json and its own dependencies. This way consuming SDKs can still make use of the exported test helpers but they would not interfer with the production usage of the package.

Would you have some advice on how to achieve this packaging change in the node core repo?
Are you suggesting we move to a monorepo where we have one package that is the core production code and another with the test helpers? One of the challenges that we would potentially have is the fact that we use the semantic-release tool to automate our release management based on commit messages, etc.   Plus by splitting out the test helpers into a separate package, that would impact existing SDK projects that currently depend on the node core, although perhaps we could do the package-splitting and create a new major version of the node core so that SDK projects would need to consciously move up to the new version.
cc: @dpopp07

@padamstx @dpopp07 I suggest to use the concept of a "secondary entry" point to expose the test level dependencies. You won't need to change the structure of the project, just adding an extra package.json to a subfolder (i.e. the secondary entry point) and removing the exports from top level is enough. This concept of secondary entry points is very popular in the Angular community but in no way restricted to Angular. But you find most of the related doc in ng docs, e.g. https://medium.com/tunaiku-tech/creating-secondary-entry-points-for-your-angular-library-1d5c0e95600a or https://github.com/ng-packagr/ng-packagr/blob/master/docs/secondary-entrypoints.md
Yes, this changes the public API of the SDK if you also remove the top level export. One approach would be to deprecate the top level exports (but leave them in) and introduce the secondary entry point in a minor version change. This way clients can move over. Together with the tree shaking PR they would still benefit from reduced dependencies.
In such an effort I recommend to revisit if the actual API the SDK exposes is indeed the intended API (I am sorry if this is sounding arrogant, but in node it's probably not evident what's exposed as the API at first glance. If the actual API actually matches the intended API, please ignore the following).
The exposed API is this:
ibm-cloud-sdk-core.api.md
Note that dependencies of this API are not the same as implementation dependencies.
You might want to consider this:

the API has a dependency on node interfaces. If the SDK is meant to be used in the browser you might want to reconsider exposing nodejs APIs as part of the SDK API.

import { OutgoingHttpHeaders } from 'http';
import { Stream } from 'stream'

the API is not complete, e.g. AuthenticateOptions is not exported but used in the API
method such as atLeastOne or atMostOne look like utility functions. Do they have to be part of the API?

I second all of the concerns outlined in the issue description
An additional concern is that the expect dependency (which should be classified as a dev dependency) is being included (as a production dependency) in OSS scans and is currently flagged with a security vulnerability that needs to be resolved

Commented accordingly in the related issue: https://github.com/IBM/node-sdk-core/issues/266#issuecomment-2115334749

+1 for this. It's extremely strange for a production library to list unit testing utilities like expect as production dependencies. expect indirectly pulls in things like chalk, @jest/expect-utils, micromatch, braces and @babel/highlight - a total of around 3.6MB of additional code. On top of everything else, this adds to the security vulnerability attack surface, as seen in the recent CVEs against micromatch and braces.

Thanks all for the comments - agreed. We will address this in the very near future.

:tada: This issue has been resolved in version 4.3.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 5.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Incorrect Documentation (Service URL)

The service throws 404s when going to http://${HOST}:${PORT}/journal
It seems the service is actually deployed at http://${HOST}:${PORT}/pass-doi-service according to the tomcat logs.

The migration of PASS repositories to eclipse-pass is complete. This issue is no longer relevant and is being closed as part of archiving https://github.com/eclipse-pass/main/issues/585.


Customize user agents to flag (such as awscli)

It would be nice to flag additional user agents
https://github.com/cloudandthings/terraform-aws-clickops-notifier/blob/be9694cda07dbe74fad9e332723be3b38bb5267b/clickopsnotifier/clickops.py#L84


Investigate possible memory leak with sample packing

Following the gist below, a dataset of size ~3 GB with 2M entries takes up more than 900 GB of CPU memory while doing offline sample packing.
https://gist.github.com/joecummings/642eaa9ce539ad93360ee3f999dbcfa3
Need to get to the bottom of this.

Based on my understanding, the ~handwavy peak memory allocated for sample packing (using 3GB dataset as example) should be:
3gb (entire original dataset loaded into memory)+
3gb (entire dataset now as packs w/ labels) +
3gb (input_pos for packs) +
3gb * 9gb (seq x seq mask for every pack)
-------------------------------------------
36gb

To process 50k samples (~ 2.2% of the entire dataset) consumes 41gb of memory.
To process 100k samples (~ 4.3% of the entire dataset) consumes 81 gb of memory.
This seems like the memory is scaling linearly, based on this looks like the entire process would consume ~1886 gb of memory.

Using the following code, I'm calculating the size of the packs directly:
packs = [
	{"tensor": ... , "mask": ... , "labels": ... , "input_pos": ... },
	{"tensor": ... , "mask": ... , "labels": ... , "input_pos": ... },
	...
]

def mem_packs(packs):
    total_size = 0
    for pack in packs:
        for elem in pack.values():
            total_size += sys.getsizeof(elem) + elem.element_size() * elem.nelement()
    return f"{round(total_size / (1024**3), 2)}G"

See comment soln seems to be to calculate mask in each batch: https://github.com/pytorch/torchtune/pull/1083/files#diff-d069dcb7c967a8f3aaaca697349854143ff7b0631f885a6e7bd62dda19f6eee1R169
We could also technically do this for input_pos. This would slow down the processing but save memory:
tokens: 88 (fixed size of Python object) + 8 (size of torch.int64) * 3072 (max seq len)
labels: 88 + 8  * 3072
----------------------
49328 bytes ~= 0.05 MB

after discussing on github this needs to be updated so atnn masks are bool, not floats, but the idea is the same.


Bug: Print errors in array validations

Hello I am trying to print the errors when I validate an array structure for example:
<select multiple name="categories[]">

# validation rules
 'categories.*' => 'is_not_unique[categories.id]'

# return errors
return redirect()->back()
                ->with('errors', $this->validator->getErrors())

then I try to print the errors in my html but it cannot show them, I suppose it is because of the key format that the function to obtain the errors returns
<?=session('errors.categories.*')?>

I think it is not the most practical way to return errors when it comes to array validations :(
thanks for everything!

If you use <?= print_r(session('errors.categories'), true) ?> will there be outputs?

If you use <?= print_r(session('errors.categories'), true) ?> will there be outputs?

Hello Paul!
try to print it that way but a null value is returned, the format of the array returned in session is as follows
array (5) {
//    ...
     ["categories.*"] =>
   string (80) "The categories.* field must contain a previously existing value in the database."
}

It would be quite nice if the ". *" Could be omitted in the name of the keys when dealing with array validations, or how could I get the error message?
Cheers!

Hello Paul!
try to print it that way but a null value is returned, the format of the array returned in session is as follows
array (5) {
//    ...
     ["categories.*"] =>
   string (80) "The categories.* field must contain a previously existing value in the database."
}

It would be quite nice if the ". *" Could be omitted in the name of the keys when dealing with array validations, or how could I get the error message?
Cheers!

Try setting a label to your validation rules:
// validation rules
'categories.*' => ['rules' => 'is_not_unique[categories.id]', 'label' => 'categories'],
// The categories field ...

But note that the array key will still be categories.* since this is the field name passed to the errors array.

Try setting a label to your validation rules:
// validation rules
'categories.*' => ['rules' => 'is_not_unique[categories.id]', 'label' => 'categories'],
// The categories field ...

But note that the array key will still be categories.* since this is the field name passed to the errors array.

It seems that the name of the label is not associated with the name of the error key because I have tried to put a label to the configuration and it keeps returning categories.*
'categories.*' => [
    'rules' => 'is_not_unique[categories.id]',
    'label' => 'categories',
],

as I mentioned before if I try to print the error as  it returns null

It seems that the name of the label is not associated with the name of the error key because I have tried to put a label to the configuration and it keeps returning categories.*
'categories.*' => [
    'rules' => 'is_not_unique[categories.id]',
    'label' => 'categories',
],

as I mentioned before if I try to print the error as  it returns null

It seems like a bug related to fact that we're using dot syntax to handle the nested array values.
Until we solve this I guess you should be able to use something like this:
<?= session('errors')['categories.*'] ?>

It seems like a bug related to fact that we're using dot syntax to handle the nested array values.
Until we solve this I guess you should be able to use something like this:
<?= session('errors')['categories.*'] ?>

But note that the array key will still be categories.* since this is the field name passed to the errors array.

It seems that the name of the label is not associated with the name of the error key because I have tried to put a label to the configuration and it keeps returning categories.*
'categories.*' => [
    'rules' => 'is_not_unique[categories.id]',
    'label' => 'categories',
],

as I mentioned before if I try to print the error as it returns null

Yes, as I've said earlier, the field name will still be used as the array key.
This is because of the following line in Validation::processRules.
https://github.com/codeigniter4/CodeIgniter4/blob/e2e5ecabf25b7c64fe6f0f3e2dcfe3f367ee507f/system/Validation/Validation.php#L326
The $field is used as the index for storing the error messages in the $this->errors array.
@michalsn The simple fix I see is using rtrim on $field before being used as index, like $this->errors[rtrim('$field', '.*')] = .. but I'm not sure if this will not cause a confusion because categories, or any other multiple-value fields, will look like a single-value field much like the other fields.

But note that the array key will still be categories.* since this is the field name passed to the errors array.

It seems that the name of the label is not associated with the name of the error key because I have tried to put a label to the configuration and it keeps returning categories.*
'categories.*' => [
    'rules' => 'is_not_unique[categories.id]',
    'label' => 'categories',
],

as I mentioned before if I try to print the error as it returns null

Yes, as I've said earlier, the field name will still be used as the array key.
This is because of the following line in Validation::processRules.
https://github.com/codeigniter4/CodeIgniter4/blob/e2e5ecabf25b7c64fe6f0f3e2dcfe3f367ee507f/system/Validation/Validation.php#L326
The $field is used as the index for storing the error messages in the $this->errors array.
@michalsn The simple fix I see is using rtrim on $field before being used as index, like $this->errors[rtrim('$field', '.*')] = .. but I'm not sure if this will not cause a confusion because categories, or any other multiple-value fields, will look like a single-value field much like the other fields.

@paulbalandan Yes, this might be confusing and a little too "magical". Also it would introduce a breaking compatibility, since this bug is valid only if we want to display errors via session.
I was thinking more about adding some new (optional) value to validation array, like:
'categories.*' => [
    'rules' => 'is_not_unique[categories.id]',
    'label' => 'Categories',
    'as' => 'categories',
],

With a note in the user guide that as should contain only alpha numeric value with dash or underscore. This way everything would be easier to understand. But these are just my initial thoughts.
There is a workaround for this bug, so it's nothing urgent, I think.

@paulbalandan Yes, this might be confusing and a little too "magical". Also it would introduce a breaking compatibility, since this bug is valid only if we want to display errors via session.
I was thinking more about adding some new (optional) value to validation array, like:
'categories.*' => [
    'rules' => 'is_not_unique[categories.id]',
    'label' => 'Categories',
    'as' => 'categories',
],

With a note in the user guide that as should contain only alpha numeric value with dash or underscore. This way everything would be easier to understand. But these are just my initial thoughts.
There is a workaround for this bug, so it's nothing urgent, I think.

It seems to be a good idea from michals, although it is not something urgent to solve since it can be solved with this <?= session('errors')['categories.*'] ?>, by the way, nice that you are here again @michalsn  :)

It seems to be a good idea from michals, although it is not something urgent to solve since it can be solved with this <?= session('errors')['categories.*'] ?>, by the way, nice that you are here again @michalsn  :)

Now getErrors() returns like this:
https://github.com/codeigniter4/CodeIgniter4/blob/6b077b7953f1ac4c34d38cd154ce23dce43f2f51/tests/system/Validation/ValidationTest.php#L873-L879

session('errors.categories.*') finds an item with dot notation, and there is no such key in the array.
Because the array key is categories.* (at the time the bug was reported), and dot_array_search() does not find such a key. So it returns null.
So this does not seem a bug.


Upgrade font-awesome dep to 4.5.0 and regenerate black & white icons.

Hmm not that I'm looking at the diff it seems that PNG files are empty. Might be the result of --optipng step?

Looks good. Sorry for taking the time to merge it.

After merging this PR car image is broken:
https://github.com/encharm/Font-Awesome-SVG-PNG/blob/master/black/svg/car.svg
https://github.com/encharm/Font-Awesome-SVG-PNG/blob/master/black/png/128/car.png
Any reason why is that?

@megawebmaster AFAIK this was broken even before we merged this PR - take a look at that file before the commit has been merged here. I noticed it while working on this update but frankly I didn't have time to take a look at that issue. There are more icons that are out of borders.

@mareksuscak - thanks, fortunately version 1.1.2 still works properly (at least for icons I'm using), so I'll keep using it.

@megawebmaster :+1:
@Rush Maybe the font awesome icons have been changed since it worked correctly back then and now it just doesn't. Do you think we can do something about it? I'd be happy to help but don't have time to spare in the upcoming weeks.


test: unit and type tests for createPages

I have not yet found a way to add type-tests for CreatePage and CreateLayout. Since the generic is on the function and not on the argument, the path property always resolves to never. I'm not sure if a test is even possible in this case. Any suggestions are welcome.

How about something like 403f4775 ?

How about something like 403f477 ?

Haha, yeah obviously. I was tunnel-visioned on ts-expect.

Same. I thought about using TypeEqual first.

It looks good overall. Let me know when it's ready to merge.

Lets go!


fix(utils): account for rule property "mimetype"

In Webpack 5, a new property "mimetype" was added to the rule definition, which must be accounted for in the rule set.
Fixes #458.
What kind of change does this PR introduce? (bugfix, feature, docs update, improvement)
Bugfix
What is the current behavior? (You can also link to an open issue here)
See #458.
Does this PR introduce a breaking change?
No.

Should be fixed by https://github.com/JetBrains/svg-sprite-loader/pull/460


SelectBox Custom: Cannot use first character to jump to item

@bpasero
@isidorn
@Tyriar
Including all who have commented :
New SelectBox implementation:
Standard native select allows navigation to an item using the first character.  This can be done both on a unexpanded or expanded select.
Should this behavior be implemented in the new custom SelectBox ?

@cleidigh yeah good catch I think it should. If the control has focus at least.

@cleidigh though assigning milestones is usualy a good idea, if you plan to fix this until the end of january just assign to December / January milestone.

@isidorn
@bpasero
Noted on the labeling issues (verified, candidate, milestones), sorry don't know all the workflows :-(
@bpasero Sorry to pepper you with a few more questions:

I saw you wrote a draft test plan, would you like me to add some more specificity on keyboard support so people know exactly what to look for?
If I have small tweaks/cleanup should I create an issue(s) or is using notes in a pull request sufficient?
Essentially when is it appropriate Not to do an issue for work in progress?

I will fix all issues posted for the December/January milestone assuming they are known in time.

@cleidigh yeah feel free to make tweaks to the test plan item: https://github.com/Microsoft/vscode/issues/40589
I also published some release notes on it here: https://github.com/Microsoft/vscode-docs/blob/vnext/release-notes/v1_20.md#custom-dropdown-windows-linux
The advantage of having an issue is that it will get verified in the endgame if it is a bug. If the thing you are changing cannot be verified or is not a bug, doing just a PR is fine.

@cleidigh no I think its fine

Fixed with: #40874


evaluate Azure DevOps plugin (export to wdi5?)

this is still on Peders plate

we have actually not decided on the scope of this upfront 😄
but creating a reusable task extension sounds good to me!
happy to hear that you'd tackle this one 👍 👍 👍


Tests aren't running (and probably broken)

Due to Github's timeout with no commits, the tests here haven't run for ages and are likely incompatible with current DDEV.
Are you still interested in maintaining this?
If so, please consider

Updating the tests.yml file to https://github.com/ddev/ddev-addon-template/blob/main/.github/workflows/tests.yml so that it doesn't time out any more.
Getting the tests (or the add-on) working again.

If not, please consider removing the ddev-get label and probably update the README to say that it's no longer maintained. Or find another maintainer?
Thanks!
-Randy

Updated and archived.


Add ability to reorder websites on dashboard

https://user-images.githubusercontent.com/51844916/180635011-5b928108-861b-471b-9ccd-a0b041d3372f.mov
Hello! Decided to take a shot at the re-ordering feature as mentioned in #1327, #1169 and #713. This implementation uses localstorage since that is what the language preference and theme uses, I figure it probably isn't worth adding to the database if language/theme isn't either. I've also added a reset button to reset back to alphabetical order. I had to add a dependency for dragging, but I hope that was alright.
Many thanks!

This is really cool! Been meaning to revamp the dashboards for a while. However, an edit interface outside of website settings would be preferable, like Edit Dashboard in the same drop-down as Toggle Charts. That way users know they are only dealing with the dashboard specifically.

https://user-images.githubusercontent.com/51844916/180713647-49f7e83e-2815-42d8-81df-06cf03f10230.mp4
Hi, thanks for taking a look! I've reverted the settings page and moved it over to the dashboard under the drop-down like you said. I do have a question though, should we leave the settings page in alphabetical order or keep it the same order as the dashboard? It's a simple fix if we revert it back to alphabetical. I'll also take a look into the weird spacing issue while dragging and reanimation of the numbers when entering/exiting sometime tomorrow.
Thanks!

Sorry if I burst into this conversation, but if you have quite a large amount of websites on the Dashboard, it will be very annoying to drag website 1 from the top to the bottom and the other way around.
So I think it would be better if all the websites would collapse and only the titles are shown. That way there won't be that much clutter on the screen.
Sorry if I'm bursting into this conversation, but if you have a large number of websites on the dashboard, it would be very annoying to drag website 1 from the top to the bottom and vice versa.
So I think it would be better if all the websites would collapse and only the titles would be shown. That way there wouldn't be so much clutter on the screen.

https://user-images.githubusercontent.com/51844916/181657965-0a0ce002-6fc9-4c20-9136-e16e56480f7f.mov
Hello! I went ahead and fixed the graphical errors, no longer will the last element cause the parent container to be shorter, and the border's won't get stuck anymore. I also thought @MagicLike 's comment was a good idea so I implemented the charts hiding when in edit mode, but we can revert that with a simple change to line 89 in the website dashboard file.
Thanks!

That's awesome! Curious... Is it possible at all to have this auto-sort by number of visitors or views rather than having to order them manually? I think both options would be great but wondering how much extra it is to have it auto-sort based on certain fields displayed (i.e. views, visitors, bounce rate, etc).