[Out] props

I originally experimented with this in prototypes but didn't commit to the idea for the public release of Fusion.
Right now, to synchronise a state object with the value of an instance's property, you have to use OnChange and initialise the state object correctly manually. It could be cleaner and less error-prone to have Fusion handle this with a new symbol key.
Hypothetical usage:
local searchText = Value()

local searchBox = New "TextBox" {
    [Out "Text"] = searchText
}

This could be highly value in conjunction with hydration and attribute support, to easily synchronise properties from services and non-Fusion code.

I love the idea of having a built in way to sync instead properties into a state object, as that is a very common pattern and my codebase is currently littered with GetPropertyChangedSignal.
I do have some questions:

Is it two way sync? IE: if I do searchText:set("foo") will that then set searchBox.Text = "foo"? If not, then searchText is not guaranteed to represent searchBox.Text.
How would you sync a state from a non-Fusion created instance? This pattern is extremely common in codebases that are slowly integrating Fusion, as they need to get their props into state objects for the Fusion side of things. Perhaps Out can take optional additional parameters so that it can be used externally as Out("Text", searchBox, searchText)?

Is it two way sync? IE: if I do searchText:set("foo") will that then set searchBox.Text = "foo"? If not, then searchText is not guaranteed to represent searchBox.Text.

This is an open question. Ideally, [Out] wouldn't be responsible for that on it's own - it would provide one way sync by default - but two way sync would be easily emulatable by setting the prop to the same state object:
-- hypothetical code
local searchText = Value()

local searchBox = New "TextBox" {
    Text = searchText,
    [Out "Text"] = searchText
}

This would be good because it gives the developer granular control over how exactly synchronisation occurs. Two way sync isn't necessary or desirable in all cases.
There is potential here for infinite looping issues, nondeterminism or other obscure race conditions though - I'd know since these came up in prototyping. It should be solvable but I don't quite remember my original solution to it.
tl;dr two way sync would be useful to have, but I'd need to prototype this first to make sure it's possible for certain

2. How would you sync a state from a non-Fusion created instance? This pattern is extremely common in codebases that are slowly integrating Fusion, as they need to get their props into state objects for the Fusion side of things. Perhaps Out can take optional additional parameters so that it can be used externally as Out("Text", searchBox, searchText)?

This would be done via instance hydration - think like New, except operating on an existing instance rather than a fresh one:
-- hypothetical code
local gameTime = Value()

Hydrate (workspace) {
    [Out "DistributedGameTime"] = gameTime
}

Hydrate would be compatible with all of the same syntax you already use with New, which has benefits outside of just getting properties into your reactive code - it would also let you add cleanup code, bind state objects, attach event listeners, add children etc.

The one way default and two way solution looks good, provided we get it working reliably. I agree, more control is better.
I must have missed the Hydrate discussion, I didn't realize it was a thing! I'm excited for that!

I like that this is explicit about one way syncing. OneWay and TwoWay sync can be great for a lot of stuff, but also a nice fast way to destroy the performance of an application and make a new user confused.
The potential for infinite looping, nondeterminism, and other properties all come with you start to two-way sync anything. I argue that the specific write-property planned here is enough to help prevent that.
I was able to write two-way sync pretty easy, with the understanding there's no magic solution to make it nice.


Prometheus workload loader

[x] Implemented ability to select an alternative workload loader
[x] Implemented Prometheus workload loader
[x] Refactored Kube API workload loader: moved separate logic (for each Kind) to a separate class
[x] List clusters on centralized prometheus
[x] Scanning multiple clusters in prometheus mode?
[ ] Test running from robusta UI
[ ] Test prometheus mode on centralized prometheus
[x] A lot of testing
[x] Fix tests (mocks are now broken)

Testing issues found:

[x] For some reason auto-discovery does not work
[x] Prometheus mode does not gather HPA data
[x] Something is now broken with HPAKey

When testing using the prometheus mode on a centralized Prometheus (VictoriaMetrics), I encountered the following errors.
I specified the labels --prometheus-label and --prometheus-cluster-label according to the docs:
python krr.py simple -p https://vmauth.xxx.com --prometheus-auth-header "<redacted>" --prometheus-label cluster -l k8s0-q --mode prometheus

To me it seems that the query includes the wrong label here, and should be avg by(cluster) instead of using the cluster name.

@deutschj please note that I changed the name of the flags a little -- see --help if necessary.

@aantn Sure, thanks a lot for working on the bugfix!
Whett testing with the options --prometheus-cluster-key and -l I encountered the following error:
krr git:(prometheus-workload-loader) python krr.py simple --prometheus-url https://vmauth.xxx.com --prometheus-auth-header "<redacted>" --mode prometheus --prometheus-cluster-key cluster -l k8s0-q

Running Robusta's KRR (Kubernetes Resource Recommender) 1.8.2-dev
Using strategy: Simple
Using formatter: table
A newer version of KRR is available: v1.11.0

[08:22:56] INFO     Connecting using Prometheus, will load the kubeconfig.
           INFO     Using Prometheus at https://vmauth.xxx.com
           INFO     Prometheus found
           INFO     Prometheus connected successfully
[08:22:57] INFO     Clusters available: k8s0-q, <cluster1>, <cluster2>, [...]
           CRITICAL Cannot scan multiple clusters for this prometheus, Rerun with the flag `-c <cluster>` where <cluster> is one of ['k8s0-q', '<cluster1>', '<cluster2>', [...]]

However, when providing only the -l option without the --prometheus-cluster-key, KRR starts to generate recommendations for the selected cluster. When generating, the following warnings are displayed for every existing workload:
WARNING  Prometheus returned no PercentileCPULoader metrics for StatefulSet xxx
WARNING  Prometheus returned no MaxMemoryLoader metrics for StatefulSet xxx
WARNING  Prometheus returned no CPUAmountLoader metrics for StatefulSet xxx
WARNING  Prometheus returned no MemoryAmountLoader metrics for StatefulSet xxx
INFO     Calculated recommendations for StatefulSet xxx (using 4 metrics)

And the table with the generated resource recommendations in the end is empty though.
So looks like now getting the existing workloads in the cluster works correctly, but getting the corresponding metrics from the centralized Prometheus doesn't yet.

@deutschj does it work if you provide --prometheus-cluster-key, -l, and --c with -c set to the same value as -l?

@aantn Unfortunately not, no - this yields the same error as above, CRITICAL Cannot scan multiple clusters for this prometheus, Rerun with the flag -c  where <cluster> is one of [...]

Would love to see this merged, using KRR from this feature branch for some time now

Hey, we're planning to get it merged, but no exact ETA yet.

hello. Are there any plans for this branch to be merged?

@arikalon1
thank you. I'm hoping this PR gets merged quickly.

is it possible to divide MR into several MR?


You can only replace ringtone code by selecting the letters

Don't know if this was the case before but it makes setting the code very obscure @deltanedas
Reproduction

Open PDA, go to Ringtone
Can't change the letters unless you select the old letter and insert the new one

Additional context
The way this should work is like a pin code, when you select any box it should wipe the whole code and put the cursor at the start, then when you insert the first character it goes to the second, insert the second, goes to the third, etc

could also just have it clear the fields when opening the ui / hitting set, then you can enter easily enough

fixed


[BUG] One host cannot mount volumes

Describe the bug
I have a running 8 node cluster with 5 storage/worker nodes and 3 control plane nodes. One worker/storage node is unable to attach volumes.
To Reproduce
Unsure on this one, the affected node is running mostly different hardware, not sure if that could be the cause though.
Majority of nodes (kube1-7):

HP ProDesk 700g3 mini
Intel i5-7500t
4 threads
32Gi RAM
No GPU

Affected node (jack):

Custom built PC
AMD Threadripper 1950x
32 threads
94Gi RAM
NVIDIA 1080ti GPU running NVIDIA Container Toolkit and GPU Operator

Expected behavior
All nodes able to mount volumes.
Support bundle for troubleshooting
https://drive.google.com/file/d/1ygoRUfQpKNWUXaEQU25b37puM3k3Wzgo/view?usp=sharing
Environment

Longhorn version: 1.6.0
Impacted volume (PV): pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): Helm
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: K8s v1.29.1, containerd 1.6.28

Number of control plane nodes in the cluster: 3
Number of worker nodes in the cluster: 5

Node config

OS type and version: Ubuntu 22.04
Kernel version: 5.15.0-94-generic
CPU per node: 1 (4 thread), (32 thread on affected node)
Memory per node: 32Gi (94Gi on affected node)
Disk type (e.g. SSD/NVMe/HDD): 2.5" SSD
Network bandwidth between the nodes (Gbps): 1

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): Baremetal
Number of Longhorn volumes in the cluster: 28

Additional context
Here is the output of the share-manager-pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4 pod on the affected node, which continues to restart and fail in the same way:
time="2024-02-11T21:40:41Z" level=info msg="starting RLIMIT_NOFILE rlimit.Cur 1048576, rlimit.Max 1048576"
time="2024-02-11T21:40:41Z" level=info msg="Ending RLIMIT_NOFILE rlimit.Cur 1048576, rlimit.Max 1048576"
time="2024-02-11T21:40:41Z" level=info msg="Listening on share manager gRPC server :9600" func=cmd.start.func2 file="server.go:134"
time="2024-02-11T21:40:41Z" level=info msg="Volume pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4 device /dev/longhorn/pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4 contains filesystem of format crypto_LUKS" encrypted=true volume=pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4
time="2024-02-11T21:40:41Z" level=info msg="Volume pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4 requires crypto device /dev/mapper/pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4" encrypted=true volume=pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4
time="2024-02-11T21:40:41Z" level=debug msg="Device /dev/mapper/pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4 is not an active LUKS device" func=crypto.DeviceEncryptionStatus file="crypto.go:89" error="failed to execute: /usr/bin/nsenter [nsenter --mount=/host/proc/2926984/ns/mnt --ipc=/host/proc/2926984/ns/ipc cryptsetup status pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4], output /dev/mapper/pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4 is inactive.\n, stderr : exit status 4"
time="2024-02-11T21:40:41Z" level=debug msg="Opening device /dev/longhorn/pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4 with LUKS on pvc-eeb3dea6-66ce-4886-afcb-641c3a7855d4" func=crypto.OpenVolume file="crypto.go:45"

Can you reproduce it on kernel 5.15.0-92? -94 seems to have a regression which makes detecting attached drives buggy.

@Isotop7 I'm not sure that's related as all hosts are running -94 and only the one host is affected, but I'll see if I can try to downgrade this weekend and re-test.

Are you using network policies?
I ran into the same issue and had to update my network policies after looking at network traffic logs: https://github.com/ArthurVardevanyan/HomeLab/commit/6a491dfacbc57985c7cf097aa08eb66dce03f089
Regardless I think both my policies and the official policies may be out of date:
Mine: https://github.com/ArthurVardevanyan/HomeLab/tree/main/kubernetes/longhorn/base/network-policy
Official: https://github.com/longhorn/longhorn/tree/master/examples/network-policy
I saw/seeing the same issue post 1.6.0 Upgrade.
Kubernetes v1.28.6+f1618d5
Distribution: 4.15.0-0.okd-2024-02-10-035534
Kernel: 6.6.9-200.fc39.x86_64
OS: Fedora CoreOS 39.20240112.3.0
Cluster Type: Hypervised on KVM with NVME Drives Passed into the VM.

I have network policies disabled.
I Haven't had a chance to look at this again, sorry about that. I'll be upgrading to the latest kernel next week on all nodes and will report back.


Add Ability To Save State

Elvish has the exec command which can be used to restart a new Elvish session in place.  I don't believe exec attempts to save the state of the shell and propagate it forward to the new session, though it does appear that Environmental Variables are saved, but not regular Elvish variables.
The other concern is that rc.elv should not be re-executed if a saved session is to be reloaded.
This should all probably be added as an optional flag to exec to preserve the current usage, which is still useful in many cases.

It's not clear what the use case is. I use exec for two reasons:

I want to run a command and and when it terminates I don't want to return to the previous shell. I just want the current terminal session to terminate. Typically for when I'm doing things like connecting to a tmux session.

I want to "undo" the effects of any experiments I've been doing in the current shell such as testing key bindings or a new module. So I'll do exec elvish to get back to a baseline instance of the shell without having to kill the current terminal session and start a new one.

For the second case above the whole point is to discard the current state of the shell. Otherwise why replace the current shell process with a new one? So I feel I must be missing something about your proposal, @iwoloschin.

Some use cases:

Updating Elvish to a newer version.
Preserving Elvish sessions across machine reboot

Updating Elvish to a newer version.

I imagine that's a bit of a niche case primarily of interest to elvish developers. Which doesn't mean it shouldn't be implemented but does rank it lower in priority than features likely to be used by the average elvish user.

Preserving Elvish sessions across machine reboot.

How would that work? It implies that the default behavior would be to restore a session rather than doing the normal init sequence of loading rc.elv. That seems likely to cause problems. Is there a single saved session or does every elvish instance save its state? If the latter you now need a way to name the sessions and select which one is loaded. And you'll need some way to perform housekeeping; i.e., removing stale saved states.
And what is included in the "session"? Non-exported variables? Function definitions? The state of things like the edit namespace? What happens when the elvish version changes in a way that invalidates the saved state? Yes, that shouldn't be much of an issue once the public APIs are frozen and backward incompatible changes stop being a frequent occurrence. But it's definitely an issue today.

I agree with @krader1961. While this idea sounds technically interesting, I'm afraid it would add a lot of implementation complexity and edge cases for minimal benefit. I like that the state of my shell is reset between sessions - that's the whole point of encoding its "desired state" in my config files. The only piece of state I might be interested in remembering is the command history, which is persisted already.

Well, this could easily have more use cases...

Tweak Elvish interactively, and dump your configuration into a file. This is now your rc.elv.

Define a function interactively, and save the function to a file. This is @zzamboni's alias module built into the language core.

If you got yourself into a state that triggers an Elvish bug and don't quite remember what you have done, you can dump it for later reproduction.

You don't have to export or import all states at once. If Elvish is engineered in such a way that all of its state is reproducible, it enables you to export and import the states selectively.

Define a function interactively, and save the function to a file.

That is a useful feature and should be implemented regardless of the resolution of this issue. Fish has the funcsave command for that purpose. It takes a single function name and creates an autoload function in your ~/.config/fish/functions. That doesn't require the ability to save and restore the entire state of an elvish session. Just basic introspection, that I believe is already available, of a function.
The other two use cases are potentially useful and would allow implementing the capability @iwoloschin wants. I have no objection to implementing that capability but it should not be the default behavior for exec. In fact, exec doesn't need to be, and shouldn't be, aware of the hypothetical save/restore state capability. If someone wants to make it easier than typing save_state /some/file (or whatever it ends up looking like) followed by exec elvish --restore /some/file a function can be written to combine the two steps.

I totally agree that this should not be the default behavior of exec; in fact, what you are proposing is exactly what I have in mind.
You can already implement some kind of funcsave in Elvish script. Retrieve the definition of a function is trivial:
~> fn funcdump [name f]{ echo 'fn '$name' '$f[def] }
~> fn l [@a]{ ls -l $@a }
~> funcdump l $l~
fn l [@a]{ ls -l $@a }

This will work only as long as the function does not have any upvalues. Elvish functions are closures and they can "capture" values in the outer scope (see also this example). The ability to serialize closures entails the ability to serialize any value and that is already very close to the ability to serialize the entire state of Elvish.

@krader1961 for the record, I specifically said this should probably not replace the default behavior of exec, but the way exec is currently used does make sense to have it gain options to load a saved state.  A new command would probably need to be created to save state, or potentially exec could be smart enough to save and reload the environment, perhaps something like exec --reload.


Spike: How do we update wego and flux components?

Short description
As a SRE I want to update wego/flux k8s cluster so that I'm running the latest code.
Flux does this by downloading new version and reinstalling...
wego upgrade
For now we will only support one previous version when upgrading. A PR will be created in the wego platform repository (assuming only one repository needs to be updated). The cluster/target will be upgraded when the PR is merged.
Discussion
Upgrading a Weave GitOps installation
The process of upgrading Weave GitOps for a single cluster consists of two pieces:

Upgrading flux controller resources
Upgrading resources generated by Weave GitOps

Flux Controller Resources
Flux controller resources should not constitute a significant problem since flux supports upgrading them by simply downloading a new flux version and re-installing flux. As we don't expect users to modify the core flux manifests, the only reason we store them at all is so that upgrading can be done via pull request. We'll simply need to run flux install --export and overwrite the existing flux manifests in the platform repository with the new ones generated by the install.
Resources Generated by Weave GitOps
Resources generated by wego app add are potentially a bit more complicated in that we expect users may modify them (to change the sync interval, for example). It would be a much better user experience if we were to keep any such changes during the upgrade. According to the flux migration timetable, any breaking changes to the CRDs will come with associated conversion mechanisms. So, we should be able to run/apply such mechanisms during an upgrade when such a change occurs. As of yet, it isn't clear what such mechanisms will look like so we will have to plan to be flexible if/when breaking changes occur. In the absence of such breaking changes, we should be able to leave flux-generated resources alone. The only resource we may have to update is our own app resource.
Pull Requests
An unfortunate side effect of our choice to allow configuration to be stored in multiple places is that we can't always upgrade everything with a single pull request. For flux-only upgrades without breaking changes, we will only need to create a PR for the platform repository. However, if we change the app resource or a breaking change occurs in the flux CRDs, we'll need to extract the config URL from each app managed by Weave GitOps and create a PR in each referenced repository with the necessary changes. The UI or CLI will have to inform the user of all the repositories that will need PRs reviewed and merged. (It's possible that this issue will be go away if the directory structure changes play out as currently discussed)
Multi-cluster
In a multi-cluster environment there will be applications that are deployed in multiple clusters. The GitOps management resources for the applications (other than the app resource), however, will be segregated by cluster. So, wego upgrade should be able to work just fine on an individual cluster. The only place in which that might be problematic is the app resource, which is shared. This suggests that we'll need to ensure all app changes are backward-compatible (for some number of releases) so that we can upgrade the app resources during an upgrade for a single cluster.

@jrryjcksn Write up looks great. Can you help me understand what the discrete work items or changes will be? A bulleted list would be great, or we can hop on a call to discuss what that list might look like.

Resources generated by wego app add are potentially a bit more complicated in that we expect users may modify them (to change the sync interval, for example). It would be a much better user experience if we were to keep any such changes during the upgrade.

This makes me nervous.  Could we give guidance on editing generated files such as "please use kustomize and we will not override those files" type of system?  I guess I view this similar to protobuf output.  You don't edit the go files because protobuf will overrides them the next time you run.

This might be more of a product question, but does wego upgrade need to update both apps and flux?  Albeit, I should read the directory ADR which you write might make all of this moot.

+1 to @jpellizzari comment.  This does look really good.  My questions above are more for my own edification @jrryjcksn.  Great work!!!

@jrryjcksn Write up looks great. Can you help me understand what the discrete work items or changes will be? A bulleted list would be great, or we can hop on a call to discuss what that list might look like.

I've been assuming we don't need to upgrade existing systems as of yet (hopefully nobody is running an important cluster w/ Weave GitOps already!). There's a plan to provide a migration script when we switch to the new directory structure and it would be good to not have to mix that up with the normal upgrade.

After the directory changes, we'll just need a wego gitops upgrade command that re-runs flux install and writes the manifests returned into the platform repo then creates a PR (we don't currently have a platform repo -- that's coming with the directory changes).

That's really all we're talking about unless/until one of the CRDs we use changes (whether a flux one like Kustomization, HelmRelease, etc. or our Application). So, not much at first. Once we see what sort of conversion support flux gives at on that day we'll have to figure out further steps.

Resources generated by wego app add are potentially a bit more complicated in that we expect users may modify them (to change the sync interval, for example). It would be a much better user experience if we were to keep any such changes during the upgrade.

This makes me nervous. Could we give guidance on editing generated files such as "please use kustomize and we will not override those files" type of system? Trying to diff the files based on the user changes them is a slipper slope. I guess I view this similar to protobuf output. You don't edit the go files because protobuf will overrides them the next time you run.

That's a good point! The kustomization approach will have direct support after the directory changes. The original design from Cornelia had us just use flux to generate the resource manifests and write them into a known location for people to edit. I don't think there's any way to stop people from directly editing the manifests, though, so we should try to be prepared for it.

This might be more of a product question, but does wego upgrade need to update both apps and flux? Albeit, I should read the directory ADR which you write might make all of this moot.

I'm not entirely sure what you mean by "apps" in this context. Are you referring to customer manifests or our Application CR? We shouldn't need to do anything to customer manifests. We only touch the automation manifests that sync their repositories. If we modify our Application CR, though, we'll have to update the existing ones in all the repositories via PRs.
The issue I suggested might be resolved by the directory changes is the possibility of having to generate multiple PRs. Hopefully we'll get back to the point where we have a single configuration repository so we can limit such changes to one PR. If we do go back to a single configuration repository, though, that change would likely come some time after the basic directory structure changes since it would be a much bigger modification to how our app works and I doubt we would have support to do it in the very near term.


chore: use pnpm v8 for deps

This PR bumps pnpm to v8

gitpod version is still 7.22.0

gitpod's pnpm version is still 7.22.0, is the plan to update it in different PR?

We are not pinning versions in GitPod. For a consistent and predictable environment, we should do that for all our tooling MongoDB, Node.js, etc.
I am not using it actively, so @ShaunSHamilton is better suited here.
But IMHO, we should build and publish a docker image (not a Dockerfile) with the tooling frozen that the instances are spun up from.


Comparison flip refactor

This is a small change that is auxiliary to the upcoming Small Materialized Aggregates SubBlock (SMA) addition.
It refactors code for flipping a comparison (less than -> greater than) to an easy to access function in ComparisonID.hpp . This new function will be used in the SMA block.

LGTM except for some minor doxygen styling issues. Fix those and we'll be good to merge.

Adjusted commenting format.

Merging.


Option to use MMIO for large Datasets

Is your feature request related to a problem? Please describe.
I'm frequently frustrated by OOMEs when building (or deserializing) Datasets that don't fit in heap memory (or even physical memory).
Describe the solution you'd like
I'd like to see an option in (or around) org.tribuo.Dataset to use mapped memory for storing Examples rather than on-heap.
Describe alternatives you've considered
I've considered subclassing Dataset and reimplementing everything that makes use of the data member, replacing it with an instance of Jan Kotek's MapDB, and using the existing protobuf implementations to marshall the Examples to/from storage. I also considered rolling my own MMIO-backed ISAM instead of MapDB, given how simple the use case is.
The reason I've not yet done these is that my Datasets are computationally expensive to prepare; I need to serialize and deserialize them when spinning processes up and down, and the existing protobuf-based implementations all instantiate Datasets with on-heap storage.
I've also considered buying a ton of physical memory. ;)

Yes, the current dataset representation isn't as memory efficient as we'd like it to be, particularly when deserializing from protobufs. The protobuf deserialization path doesn't deduplicate the feature name strings on the way through (unlike Dataset.add or Java serialization), which will increase the memory consumed unless you're using a new enough JDK which automatically does string deduplication in the garbage collector. If your data is dense we're already planning to build a dense example which should have half the memory usage of the current sparse example (as it will only store the feature values not the names).
Could you provide a little detail on the size and shape of the datasets you want to work with? Are they sparse or dense? How many features & examples are there?
Tribuo isn't particularly designed for very large datasets as most of the training methods create a copy of the data to get it into a more compute friendly representation, and the original dataset will still have a reference on the stack during a training call meaning it can't be garbage collected. We're investigating online learning support which will allow some models to scale much further wrt to data size as you'll only need a portion of it in memory while it is used for training, but that won't be possible for all model types.

My current use case has about 80k examples, with about 1000 dense features each. (Short phrases that have been run through BERT or another embedding, plus a few fistfuls of contextual features.)
I anticipate that my number of examples is going to grow exponentially.
Online learning could be made to work (and would be a fantastic addition for other use cases), but I'd much prefer being able to work with the existing Dataset abstraction, and provide my own backing store.

Ok, sounds like the dense example will help you quite a bit. Moving to memory mapped IO as a supported Tribuo Dataset class will be hard as neither the protobuf format we're moving to, nor the current java.io.Serializable representation are suitable for that, so we'd need to design a new disk representation and then we'd have to live with it for a long time.
If you're happy with writing your own dataset, then the protobuf serialization mechanisms will accept other classes that implement ProtoSerializable, so you could override all the serialization stuff and have your own Example and Dataset that each implement ProtoSerializable for the wrapper types, but contain your own implementation of the protobuf Message. The serialization infrastructure will then route the deserialization of that message to your class automatically via reflection because the class name is stored in the message. You could then sidestep the examples and just load the dataset with Dataset.deserialize which will call through to YourMMIODataset.deserializeFromProto(int version, String className, Any message).


Wrong straight recipe format in the documentation

In the file docs/getting_started.org, the straight recipes might be using an old syntax. I believe the :repo keyword now needs a list as an argument. For example, in the documentation we find this:
(package! example :recipe
  (:host github
   :repo "username/my-example-fork"
   :files ("*.el" "src/lisp/*.el")))

I believe the correct syntax should be:
(package! example :recipe
  (:host github
   :repo ("username/my-example-fork")
   :files ("*.el" "src/lisp/*.el")))

Notice the how parenthesis are used in the line :repo ("username/my-example-fork"). Please correct me if I'm wrong though. In particular, this did not work when running doom sync:
(package! org-noter-pdftools
  :recipe (:host github :repo "fuxialexander/org-pdftools" :files ("org-noter-pdftools.el")))

... but this did work
(package! org-noter-pdftools
  :recipe (:host github :repo ("fuxialexander/org-pdftools") :files ("org-noter-pdftools.el")))

With the first example, doom sync would complain with the error:
x There was an unexpected error:
  Type: doom-package-error
  Message:
    Error with packages
  Data:
    "org-noter-pdftools"
    (error "Could not find package org-pdftools in recipe repositories: (org-elpa melpa gnu-elpa-mirror emacsmirror-mirror)")
    "/dev/null
    "

Fake news!
The error stemmed from the order of package! declarations.


[Snyk] Security upgrade del from 4.1.1 to 8.0.0

Snyk has created this PR to fix one or more vulnerable packages in the `npm` dependencies of this project.
Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

package.json
package-lock.json

Vulnerabilities that will be fixed
With an upgrade:

Severity
Priority Score (*)
Issue
Breaking Change
Exploit Maturity

631/1000  Why? Proof of Concept exploit, Has a fix available, CVSS 6.2
Missing Release of Resource after Effective Lifetime SNYK-JS-INFLIGHT-6095116
Yes
Proof of Concept

479/1000  Why? Has a fix available, CVSS 5.3
Regular Expression Denial of Service (ReDoS) SNYK-JS-MINIMATCH-3050818
Yes
No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Commit messages

Package name: del
    The new version differs by 43 commits.

51344d2 8.0.0
7778d71 Meta tweaks
e2e5b71 Require Node.js 18 and update dependencies (#161)
7df7949 7.1.0
f5d31e6 Add `path` to `onProgress` event (#155)
e9b0d71 Fix CI
9e17de8 Document `dot` option (#148)
5345ef6 7.0.0
e02a96b Meta tweaks
106d7d8 Move to ESM (#143)
5de56fd 6.1.1
7b4c881 Fix `ProgressData#deletedCount` (#142)
c12fddd 6.1.0
8742649 Add `onProgress` option (#141)
7c756b3 Rename `master` branch to `main`
d0030ba Move to GitHub Actions (#131)
72007d7 Fix readme typo (#129)
cd0543e 6.0.0
6c99805 Require Node.js 10
1df5280 Readme tweaks
4acd962 Add tip about deleting subdirectories (#116)
1a0c36c Add mention of GitHub Sponsors
1e0d705 Tidelift tasks
892ce2a Make it clear that it recursively deletes directories (#113)

See the full diff

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings
📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Regular Expression Denial of Service (ReDoS)

👷 Deploy Preview for optimistic-mahavira-71b58e processing.

Name
Link

🔨 Latest commit
b976b3da4f137857f8889befe19db588afe6dce4

🔍 Latest deploy log
https://app.netlify.com/sites/optimistic-mahavira-71b58e/deploys/670b1f632b68d50008cbf44a


Running on machine with too low GPU memory

Hej,
Most likely I won't be able to run ProteinNPT on a  A100 GPU with 40GB or 80GB, but will only have 24 GB GPU memory available - that means step 3 of the pipeline is currently crashing since it runs out of memory. Do you have a recommendation for a work-around? Does training etc. get severely impaired if I reduce batch size or similar?
Cheers,
Ute

Managed to get access to a more powerful machine, case closed :)


Cannot create collection

I am trying to create a collection using Fongo 2.0.12 and I always get the following exception:
com.github.fakemongo.FongoException: Not implemented for command : { "create" : "collectionUpdateDocument" , "capped" : false}

Does not Fongo support such a basic command as creating a collection? How am I supposed to write any test without it?

Rather than create collection you should use :  db.getCollection("CollectionName");


[SPARK-38104][SQL] Migrate parsing errors of window into the new error framework

What changes were proposed in this pull request?

In this PR, I migrated parsing errors of window listed below into the new error framework.

repetitiveWindowDefinitionError
invalidWindowReferenceError
cannotResolveWindowReferenceError

Why are the changes needed?

Porting the parsing errors of window into the new error framework should improve user experience with Spark SQL.
Does this PR introduce any user-facing change?

No
How was this patch tested?

$ build/sbt "test:testOnly *QueryExecutionErrorsSuite"

@MaxGekk Could you review this? Thanks in advance.

Also cc @cloud-fan

@yutoacts The failure is related to your changes:
[info] - window.sql *** FAILED *** (5 seconds, 833 milliseconds)
[info]   window.sql
[info]   Expected "...ser.ParseException
[info]
[info]   [The definition of window 'w' is repetitive](line 8, pos 0)
[info]
[info]   == ...", but got "...ser.ParseException
[info]
[info]   [Invalid SQL syntax: The definition of window 'w' is repetitive.](line 8, pos 0)

Could you fix it, please.


Intelligently convert C/C++ comments to Rust

With this change, we can correctly parse C++ block comments.
/**
 * Does a thing
 *
 * More documentation. This test does something
 * useful.
 */

into
/// Does a thing
///
/// More documentation. This test does something
/// useful.

Fixes servo/rust-bindgen#426.

Note that this does not implement the ///< doxygen syntax (which causes the doc comment to apply to the previous item).
As far as I can tell, this is currently already broken and is causing us to emit the wrong doc comments off by one on struct fields/enum variants which use the syntax.

I need to modify this so that it also treats regular comments (//) as doc comments, will do so later today.

This is lovely, thank you!
Could we also add the test-case from #426?

I can do that.
Also, I'm going to refactor this so that rather than doing the C++ -> Rust comment conversion at the codegen level, we do it when building the AST. As far as I understand it, that would be a more appropriate place and (depending on how we build the AST - I haven't looked at it yet), we may be able to have a single call which converts comments rather than having multiple over the place.

Yeah, I think that's fine, though if it gets nasty we could also move the preprocess call to attributes::doc, and use it where it's not currently used. That'd remove the duplication too.

I've added the test case from #426 and also moved the preprocessing to the IR stage.
Let me know if you'd like me to change anything in order to merge :)

Thanks again for doing this btw! :)

Thanks again for doing this btw! :)

No problem, thanks for working on the project :)
I've fixed the comment issue but I can't get the convert-cpp-comments-to-rust test I added to match the expected header file.
Here is a part of the diff from the travis run
It seems as if it's related to environment differences between my computer and CI, but I can't really replicate the exact same build setup as them.
pub const _STDINT_H: ::std::os::raw::c_uint = 1;
 pub const _FEATURES_H: ::std::os::raw::c_uint = 1;
+pub const __USE_ANSI: ::std::os::raw::c_uint = 1;
 pub const _ISOC95_SOURCE: ::std::os::raw::c_uint = 1;
 pub const _ISOC99_SOURCE: ::std::os::raw::c_uint = 1;
-pub const _ISOC11_SOURCE: ::std::os::raw::c_uint = 1;
 pub const _POSIX_SOURCE: ::std::os::raw::c_uint = 1;
 pub const _POSIX_C_SOURCE: ::std::os::raw::c_uint = 200809;
 pub const _XOPEN_SOURCE: ::std::os::raw::c_uint = 700;
 pub const _XOPEN_SOURCE_EXTENDED: ::std::os::raw::c_uint = 1;
 pub const _LARGEFILE64_SOURCE: ::std::os::raw::c_uint = 1;
-pub const _DEFAULT_SOURCE: ::std::os::raw::c_uint = 1;
+pub const _BSD_SOURCE: ::std::os::raw::c_uint = 1;
+pub const _SVID_SOURCE: ::std::os::raw::c_uint = 1;
 pub const _ATFILE_SOURCE: ::std::os::raw::c_uint = 1;
-pub const __USE_ISOC11: ::std::os::raw::c_uint = 1;
 pub const __USE_ISOC99: ::std::os::raw::c_uint = 1;
 pub const __USE_ISOC95: ::std::os::raw::c_uint = 1;
 pub const __USE_POSIX: ::std::os::raw::c_uint = 1;
 pub const __USE_POSIX2: ::std::os::raw::c_uint = 1;
 pub const __USE_POSIX199309: ::std::os::raw::c_uint = 1;
 pub const __USE_POSIX199506: ::std::os::raw::c_uint = 1;
 pub const __USE_XOPEN2K: ::std::os::raw::c_uint = 1;
 pub const __USE_XOPEN2K8: ::std::os::raw::c_uint = 1;
 pub const __USE_XOPEN: ::std::os::raw::c_uint = 1;
 pub const __USE_XOPEN_EXTENDED: ::std::os::raw::c_uint = 1;
 pub const __USE_UNIX98: ::std::os::raw::c_uint = 1;
 pub const _LARGEFILE_SOURCE: ::std::os::raw::c_uint = 1;
 pub const __USE_XOPEN2K8XSI: ::std::os::raw::c_uint = 1;
 pub const __USE_XOPEN2KXSI: ::std::os::raw::c_uint = 1;
 pub const __USE_LARGEFILE: ::std::os::raw::c_uint = 1;
 pub const __USE_LARGEFILE64: ::std::os::raw::c_uint = 1;
 pub const __USE_MISC: ::std::os::raw::c_uint = 1;
+pub const __USE_BSD: ::std::os::raw::c_uint = 1;
+pub const __USE_SVID: ::std::os::raw::c_uint = 1;

Does this ring any bells @emilio?

Tests are now green @emilio, thanks for the help!

Looks great, thanks again for this!
@bors-servo r+

:pushpin: Commit 239a015 has been approved by emilio

:hourglass: Testing commit 239a0154cdab5fa0052f2ce98129e2c169dc1cc4 with merge f19e576961d2e3832760ca22a8e62f8ab4f7f078...

:sunny: Test successful - status-travis
Approved by: emilio
Pushing f19e576961d2e3832760ca22a8e62f8ab4f7f078 to master...

Author of #426 here. I didn't get a notification when this PR was opened.
Why are you converting into Rust doc comments? It seems highly unlikely that rustdoc or rustdoc --test will be able to deal with random C++ comments correctly.

Why are you converting into Rust doc comments? It seems highly unlikely that rustdoc or rustdoc --test will be able to deal with random C++ comments correctly.

Well, sure, but that's followup work anyway. I think it's nice to have the comment in a processable format, which is what this PR enables. This PR also fixes the most common cases where people doesn't use markdown for their C/C++ docs.
If we want to support fully rustdoc, etc, there may be more followup work needed. An easy way to generate comments and make rustdoc not complain is making comments normal // comments.
Anyway, I've followed up in #799 with fixes and a few features that this PR enabled. I think Markdown/Doxygen support can be discussed in a new issue. Will file one to that effect.

I opened #800 for that.

@emilio would it be possible for a new revision to be published so I can use this in one of my own crates?

@dylanmckay sure, though I'd wait for #799 to merge to fix the panic when a class is documented with an empty multiline comment (/**/), I don't want to release known panics. I'll push a version bump to that PR :)

Sounds good to me, thanks @emilio :)


function "page" not defined

Issue description
I'm attempting to build my first Hugo site using the Congo theme. Looks beautiful! When I run the instructions on the installation site I end up with an error though.

Theme version
v2.7.6
Hugo version
hugo v0.101.0+extended linux/amd64 BuildDate=unknown
Which browser rendering engines are you seeing the problem on?
No response
URL to sample repository or website
No response
Hugo output or build error messages
go: no module dependencies to download
go: added github.com/jpanther/congo/v2 v2.7.6
Error: add site dependencies: load resources: loading templates: "/tmp/hugo_cache/modules/filecache/modules/pkg/mod/github.com/jpanther/congo/v2@v2.7.6/layouts/partials/footer.html:7:1": parse failed: template: partials/footer.html:7: function "page" not defined

If I follow the git install instructions I get a similar error.
Error: add site dependencies: load resources: loading templates: "/home/<user>/Documents/hugo/<sitename>/themes/congo/layouts/partials/footer.html:7:1": parse failed: template: partials/footer.html:7: function "page" not defined

Interestingly enough, this morning it seems to be working fine. I have a suspicion that I was doing something funky with the Git config and switching folders that was causing it. I'm on a different machine today though as well so there's always a chance it is related to that. Either way, I appreciate your willingness to look at my issue so quickly and provide a response!


Multi-window Support

What would you like to be added:
Possibility to spawn multiple app windows.

Came here to look for this - currently i cant have 2 lens windows open to show 2 different clusters for easy comparison (staging vs prod cluster for example) - would love this

Pleeeeasseeee!

Still nothing 4 years later. Sigh....

AND????


With no access to internet, machine drivers will continually re-try to download driver and are not able to be disabled

By default, the machine drivers shouldn't include any driver that includes downloading something from github as if you don't have internet access, this causes errors in rancher-server.
Specifically, the packet driver requires it.

The machine service should just give up after a couple tries and go to error instead of trying forever.  Removing things by default doesn't solve the problem, it just hides it until someone tries any of the not-built-in drivers.

Since you can deactivate the driver, this can stop the errors. This will not be addressed in 1.2.0

After speaking with @chrisurwin, the issue is that you cannot actually deactivate the driver when there is no internet. It is automatically being re-activated even though you have de-activated it and continually trying to download the driver details.
There should probably be 2 fixes:

If there is no internet access and we disable the driver, it should remain disabled and not reactivate.
If there is no internet access, we should give up and error instead of trying forever.

Tested with  rancher/server:v1.2.0-rc1
Launched rancher-server in a setup which has no internet access:
I see the following message repeatedly in the logs indicating that there is continuous attempt made to download packets driver that is failing:
time="2016-11-29T20:00:18Z" level=error msg="Error processing event" err="Get https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip: dial tcp: lookup github.com on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:41060-><IP_ADDRESS>:53: i/o timeout" eventId=4edcf26d-4c0e-44d1-8b4f-b4c8ebd87237 eventName="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:00:43Z" level=info msg=Event eventId=4e71b4be-e17f-413a-8e25-c05a577280f5 name="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:00:43Z" level=info msg="Download https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip"
time="2016-11-29T20:01:03Z" level=error msg="Error processing event" err="Get https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip: dial tcp: lookup github.com on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:37252-><IP_ADDRESS>:53: i/o timeout" eventId=4e71b4be-e17f-413a-8e25-c05a577280f5 eventName="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:01:28Z" level=info msg=Event eventId=cbd089f5-9a73-4af0-8af3-5da656d6d7dc name="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:01:28Z" level=info msg="Download https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip"
time="2016-11-29T20:01:48Z" level=error msg="Error processing event" err="Get https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip: dial tcp: lookup github.com on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:42980-><IP_ADDRESS>:53: i/o timeout" eventId=cbd089f5-9a73-4af0-8af3-5da656d6d7dc eventName="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:02:13Z" level=info msg=Event eventId=0bd87299-cc6d-483f-ae05-402d9973bb1a name="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:02:13Z" level=info msg="Download https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip"
time="2016-11-29T20:02:33Z" level=error msg="Error processing event" err="Get https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip: dial tcp: lookup github.com on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:53267-><IP_ADDRESS>:53: i/o timeout" eventId=0bd87299-cc6d-483f-ae05-402d9973bb1a eventName="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:02:58Z" level=info msg=Event eventId=25144ac4-7d1c-4b01-bc2f-7cf11762e7d5 name="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:02:58Z" level=info msg="Download https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip"
time="2016-11-29T20:03:18Z" level=error msg="Error processing event" err="Get https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip: dial tcp: lookup github.com on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:43845-><IP_ADDRESS>:53: i/o timeout" eventId=25144ac4-7d1c-4b01-bc2f-7cf11762e7d5 eventName="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:03:43Z" level=info msg=Event eventId=8606c296-4423-4aba-ac5e-0234279d3ae8 name="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-29T20:03:43Z" level=info msg="Download https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip"
time="2016-11-29T20:04:03Z" level=error msg="Error processing event" err="Get https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip: dial tcp: lookup github.com on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:47671-><IP_ADDRESS>:53: i/o timeout" eventId=8606c296-4423-4aba-ac5e-0234279d3ae8 eventName="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1

Tested with rancher/server:v1.2.0-rc2
Launched rancher-server in a setup which has no internet access:
I see the following error to download machine config happen only once.
There is no repeated attempt to  download machine config .
time="2016-11-30T22:02:31Z" level=error msg="Error processing event" err="Get https://github.com/packethost/docker-machine-driver-packet/releases/download/v0.1.2/docker-machine-driver-packet_linux-amd64.zip: dial tcp: lookup github.com on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:36508-><IP_ADDRESS>:53: i/o timeout" eventId=9be1e714-78e5-47b6-b30c-912faf262391 eventName="machinedriver.activate;handler=goMachineService-machine" resourceId=1md1
time="2016-11-30T22:02:31Z" level=info msg=Event eventId=21513f5d-23a8-41a6-96ba-6baa65f8dc49 name="machinedriver.error;handler=goMachineService-machine" resourceId=1md1
time="2016-11-30T22:02:36Z" level=error msg="Failed to pull the catalog from git repo https://git.rancher.io/community-catalog.git, error: exit status 1"
time="2016-11-30T22:03:17Z" level=error msg="Failed to pull the catalog from git repo https://git.rancher.io/rancher-catalog.git, error: exit status 1"
time="2016-11-30T22:08:57Z" level=error msg="Failed to pull the catalog from git repo https://git.rancher.io/rancher-catalog.git, error: exit status 1"
time="2016-11-30T22:09:37Z" level=error msg="Failed to pull the catalog from git repo https://git.rancher.io/community-catalog.git, error: exit status 1"
time="2016-11-30T22:13:57Z" level=error msg="Failed to pull the catalog from git repo https://git.rancher.io/rancher-catalog.git, error: exit status 1"
time="2016-11-30T22:14:37Z" level=error msg="Failed to pull the catalog from git repo https://git.rancher.io/community-catalog.git, error: exit status 1"


Upgrade to generator-jhipster 6.3.1

Please make sure the below checklist is followed for Pull Requests.

[ ] Travis tests are green

[ ] Tests are added where necessary

[ ] Documentation is added/updated where necessary

[ ] Coding Rules & Commit Guidelines as per our CONTRIBUTING.md document are followed

Looks like we have to adapt the e2e tests, It doesn't look like a hiccup as basically all are failing for each variant

sad if the e2e tests failed again...

Is there a real issue as I see MapsIdUserProfileWithDTO e2e test fails consistently on few jobs?

or if these are due to slow api response or other animations, then, we can try to increase wait from 10s to 30s

I will check later if they fail locally.

I could not make e2e fail on my local machine using latest main generator master and blueprint

Closing in favor of https://github.com/jhipster/jhipster-vuejs/pull/471

For your information, there is no flaky e2e tests here, only real bugs, related to:

something related MapsId
new readonly feature
broken couchbase +  jdk8
Travis builds not up-to-date (as jdk8 has been removed by default)

So no worry @vishal423, your work was good, thanks so much.
I will do my best to fix all these items tomorrow and do a new release of this blueprint.
It's my priority !


Add dockerfile, docker-compose.yaml

There are still some issues & work remaining, but would you all be generally interested in something like this to use docker/docker-compose to simplify the install steps?
I'm getting some errors but the app mostly comes up - I'm not sure if it's expected to fully work before https://github.com/Metaculus/metaculus/pull/194 is merged, so I'll re-test after that, but let me know if you have any thoughts/questions/feedback whenever.

There are still some issues & work remaining, but would you all be generally interested in something like this to use docker/docker-compose to simplify the install steps?
Certainly, I think being able to run metaculus by just running a docker-compose file is quite appealing.

Would probably add the same waiting strategy you have for postgres for redis.
I'm curious if it wouldn't be easier for us to just add CREATE EXTENSION IF NOT EXISTS vector; as sql always run by our migration command.
The main downside with this is that I expect that using this in a development environment would be rather slow.

Having the docker compose is quite useful, I believe it should now work with our standard docker file so I'm removing yours + the entrypoint script.


A problem with loss computation.

output = output[1:].view(-1, output_dim)
trg = trg[1:].view(-1)
loss = criterion(output, trg)
Above is a part of your codes. The loss computed by the above line is the average at every time step, which can cause it difficult to train the model.
So I suggest accumulating the loss at every time step. In my experiments, this makes it easier to train the model.

@yxdr
I've experimented with this before and found that you are correct, it does give lower loss values. I'll double check it actually gives better translations/better BLEU scores and then make it from a per sequence loss instead of per token.
Do you have any intuition of why training this way gives a lower loss? Can't think of one off the top of my head.

@bentrevett  its because the gradients are fluctuating from - to + ve extremes like sine curve for every update that the network does for each of the time stamp. If one accumulates all the gradient for one sequence then the updated values of the weights/parameters would be the average of the gradients  faster to converge in less number of epochs.  I read this on one article from this repository https://github.com/allenai/writing-code-for-nlp-research-emnlp2018 . I may be wrong but this could be the answer.

@thak123 Thanks for the link, I'll check it out.

@yxdr
I've experimented with this before and found that you are correct, it does give lower loss values. I'll double check it actually gives better translations/better BLEU scores and then make it from a per sequence loss instead of per token.
Do you have any intuition of why training this way gives a lower loss? Can't think of one off the top of my head.

Accumulating the loss during decoding makes the total loss used to compute gradients t times larger, where t is the length of the target sequences, and so makes the gradients larger.  If you use the reshaped trg and output to compute the total loss, the total loss will be smaller.
This is my intuition, but I am not very sure that this is right.

@yxdr
I've experimented with this before and found that you are correct, it does give lower loss values. I'll double check it actually gives better translations/better BLEU scores and then make it from a per sequence loss instead of per token.
Do you have any intuition of why training this way gives a lower loss? Can't think of one off the top of my head.

Accumulating the loss during decoding makes the total loss used to compute gradients t times larger, where t is the length of the target sequences, and so makes the gradients larger. If you use the reshaped trg and output to compute the total loss, the total loss will be smaller.
This is my intuition, but I am not very sure that this is right.

That makes sense. I think I assumed that the larger gradients would give a more unstable learning experience. If the gradients just need to be larger, could we not just increase the learning rate?

@bentrevett idea is not to have large gradients,as it will cause unstable learning. the aim is to update in such a way that bouncy gradient values are not attained (extreme -ve and +ves)

@thak123 Can you tell how this can be fixed in code?

To set it so the losses are summed over the sequences and then averaged, thus giving you average loss per sequence instead of loss per sequence, first change the criterion to:
criterion = nn.CrossEntropyLoss(ignore_index = TRG_PAD_IDX, reduction = 'sum')

That sums all of the losses for every single token. You then want to divide this by the batch size to get average loss per sequence, which can be done when we calculate the loss with the criterion as:
loss = criterion(output, trg).div_(batch_size)

So I have now done quite a few experiments using the per sentence loss instead of per token.
For these models and the dataset used for these tutorials the per token loss gives the superior BLEU quality, usually around 5 BLEU points higher.
Therefore I am going to keep the loss calculations as they are and close this issue.


Make taxon text search db easier to build

Continuing from #30, #32, and #34.
Currently, it's a bit cumbersome to generate a taxon text search db from scratch, and is only partially documented. First, these functions need a bit of cleanup and more docs.
Hosting
Second, I'm thinking it would be more useful to host a copy of the competed SQLite db file somewhere, so others can more easily make use of this without all the setup steps. Depending on which languages are included, it's about 20-120MB compressed. Options:

S3 (if storage & traffic stayed within the free tier)
GitHub large file storage (e.g., just commit the tarball to the repo)
GitHub releases
Publish in a separate package on PyPI (along with the TaxonAutocompleter class)

Partial hosting
The next best thing would be to only store a couple relevant components of the database. For example, the step of generating aggregated taxon counts requires 25Gb+ of local storage (for the main DwC-A observation archive) and several hours to run, and the end result can be minified and compressed down to about 2MB.
Just hosting that somewhere would make it a lot more convenient to build the text search db. It would then just require about 200MB of local storage (for the DwC-A taxonomy archive) and maybe 5-10 minutes to build.
Automation
It would be really nice to have some automation that generates an updated database on a schedule (maybe every few months) based on the latest DwC-A release, although this may be more work than it's worth. Currently the process requires too much local storage, memory, and CPU time to fit inside a GitHub Actions runner. There's a lot of room for optimization, though, especially in aggregate_taxon_counts().

Decided to go with GitHub Releases for this: https://github.com/pyinat/pyinaturalist-convert/releases/tag/v0.4.0


[Bug]: Install Script Proceeds To Install v4.0.0-beta.294

Description
I just run curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash  and it started to install 294.
https://cdn.coollabs.io/coolify/versions.json
{
    "coolify": {
        "v4": {
            "version": "4.0.0-beta.294"
        },
        "sentinel": {
            "version": "0.0.4"
        }
    }
}

Welcome to Coolify v4 beta installer!
This script will install everything for you.
(Source code: https://github.com/coollabsio/coolify/blob/main/scripts/install.sh )

-------------
OS: ubuntu 22.04
Coolify version: 4.0.0-beta.294
-------------
Installing required packages...

Minimal Reproduction (if possible, example repository)
curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash
Exception or Error
No response
Version
4.0.0-beta.294

Workaround - Update script to install beta 297:
curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash -s 4.0.0-beta.297

Found the bug, the following file needs updating,
https://cdn.coollabs.io/coolify/versions.json
In the repo the file is updated correctly

https://github.com/coollabsio/coolify/issues/2475#issuecomment-2162625725

@andrasbacsai I just did the command and it didn't worked, installed again the v294

@mbeaudru I added a readme to clarify this: https://github.com/coollabsio/coolify/blob/main/RELEASE.md


Adding SpanNear, SpanMulti and SpanTerm Queries

Adding SpanNear, SpanMulti and SpanTerm Queries.
Also adding simple tests on query generation.

Thanks for adding these. Unfortunately, they need some more work to be included. There's little things missing (like the Copyright notice in the header of the files), and some bigger things. For the latter, it's almost always to align with the source code of the Elasticsearch server. E.g. SpanMulti is SpanMultiTermQuery in Java. We also need to check if the span queries in Java support things that are not included in your PR. Sometimes, they implement things not even mentioned in the official docs.
 The latter might take some time.
Notice you can use your implementation just fine as it implements the elastic.Query interface, and then later switch to the "official" queries.

No problem!
I'll try to sneak some time there to make those changes. :)

I will not include Span queries in v6.


Broker level dispatch rate limiter

Is your feature request related to a problem? Please describe.
Currently, the Pulsar support topic level and subscription level dispatch rate limiter. When we have more topics and consumers(with high receive queue size) attach many topics, this will lead to a sudden increase in entry read traffic.
It may bring unstable factors to the cluster(may cause broker outgoing network bandwidth with the heavy workload, may cause too many read request to bookies, etc.)
The topic level rate limiter and the subscription level rate limiter unable to limit the resources of the entire broker because of the topics and the subscriptions may keep increasing.

The above screenshot is happening when I test read entries with 200MB/s, but it reached 600MB/s at the moment of starting the consumer, and the message writing also been affected.
Describe the solution you'd like
Introduce broker level dispatch rate limiter. This will make it easier for us to do broker resource planning.

Please assign it to me


Butter-fly pattern

We can create a code for printing Butter-fly pattern

**
**

As it is bit confusing for beginners

@aaryahjolia Kindly assign this issue to me


Add Check Package and Build Vignettes commands

@renkun-ken Thank you for PR!
I forgot to add the TODO on README.md, and your work is great!
However, these functions look easy to run without command and not be called than other packages function.
In my opinion

Adding the command is good :)
In the future, this extension should add many important commands (like TODO that you added).
But many commands without the button or menu make the user confuse important command (like Run, Create Terminal).

I think, first, we should add menu or button for these commands before merging.

Thanks for your suggestions! It is a bit like an initial attempt for me to start working on this extension in hopes of making it somewhat comparable with RStudio (both Desktop and Server) in the long term.
I'd love to add menu or button for those.


Can't download current builds

Expected behavior
It would be nice to have an opportunity to download and test current builds.
Actual behavior
"Build status" from README.md redirects to notice that DEV@Cloud (Cloudbees) reached the "end of life" status.

That  would be terrific  - also wondering if you're getting close to a 5.700 beta release too. My app is on the beta channel but  it too hasn't updated since 5.600 last fall.
Really looking forward to trying out the material design changes. Thank you for all the work you guys do and also for your leadership participation in openpgp.

It would be nice to have an opportunity to download and test current builds.

Yep, that'd be a really good idea. master build is surprisingly stable, I'm using it daily for months and did not see any major problems.

I don't want end users without development experience testing development builds.


LangchainProcessor can't use custom invocation parameters as config

This is a strategically important issue for me to use pipecat for good.
pipecat:LangchainProcessor wraps langchain:RunnableWithHistory. I need to replace session_id with two fields user_id and conversation_id as shown in the Langchain documentation.
My code
        chain_runnable = RunnableWithMessageHistory(
            prompted_chat_model,
            # lambda session_id: get_session_history(database_label, session_id),
            lambda session_id: get_session_history(database_label, user_id, conversation_id),
            history_messages_key="chat_history",
            input_messages_key="input",
            **memory_kwargs,
            history_factory_config=[
                ConfigurableFieldSpec(
                    id="user_id",
                    annotation=str,
                    name="User ID",
                    description="Unique identifier for the user.",
                    default="",
                    is_shared=True,
                ),
                ConfigurableFieldSpec(
                    id="conversation_id",
                    annotation=str,
                    name="Conversation ID",
                    description="Unique identifier for the conversation.",
                    default="",
                    is_shared=True,
                ),
            ],
        )

Error
The above code throws
ValueError: Missing keys ['conversation_id', 'user_id'] in config['configurable'] Expected keys are ['conversation_id', 'user_id'].When using via .invoke() or .stream(), pass in a config; e.g., chain.invoke({'input': 'foo'}, {'configurable': {'conversation_id': '[your-value-here]', 'user_id': '[your-value-here]'}})

Tried changing the pipecat code to replace session_id in langchain.py as:
    async def _ainvoke(self, text: str):
        logger.debug(f"Invoking chain with {text}")
        await self.push_frame(LLMFullResponseStartFrame())
        try:
            async for token in self._chain.astream(
                {self._transcript_key: text},
                # config={"configurable": {"session_id": self._participant_id}},
                config={"configurable": {
                    'user_id': self._user_id,
                    'conversation_id': self._conversation_id
                }},
            ):
                await self.push_frame(TextFrame(self.__get_token_value(t

but this throws
ValueError: Expected keys ['conversation_id', 'user_id'] do not match parameter names ['session_id'] of get_session_history.

I could extend that to accept the custom parameters but i was looking for a more general solution which established pipecat contributors like @TomTom101 could think of.
Langchain code
As I understand the langchain code, this is how the custom config can be passed:
langchain_core/runnables/base.py, line 5242, in astream():
self._merge_configs(config)

which calls langchain_core/runnables/history.py, lines 533ff:
expected_keys = [field_spec.id for field_spec in self.history_factory_config]
...
            message_history = self.get_session_history(
                **{key: configurable[key] for key in expected_keys}
            )

Pipecat code
So in pipecat/processors/frame_processor.py, the _ainvoke method uses only the session_id
    async def _ainvoke(self, text: str):
        logger.debug(f"Invoking chain with {text}")
        await self.push_frame(LLMFullResponseStartFrame())
        try:
            async for token in self._chain.astream(
                {self._transcript_key: text},
                config={"configurable": {"session_id": self._participant_id}},
            ):

Suggested Solution
So _ainvoke could be extended to pass the the keys from langchain's self.history_factory_config?
Maybe something along this snippet:
    async def _ainvoke(self, text: str):
        logger.debug(f"Invoking chain with {text}")
        await self.push_frame(LLMFullResponseStartFrame())

        if hasattr(self, '_configurable'):
            configurable = self._configurable
        else:
            configurable = {"configurable": {"session_id": self._participant_id}}
        try:
            async for token in self._chain.astream(
                {self._transcript_key: text},
                config=configurable
            ):
                await self.push_frame(TextFrame(self.__get_token_value(token)))

OR
is there another way to pass the config dict for the new parameters to LangchainProcessor?

Thanks! Try this branch and set your configurables like this
config = {"user_id": "123", "conversation_id": "1"}
lc.set_configurable(config)

session_id is an configurable on its own and should not be hard-coded anyway.

@TomTom101 I tried this but didn't work.
Doesn't the invoke method need to access it like this?
    async def _ainvoke(self, text: dict):
...
        if hasattr(self, '_configurable'):
            configurable = self._configurable
        else:
            configurable = {"configurable": {"session_id": self._participant_id}}


Add structure for istioapi in a new repo

This repo does not depend on anything else
Any shared, or user visibile schema, api or vocabulary should be placed
here

Config schemas should not be k8s-specific.
We can use k8s-specific means for distributing config in a k8s cluster, but
the general config model we use should be agnostic of k8s constructs.
On Thu, Dec 15, 2016 at 1:04 PM, Kuat<EMAIL_ADDRESS>wrote:

@kyessenov commented on this pull request.
In README.md
https://github.com/istio/istioapi/pull/1#pullrequestreview-13220090:

@@ -1,2 +1,23 @@

istioapi
-API, Config Definitions and Standard Vocabulary definitions for the istio project
+API, Config Schema Definitions and Standard Vocabulary definitions for the istio project
+
+
+Usually definitions are in the form on .proto files.
Can you please add that "proto is a schema"?
In README.md
https://github.com/istio/istioapi/pull/1#pullrequestreview-13220090:

@@ -1,2 +1,23 @@

istioapi
-API, Config Definitions and Standard Vocabulary definitions for the istio project
+API, Config Schema Definitions and Standard Vocabulary definitions for the istio project
+
+
+Usually definitions are in the form on .proto files.
+
+##
+All other repositories can depend on istioapi
+This repository will not depend on any other repos
+
+##
+We may check-in generated .pb.go and .pb.cc files here.
+
+
+## API definitions
Shouldn't everything be API? Config schemas are going to be Kube APIs via
TRPs.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istioapi/pull/1#pullrequestreview-13220090, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AVucHQ4_5UXigrQ0IVLzz2xGG0VDMp6jks5rIategaJpZM4LOjmN
.


EventsCenter: remove registerHandler method return type #598

#598
Cleaning up this issue so that I can skip adding header comment for this trivial method.

v1
@eugenepeh submitted v1 for review.

[v1 1/1] EventsCenter: remove registerHandler method return type

(:books: Archive)

Checkout this PR version locally
git fetch https://github.com/se-edu/addressbook-level4.git refs/pr/630/1/head:BRANCHNAME

where BRANCHNAME is the name of the local branch you wish to fetch this PR to.

v2
@eugenepeh submitted v2 for review.

[v2 1/1] EventsCenter: change registerHandler method return type

(:books: Archive) (:chart_with_upwards_trend: Interdiff between v1 and v2)

Checkout this PR version locally
git fetch https://github.com/se-edu/addressbook-level4.git refs/pr/630/2/head:BRANCHNAME

where BRANCHNAME is the name of the local branch you wish to fetch this PR to.


Problem with Lets Encrypt Certificates and the Git client

Describe the bug

I have found a bug in the release 1.4.0 and 1.5.0 unfortunately in connection with the Git Client and Lets Encrypt Certificates. I have a self-hosted GitLab server.
Unfortunately, when I do a Git push, I get the following error message:
fatal: unable to access 'https://git.isp-serverfarm.de/laravel-training/laravel-blog-v2.git': server certificate verification failed. CAfile: none CRLfile: none
Screenshots

Steps to reproduce

Use Gitlab self-hosted with lets encrypt certificates and load the workspace as usual.
Expected behavior

A simple "sudo update-ca-certificates" does not work. Also the URL https://git.isp-serverfarm.de can be accessed with curl or wget without problems. Only the Git client itself has a problem with it.
Additional information

There are alot of moving parts on this one from me to even to be able to test it out. Not super sure where to start.

Something along the lines of this?
https://docs.gitlab.com/ee/integration/gitpod.html

Did you register your self hosted gitlab instance with Gitpod?
https://www.gitpod.io/docs/gitlab-integration#registering-a-self-hosted-gitlab-installation

I've been using GitPod with my GitLab self-hosted instance since GitPod first supported it.
So I don't see any problem with the GitPod integration at the moment.
I think GitPod has changed something in the base image again and that's why the build doesn't work anymore.
However, I just created a new repository and then used the "gitpod/workspace-mysql" image as a base. Here there are no problems when pushing to the gitlab instance from me.

So, I tried something again. Changed the base image to this day.
FROM gitpod/workspace-mysql:legacy-dazzle-v1
With this, both the build of the workspace and my git push works again.

Ok, have to correct myself a bit. With the FROM gitpod/workspace-mysql:legacy-dazzle-v1 only the 1.4.0 image works again. With the 1.5.0 image I still have problems with the Git client and my own GitLab instance.

Ok we can fix this by pinning the Gitpod workspace-mysql image version to a specific date:
https://github.com/gitpod-io/workspace-images/issues/595#issuecomment-1033951855
Not my favorite solution but perhaps we can create and maintain our own underlying image in the future.

@strausmann
Can you try this development branch in gitlab and verify the fix?

workspace-mysql::2022-02-04-10-54-10 did not work for me

It looks like there is no version of workspace-mysql where nginx, PHP, and Apache are retained but ca certificates work. Since these are the only two tags we have to work with:
(contains nginx, PHP and apache but has buggy ca certificates)
https://hub.docker.com/layers/gitpod/workspace-mysql/legacy-dazzle-v1/images/sha256-d75a730768eba3eacf854e4dd671967a2c07d81cee44ec51e85c1265ec1b2baf?context=explore
and
(does not contain nginx, PHP and apache but ca certificates are fixed?)
https://hub.docker.com/layers/gitpod/workspace-mysql/2022-02-04-10-54-10/images/sha256-89974ad5bffbd7f1a5f68070f81e414d9a4d1be923d0b606177711462c83c1df?context=explore
Please advise how I can workaround this.
Its looking like I will have to use workspace-full and then integrate mysql on my own? I would really rather not do that to hotfix this breaking change in a bunch of my projects, including portfolio pieces.

@strausmann
I am hoping for a suggested workaround (from Gitpod) that actually works.
Otherwise I will have to integrate mysql in this projects gitpod.Dockerfile and build the project on workspace-full:some pinned-date.
Thoughts?

@strausmann
I will assume this has been fixed since it was a regression issue that was addressed by Gitpod.
Feel free to re-open this after v.16 is released if you continue to experience the issue.

@apolopena
Sorry. forgot to give feedback. :-)
With the new base image from GitPod my LE problem is also fixed.


How do I limit aider to only look at one file?

Issue
Fundamentally, this is what I am trying to do:
const prompt = multiline`
  Convert all snake_case column names in SQL \`SELECT\` statements to camelCase by adding aliases where needed.

  - You are working with PostgreSQL syntax.
  - Only update columns that contain underscores and are not already aliased.
    **Example**: \`SELECT sp1.llm_generated_title\` becomes \`SELECT sp1.llm_generated_title "llmGeneratedTitle"\`.
  - If a column already has an alias and alias contains _ (underscores), change it to camelCase.
    **Example**: \`SELECT sp1.llm_generated_title generated_title\` becomes \`SELECT sp1.llm_generated_title "generatedTitle"\`.
  - Only add alias to the parts of the query that affect the query result.
  - If a function does not have an alias, use camelCase for the function name, e.g. \`SELECT add_contra_money(('CAD', 100.11)::contra_money, ('CAD', 200.22)::contra_money)\` must be aliased as "addContraMoney".
  - If the query is invoked using \`pool.oneFirst\` or \`pool.anyFirst\` or \`pool.manyFirst\`, then avoid adding aliases altogether.
  - Ignore sub-queries. We only want to alias columns that describe the final result of the query.
`;

const main = async () => {
  for (const file of files) {
    await $`aider --no-gui --4o --no-gui --no-auto-commits --yes --openai-api-key ${OPENAI_API_KEY} ${file}  --message ${prompt}`;
  }
};

main();

It looks like aider is too smart for its own good, and pulls a lot of extra context which is not useful for the refactor. From what I can tell, the provided context includes too much noise, and as a result the suggested changes do not follow the instructions.
Version and model info
aider 0.62.1
Tried various models (gpt4o, mini). Same results.
I think the problem is not the model, but the prompts used to perform the update.

Thanks for trying aider and filing this issue.
You can disable the extra context with --map-tokens 0.

Voila ! works perfect. Thank you

@paul-gauthier Do you happen to know if there is some odd cache or anything like that I should be aware of?
I am surprised by some of the changes, e.g.
test('can aggregate subtraction amounts with the same currency', async () => {
  await runWithDatabaseConnectionPool('postgres', async ({ pool }) => {
    const usdAmount = await pool.oneFirst(sql.unsafe`
      WITH money_values AS
      (
        SELECT unnest(ARRAY[
          ('USD', 100.00),
          ('USD', 50.00),
          ('USD', 25.00)
        ]::contra_money[]) as value
      )

-      SELECT subtract_contra_money(value) FROM
+      SELECT subtract_contra_money(value) "subtractContraMoney" FROM
      money_values
    `);

    expect(usdAmount).toStrictEqual(Money.fromUsdDollars(25));

    const cadAmount = await pool.oneFirst(sql.unsafe`
      WITH money_values AS
      (
        SELECT unnest(ARRAY[
          ('CAD', 100.00),
          ('CAD', 50.00),
          ('CAD', 25.00)
        ]::contra_money[]) as value
      )

      SELECT subtract_contra_money(value) FROM
      money_values
    `);

    expect(cadAmount).toStrictEqual(Money.deserialize('CAD:25'));
  });
});

This is despite the prompt very explicitly saying to ignore queries if they are invoked with oneFirst.
const prompt = multiline`
  Convert all snake_case column names (must contain underscore) in \`SELECT\` statements to camelCase by adding aliases as needed.

  - You are working with PostgreSQL syntax.
  - Only update top-level SELECT statements; ignore nested SELECT statements.
  - Only update columns that contain underscores and are not already aliased.
    **Example**: \`SELECT sp1.llm_generated_title\` becomes \`SELECT sp1.llm_generated_title "llmGeneratedTitle"\`.
  - If a column already has an alias and alias contains _ (underscores), change it to camelCase.
    **Example**: \`SELECT sp1.llm_generated_title generated_title\` becomes \`SELECT sp1.llm_generated_title "generatedTitle"\`.
  - Only add alias to the parts of the query that affect the query result.
  - If a function does not have an alias, use camelCase for the function name, e.g. \`SELECT add_contra_money(('CAD', 100.11)::contra_money, ('CAD', 200.22)::contra_money)\` must be aliased as "addContraMoney".
  - Ignore queries that are invoked using \`oneFirst\` or \`anyFirst\` or \`manyFirst\`.
`;

If I paste the exact same code / prompt into ChatGPT and use gpt-4o-2024-08-06, it doesn't do these mistakes.


ページ削除時のリンクチェック

Coverage increased (+0.05%) to 85.717% when pulling 09e3fa643ca14d0f7a6b048568fdc2fea4b59dc0 on itowtips:confirmation into aae9b5d04754ff7e676ee038c73a337dc8e336af on shirasagi:master.

Coverage decreased (-0.08%) to 84.777% when pulling 2c9dbbf5c861eb9d7cbfbb1b896ff80945c5b623 on itowtips:confirmation into 4d9bdea1738a02a2ce0929ed05968871f6211f6f on shirasagi:master.

Coverage decreased (-0.04%) to 84.817% when pulling 2c9dbbf5c861eb9d7cbfbb1b896ff80945c5b623 on itowtips:confirmation into 4d9bdea1738a02a2ce0929ed05968871f6211f6f on shirasagi:master.

Coverage increased (+0.008%) to 84.868% when pulling 75ca3d687a3db2f42c1506dc33f30c9758f118ca on itowtips:confirmation into 4d9bdea1738a02a2ce0929ed05968871f6211f6f on shirasagi:master.


Highlights Crashing Chatterino

Adding New Word is fine, changing colour of word will create crash.
White dialogue box appears then crash happens.
running 2.3.3

I have 38 highlight words. i know this is excessive, but i use it for moderating 4 partners and a bunch of affiliates

Please use issue templates, those exist for a reason.
Can you replicate this crash with any other Chatterino version? Please also include the version commit that your application is on. You can find full version information in Settings > About.

I clicked on the link that said "Bug Report" from a thread that i was linked to from the discord. Not my fault it linked me to here.

Chatterino 2.3.3 (commit c8c28d7f5)
Built with Qt 5.15.2, Windows SDK, MSVC 192930038
Windows 21H1 (OS Build 19043.1110)

I clicked on the link that said "Bug Report" from a thread that i was linked to from the discord.

I've asked for this comment to be updated to prevent any issues for anybody else in the future and it has been. 🙂

Unable to replicate. Closing as stale, please create a new issue if the crash still is still there.


Option for using / not-using space delimited namespaces.

Personally I find logger names with spaces more readable 'Something Long Here"  rather than "Something:LongHere".
Not very fully realised; but hopefully something useful if / when updates to debug.js happen.

No worries - (yah figured it was more of a personal taste / much-of-a-muchness thing ).
Thanks for debug.js :)


I want to make a plug-in framework, but I don't intend to use MEF. Can I use style to implement it directly?

For example, using AddModules, how to directly obtain all the independent class library projects that inherit StyletIoCModule through style? Or are there any good examples?

You can use Stylet, although you'll also need to use plenty of reflection. Other more complex IoC frameworks have some more built-in tools to help you.
You'll have to use reflection to list/load all of your assemblies, then use Assembly.GetTypes() to get all of the types, and find the ones which inherit from StyletIoCModule. Then use Activator.CreateInstance to instantiate them.

OK, I see. Thank you very much~

I'm doing this right now, for what it's worth, and seems to work okay. I did run into a problem #109 in a certain scenario, though. You might want to take a look at my bootstrapper to see how I handle this.


On Android Kitkat nothing is getting triggered

Cordova - 6.2.0
Android API - 19
Android Version - 4.4.4
Device - Nexus 5

When I call this method from android, nothing is happening. Code works completely fine on iOS.
var options = {
        maxNum: 9
};

window.MultiImagePicker.getPictures(
  (selectedPath) => {
    for (var i = 0; i < selectedPath.length; i++) {
        alert('Image URI: ' + selectedPath[i]);
    }
  },
  (error) => {
    alert("error");
  },
  options
);

Hmm that's wierd. Is there any logs or errors?

Mistake was on my side.


Use the number input example instead of the password input example

The number input docs page incorrectly shows the password input code example. This PR shows the correct number input code example.

@squatto Thank you very much for the contribution!

You're very welcome!

@joaopalopes24 do you need me to do something with signing my commits in order for this to be merged?

@squatto No changes are required. We're just finalizing some tweaks to merge when we publish the release.

@joaopalopes24 OK great, thank you!
I'm currently participating in Hacktoberfest. If it wouldn't be too much of a pain, could you label this PR with "hacktoberfest-accepted"? Thank you so much!

@squatto of course, thanks for contributing

Thank you @PH7-Jack, I appreciate it!


delete_forward not working

Doesn't matter what key I try mapping to delete_foward, it doesn't seem to work. I am using 0.91.6. Nothing interesting in the log. Let me know if you need anything to debug.

delete_forward is working now I guess (I have had F12 mapped to it for some time).


Having issue with deselect cells

Hello I'm using last version of JTAppleCalendar (7.1.6)
I have a wired issue, as you can see in this image, if I select a date and that date is before last date that I selected it won't deselect other cells, funny thing is it's fine for the first time but after that I have this problem!

look at this gif file too:

here is my code:
 func calendar(_ calendar: JTAppleCalendarView, didSelectDate date: Date, cell: JTAppleCell?, cellState: CellState) {
    if endDate != nil {
        startDate = nil
        endDate = nil
        startDate = date
        persianCalendar.deselectAllDates(triggerSelectionDelegate: false)
        handleCellSelected(cell: cell, cellState: cellState)
        handleCellTextColor(cell: cell, cellState: cellState)

    } else if startDate != nil {
        if date > startDate! {
            persianCalendar.selectDates(from: startDate!, to: date,  triggerSelectionDelegate: false, keepSelectionIfMultiSelectionAllowed: true)
            endDate = date
        } else {
            endDate = nil

            startDate = nil
            persianCalendar.deselectAllDates(triggerSelectionDelegate: false)
            startDate = date
            handleCellSelected(cell: cell, cellState: cellState)
            handleCellTextColor(cell: cell, cellState: cellState)

        }
    } else {
        startDate = date
        handleCellSelected(cell: cell, cellState: cellState)
        handleCellTextColor(cell: cell, cellState: cellState)
    }
}

func calendar(_ calendar: JTAppleCalendarView, didDeselectDate date: Date, cell: JTAppleCell?, cellState: CellState) {

    if endDate != nil {
        startDate = date

        endDate = nil
        persianCalendar.deselectAllDates()
        if let validCell = cell as? CalendarCell {
            validCell.dateLabel.textColor = .white
            validCell.selectedView.backgroundColor = UIColor(displayP3Red: 255/255, green: 105/255, blue: 104/255, alpha: 1)
            validCell.selectedView.isHidden = false
        }

    } else {
        handleCellSelected(cell: cell, cellState: cellState)
        handleCellTextColor(cell: cell, cellState: cellState)
    }
}
  func handleCellSelected(cell: JTAppleCell?, cellState: CellState) {

    if let validCell = cell as? CalendarCell {
        if cellState.isSelected && cellState.dateBelongsTo == .thisMonth {
            validCell.selectedView.isHidden = false
        } else {
            validCell.selectedView.isHidden = true
            validCell.boarderView.borderColor = .white
        }

    }
}

fixed it, persianCalendar.deselectAllDates() function has problem I don't know what! I changed my code to this and it's fine now:
 func calendar(_ calendar: JTAppleCalendarView, didSelectDate date: Date, cell: JTAppleCell?, cellState: CellState) {
    if endDate != nil {
        self.persianCalendar.deselectDates(from: self.startDate!, to: self.endDate!, triggerSelectionDelegate: false)

        startDate = nil
        endDate = nil
        startDate = date

//            persianCalendar.deselectAllDates(triggerSelectionDelegate: false)
handleCellSelected(cell: cell, cellState: cellState)
handleCellTextColor(cell: cell, cellState: cellState)
    } else if startDate != nil {
        if date > startDate! {
            persianCalendar.selectDates(from: startDate!, to: date,  triggerSelectionDelegate: false, keepSelectionIfMultiSelectionAllowed: true)
            endDate = date
        } else {

            self.persianCalendar.deselectDates(from: self.startDate!, to: self.startDate!, triggerSelectionDelegate: false)
            startDate = date
            handleCellSelected(cell: cell, cellState: cellState)
            handleCellTextColor(cell: cell, cellState: cellState)

        }
    } else {
        startDate = date
        handleCellSelected(cell: cell, cellState: cellState)
        handleCellTextColor(cell: cell, cellState: cellState)
    }
}


Messages not immediately visible after sending or retrieving

My Setup

Operating System: Windows 10 Pro
App Version: 3.1.1
Installation type: exe

[x] I have tested with the latest version
[x] I can simulate the issue easily

Description

Current Behavior

When I send or retrieve messages they sometimes take a minute to become visible. I can hear the sound that I got a message but it's not visible. Also when I send a message it takes a minute to become visible to the other person. This happens for all my co workers.
Expected Behavior

The message should be vissible immediately after retrieving and sending.

Please test that issue in web client. If it behaves same, that means it's servers issue, but not Electron application issue.
And if your server have few instances, check how they interoperate.

Please test that issue in web client. If it behaves same, that means it's servers issue, but not Electron application issue.
And if your server have few instances, check how they interoperate.

We have the same issue on the web client. You think our server is the issue?

We have the same issue on the web client. You think our server is the issue?

We have the same issue on the web client. You think our server is the issue?

I think you do have a problems at server level.
If you have more then 1 instance, check that doc https://docs.rocket.chat/installation/docker-containers/high-availability-install
INSTANCE_IP is really important in that case.

We have the same issue on the web client. You think our server is the issue?

I think you do have a problems at server level.
If you have more then 1 instance, check that doc https://docs.rocket.chat/installation/docker-containers/high-availability-install
INSTANCE_IP is really important in that case.

I just learned that we don't have our own RocketChat server. It is hosted by RocketChat. So what can be the problem?

I just learned that we don't have our own RocketChat server. It is hosted by RocketChat. So what can be the problem?

I just learned that we don't have our own RocketChat server. It is hosted by RocketChat. So what can be the problem?

In that case you must write to<EMAIL_ADDRESS>

I just learned that we don't have our own RocketChat server. It is hosted by RocketChat. So what can be the problem?

In that case you must write to<EMAIL_ADDRESS>

Please test if the latest release did solve this. If not please reopen the issue or create a new one and tell us. I will be closing this one for now.
Thank you!


Webpack out-of-box compatibility

It seems Webpack tries to build in its AMD shim when bundling the .max version in your bundle/ directory. I'm assuming that it finds what it thinks are references to define being used, but I haven't singled out exactly which lines of code are causing it to do that yet.

Figured it out and raised a PR. Unfortunately, I don't know of a good way to write a test for this (but I did run your tests and they pass).

the and module is https://github.com/WebReflection/document-register-element/blob/master/build/document-register-element.max.amd.js
however, I use define as function name in the generic one which is wrapped in a closure so I'm not sure why that bothers webpack.


experiment with php-vips as the backend

Hello,
This experimental PR swaps the backend from imagick to php-vips:
https://github.com/libvips/php-vips
Tests pass and it seems to work, but it is not production-ready ... benchmarking and further tests only! I added some comments for things that would need fixing or are only half-done.
php-vips uses poppler rather than ghostscript for PDF rendering. Poppler is GPL, not AGPL, so php-vips is able to link directly to the library rather than being forced to go via temporary files. Additionally, like PDFium, poppler is based on a desktop PDF previewer, so it renders anti-aliased images. There's no need to render at high-resolution and downsample.
The downside is that php-vips will always generate RGB output, even for CMYK PDFs.
This change has the following potential benefits:
Speed
php-vips is more than 10x faster for this kind of processing. Here are two simple benchmarks:
#!/usr/bin/env php
<?php

require __DIR__ . '/vendor/autoload.php';

use Jcupitt\Vips;

foreach (range(1, count($argv)) as $i) {
  $image = Vips\Image::newFromFile($argv[$i]);
  $n_pages = $image->get("n-pages");
  echo($argv[$i] . " has " . $n_pages . " pages\n");

  foreach (range(0, $n_pages - 1) as $n) {
    echo("  rendering page " . $n . " ...\n");
    $page = Vips\Image::newFromFile($argv[$i], [
      "dpi" => 30,
      "page" => $n,
      # this enables image streaming
      "access" => "sequential"
    ]);
    $page->writeToFile($argv[$i] . "_page_" . $n . ".png");
  }
}

And:
#!/usr/bin/env php
<?php

for ($i = 1; $i < count($argv); $i++) {
  $imagick = new Imagick();
  $imagick->setResourceLimit(6, 1);
  $imagick->setResolution(30, 30);
  $imagick->readImage($argv[$i]);
  $pages = $imagick->getNumberImages();
  echo($argv[$i] . " has " . $pages . " pages\n");

  for ($x = 0; $x < $pages; $x++) {
    echo("  rendering page " . $x . " ...\n");
    $imagick->readImage($argv[$i] . "[" . $x . "]");
    $imagick->setImageFormat("png");
    $imagick->writeImage($argv[$i] . "_page_" . $x . ".png");
  }

  $imagick->clear();
  $imagick->destroy();
}

On this 2015 laptop, I see:
$ time ./convert-vips.php ~/pics/nipguide.pdf
/home/john/pics/nipguide.pdf has 58 pages
  rendering page 0 ...
  rendering page 1 ...
...
  rendering page 56 ...
  rendering page 57 ...

real	0m1.765s
user	0m1.645s
sys	0m0.230s

And:
$ time ./convert-imagick.php ~/pics/nipguide.pdf
/home/john/pics/nipguide.pdf has 58 pages
  rendering page 0 ...
  rendering page 1 ...
...
  rendering page 56 ...
  rendering page 57 ...

real	0m21.457s
user	0m19.887s
sys	0m1.476s

Quality
php-vips renders high-quality, anti-aliased PDFs. There's no need to render at a high resolution and downsample.
Memory use
Memory use is lower as well, especially with large PDFs or with high-resolution output. For example:
$ /usr/bin/time -f %M:%e vips pdfload nipguide.pdf y.png --dpi 600 --page 12
183184:2.67
$ /usr/bin/time -f %M:%e convert -density 1200 nipguide.pdf[12] -depth 8 -resize 50% x.png
1917444:21.19
$ vipsheader x.png y.png
x.png: 4961x7016 uchar, 4 bands, srgb, pngload
y.png: 4961x7016 uchar, 4 bands, srgb, pngload

Down from 21.2s and 2GB of memory to 2.7s and 180MB.

I'm going to pass on this one for now.
Thanks!


SyntaxError: Unexpected token '#', "

When i run PHPCBF : fix this file in vscode
it's fixing my file but add a '#' at start of my file.
Example :
<?php
if (! defined('ABSPATH')) {
    exit; // Exit if accessed directly
}

will result :
# <?php
if (! defined('ABSPATH')) {
    exit; // Exit if accessed directly
}

If i run the command line :
C:\Users\jinx\AppData\Roaming\Composer\vendor\bin\phpcbf.bat -q --standard=WordPress "c:\wamp64\www\projects\php-test\test.php"
file is fixed without # but there is my log :
C:\Users\jinx\AppData\Roaming\Composer\vendor\bin\phpcbf.bat -q --standard=WordPress "c:\wamp64\www\projects\php-test\test.php"                                                                                \php-test\test.php"

#
PHPCBF RESULT SUMMARY
----------------------------------------------------------------------
FILE                                                  FIXED  REMAINING
----------------------------------------------------------------------
C:\wamp64\www\projects\php-test\test.php              1      3
----------------------------------------------------------------------
A TOTAL OF 1 ERROR WERE FIXED IN 1 FILE
----------------------------------------------------------------------

Time: 481ms; Memory: 12MB

We can see a # above PHPCBF RESULT SUMMARY
Do you have any idea where the problem comes from?

Hi @almeidaz based on the code, it looks like this is a WordPress plugin file.
I just tested your sample code on my setup, and I cannot replicate the error, so it looks like there's either something on your local Windows environment that's causing this, or the version of phpcbf you have installed.
Can you tell me what your local development environment set up is, and how you installed phpcs and phpcbf.

Win10
install via composer global require squizlabs/php_codesniffer
 composer -V
Composer version 2.7.1 2024-02-09 15:26:28

phpcs --version

# PHP_CodeSniffer version 3.9.0 (stable) by Squiz and PHPCSStandards

my settings.json
"php.validate.executablePath": "C:\\wamp64\\bin\\php\\php7.4.33\\php.exe",
"php.executablePath": "C:\\wamp64\\bin\\php\\php7.4.33\\php.exe",
"phpsab.standard": "WordPress",
// "phpsab.standard": "PSR2",
"phpsab.executablePathCS": "C:\\Users\\jinx\\AppData\\Roaming\\Composer\\vendor\\bin\\phpcs.bat",
"phpsab.executablePathCBF": "C:\\Users\\jinx\\AppData\\Roaming\\Composer\\vendor\\bin\\phpcbf.bat",
"phpsab.debug": true,

  "[php]": {
    "editor.defaultFormatter": "valeryanm.vscode-phpsab"
    // "editor.formatOnSave": true
  },

where phpcs
C:\Users\jinx\AppData\Roaming\Composer\vendor\bin\phpcs
C:\Users\jinx\AppData\Roaming\Composer\vendor\bin\phpcs.bat

where phpcbf
C:\Users\jinx\AppData\Roaming\Composer\vendor\bin\phpcbf
C:\Users\jinx\AppData\Roaming\Composer\vendor\bin\phpcbf.bat

Do you have also the # when you type phpcs --version like :
# PHP_CodeSniffer version 3.9.0 (stable)...

Do you have also the # when you type phpcs --version like :

No, when I run phpcs --version I don't get the #
PHP_CodeSniffer version 3.9.0 (stable) by Squiz and PHPCSStandards

Do you get the # when you run any other terminal command in Windows, or only phpcs --version

yes, only with phpcs command. it's weird 😅

Haha that's funny, I was just about to ask if you'd been editing your phpcs config...glad you found it.