K8SPXC-933: Fix deprecated affinity keys

See:
https://kubernetes.io/docs/reference/labels-annotations-taints/#failure-domainbetakubernetesioregion
https://kubernetes.io/docs/reference/labels-annotations-taints/#failure-domainbetakubernetesiozone

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Test name
Status

scheduled-backup
failed

recreate
passed

upgrade-haproxy
passed

scaling
passed

big-data
passed

storage
passed

haproxy
passed

scaling-proxysql
passed

self-healing
passed

self-healing-chaos
failed

init-deploy
passed

restore-to-encrypted-cluster
passed

security-context
passed

upgrade-proxysql
passed

limits
passed

self-healing-advanced
passed

self-healing-advanced-chaos
failed

demand-backup
passed

smart-update
failed

monitoring-2-0
passed

operator-self-healing
passed

operator-self-healing-chaos
failed

affinity
failed

one-pod
passed

pitr
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

demand-backup-encrypted-with-tls
passed

tls-issue-self
passed

tls-issue-cert-manager
failed

upgrade-consistency
failed

tls-issue-cert-manager-ref
passed

validation-hook
passed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/1079/commits/2aad9a465f63a4953292c27ef98777976b82c531
image: perconalab/percona-xtradb-cluster-operator:PR-1079-2aad9a46

link to ticket, please

Test name
Status

scaling
passed

storage
passed

recreate
passed

big-data
passed

scheduled-backup
passed

upgrade-haproxy
passed

haproxy
passed

self-healing
passed

self-healing-chaos
passed

scaling-proxysql
passed

init-deploy
passed

restore-to-encrypted-cluster
passed

security-context
passed

upgrade-proxysql
passed

self-healing-advanced
passed

self-healing-advanced-chaos
passed

limits
passed

demand-backup
passed

smart-update
passed

monitoring-2-0
passed

operator-self-healing
passed

operator-self-healing-chaos
passed

affinity
failed

one-pod
passed

pitr
passed

auto-tuning
passed

upgrade-consistency
passed

proxysql-sidecar-res-limits
passed

users
passed

demand-backup-encrypted-with-tls
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

tls-issue-cert-manager-ref
passed

validation-hook
failed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/1079/commits/5387eaf01d0342475a7de5f7b6e924ae2807980a
image: perconalab/percona-xtradb-cluster-operator:PR-1079-5387eaf0

Test name
Status

upgrade-haproxy
passed

scheduled-backup
failed

storage
passed

big-data
passed

scaling
passed

recreate
passed

cross-site
passed

haproxy
passed

scaling-proxysql
passed

self-healing
passed

self-healing-chaos
failed

init-deploy
passed

restore-to-encrypted-cluster
passed

security-context
passed

upgrade-proxysql
passed

limits
passed

self-healing-advanced
passed

self-healing-advanced-chaos
failed

demand-backup
passed

monitoring-2-0
passed

operator-self-healing
passed

operator-self-healing-chaos
failed

smart-update
passed

affinity
passed

one-pod
passed

auto-tuning
passed

pitr
passed

proxysql-sidecar-res-limits
passed

users
passed

upgrade-consistency
passed

demand-backup-encrypted-with-tls
failed

tls-issue-self
passed

tls-issue-cert-manager
failed

tls-issue-cert-manager-ref
failed

We run 34 out of 36

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/1079/commits/71fdb7cdcac078da66945b24f4fb393dbb66da12
image: perconalab/percona-xtradb-cluster-operator:PR-1079-71fdb7cd

Test name
Status

storage
passed

scheduled-backup
passed

big-data
passed

cross-site
passed

haproxy
passed

self-healing
passed

scaling
passed

upgrade-haproxy
passed

init-deploy
passed

recreate
passed

self-healing-chaos
failed

scaling-proxysql
passed

limits
passed

restore-to-encrypted-cluster
passed

security-context
passed

demand-backup
passed

monitoring-2-0
passed

pitr
passed

affinity
passed

demand-backup-encrypted-with-tls
passed

one-pod
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

upgrade-proxysql
passed

smart-update
passed

upgrade-consistency
passed

self-healing-advanced
passed

self-healing-advanced-chaos
failed

tls-issue-cert-manager-ref
passed

operator-self-healing
passed

operator-self-healing-chaos
failed

validation-hook
passed

proxy-protocol
passed

We run 36 out of 36

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/1079/commits/71fdb7cdcac078da66945b24f4fb393dbb66da12
image: perconalab/percona-xtradb-cluster-operator:PR-1079-71fdb7cd

Test name
Status

scaling
passed

cross-site
passed

scheduled-backup
passed

scaling-proxysql
passed

haproxy
passed

recreate
passed

storage
passed

big-data
passed

security-context
passed

init-deploy
passed

restore-to-encrypted-cluster
passed

self-healing
passed

upgrade-haproxy
passed

limits
passed

demand-backup
passed

self-healing-chaos
passed

pitr
passed

monitoring-2-0
passed

demand-backup-encrypted-with-tls
passed

affinity
passed

one-pod
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

tls-issue-cert-manager-ref
passed

validation-hook
passed

proxy-protocol
passed

upgrade-proxysql
passed

smart-update
passed

upgrade-consistency
passed

self-healing-advanced
passed

self-healing-advanced-chaos
passed

operator-self-healing
passed

operator-self-healing-chaos
passed

We run 36 out of 36

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/1079/commits/71fdb7cdcac078da66945b24f4fb393dbb66da12
image: perconalab/percona-xtradb-cluster-operator:PR-1079-71fdb7cd


[WIP] net: change enc28j60 to new SPI API

Signed-off-by: Matthias Boesl<EMAIL_ADDRESS>

closed in favor of #662


Async ZIP Streaming?...

Hi
I was wondering if it was possible to create ZIP files on the fly so that one could stream them through an HTTP connection, for instance. But reading the issues and the docs I am not sure if that's supported at all...

Sorry for the delayed response here. This would definitely be possible - ZipFileWriter is generic over AsyncWrite. So you'd probably want to get a TcpStream and then use the with_tokio() method.

I see, yes...but it seems that the same is not possible for extracting a ZIP, correct? Because of the seek requirement on the reader...

I'm trying to use a tokio DuplexStream for the writer. When trying to add files to the ZipFileWriter, I currently always get a UpstreamReadError(Kind(BrokenPipe)). Even if the data used is not coming from a file but static:
                let ze = ZipEntryBuilder::new(ZipString::from("test"), Compression::Deflate).build();
                let data = b"This is an example file.";
                w.write_entry_whole(ze, data).await.expect("Failed to write entry");

And it seems with a custom writer implementation it stops writing to the zip file exactly after 32 entries,.

Ok, maybe that helps somebody else too, here is some sample code for streaming ZIP receiving and unpacking as well as streaming ZIP generation with the Rocket web framework:
Receiving a ZIP file:
#[post(
"/zip",
format = "plain",
data = "<data>"
)]
async fn zip_receive(
    data: Data<'_>,
)  {
    let ds = data.open(1.tibibytes());
    let mut reader = ZipFileReader::with_tokio(ds);

    let mut buf = [0;65536];

    loop {
        if let Some(mut x) =  reader.next_with_entry().await.expect("Failed to open reader") {

            let e = x.reader().entry();
            println!("{}", e.filename().as_str().expect("Failed to extract zip files"));

            // Actual extraction would need to happen here,...
            loop {
                let read = x.reader_mut().read(&mut buf).await.expect("Failed to read from entry");

                if read == 0 {
                    break;
                }
            }
            reader = x.done().await.expect("Failed to finish entry");
        } else {
            break;
        }

    }

    ()
}

Generation a ZIP file:
#[get("/zip")]
async fn zip_stream() -> ByteStream![Vec<u8>]   {
    let (sender, mut receiver) = duplex(65536);
    let base_path = std::path::Path::new("/some/dir/to/zip");
    let t = tokio::task::spawn( async move {
        let mut w = ZipFileWriter::with_tokio(sender).force_zip64();

        for e in WalkDir::new(&base_path) {
            let e = e.unwrap();

            if e.metadata().unwrap().is_file() {
                let p = e.path().strip_prefix(base_path.parent().unwrap()).unwrap().to_str().unwrap();
                let ze = ZipEntryBuilder::new(ZipString::from(p), Compression::Stored).build();
                let data = tokio::fs::read(e.path()).await.expect("Could not read file");
                w.write_entry_whole(ze.clone(), &data).await.expect("Could not write entry");
            }
        }

        w.close().await.expect("Failed to close writer");
    });

    let mut buf = vec![0; 65536];

    ByteStream! {
        loop {
            let read = receiver.read(&mut buf).await.expect("F");
            if read > 0 {
                yield buf[0..read].to_vec();
            } else {
                if t.is_finished() {
                break;
                }
            }
        }
    }
}

Would this support folders too?

@Wamy-Dev yes, as it walks the directory tree that would work. If you really want to write the ZIP in streaming fashion you should not use write_entry_whole (as that needs to allocate at least as much memory as the file written is in size),...but write_entry_stream (the API is a bit different),...happy to paste a sample.

That would be awesome! Thank you. Yeah, I deal with some large files so streaming is the best option.

Thank you so much! One last thing, I've been searching for it around here, but I cant seem to figure it out. Is there a way to calculate the final size of the zip so I can pass it through a header?
I tried adding up the file sizes and the headers but my math might be wrong, since it always fails at the end.

Not that I am ware of. I use a streaming receiver (javascript stream to file),...

ah, thanks anyways

There is still the header for every ZIP file (although not fully populated) and the central directory. So you cannot just add all the file sizes even if using 'Stored'.
Then there might also be some padding,...I think it should be doable, but you'd need to test.

using large files, the code you provided @inzanez still takes up an equal amount of memory :(. Is there a way around this? I can't seem to figure it out. For example, I am trying a 150gb zip of lots of medium sized files, around 3-4gb each. Perhaps it opens the entire file in memory?


pkgs.mkShell is executing scripts added to buildInputs

Describe the bug
Using pkgs.writeScript" in pkgs.mkShellasbuildInputstriggers them script to be run on runningnix-shell`, which unitutive
To Reproduce
using this shell.nix
{ pkgs ? import <nixpkgs> { } }:
let
in pkgs.mkShell {
  buildInputs = with pkgs; [
    (pkgs.writeScript "hack-writeScript" ''
      echo owned-writeScript >> /tmp/hacked
    '')
    (pkgs.writeScriptBin "hack-writeScriptBin" ''
      echo owned-writeScriptBin >> /tmp/hacked
    '')
    (pkgs.writers.writeBash "hack-writeBash" ''
      echo owned-writeBash >> /tmp/hacked
    '')
    (pkgs.writers.writeBashBin "hack-writeBashBin" ''
      echo owned-writeBashBin >> /tmp/hacked
    '')
  ];
}

Run nix-shell --run 'cat /tmp/hacked' and you get:
owned-writeScript
owned-writeBash

Expected behavior
The file /tmp/hacked shouldn't be created at all. The scripts are not just build, they are executed.
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.97, NixOS, 19.09.git.1dadefe (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.2`
 - nixpkgs: `/var/src/nixpkgs`

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute:
- pkgs.mkShell
- pkgs.writeShell
- pkgs.writers.writeBash
- pkgs.writers.writeDash
# a list of nixos modules affected by the problem
module:

This is intentional. Scripts as buildInputs are used as setup hooks, i.e. stdenv's setup sources them (doesn't just execute them) so they can provide additional functionality to a build. Don't nix-shell expressions you don't trust :)

I thought this is what shellHook is for?
Ok I guess I will just accept it. Are there all text files sourced, which I name in buildInputs?

Yeah, this is an unusual feature in the stdenv, given that this functionality was already provided via setup-hooks in a more explicit way: b23dbb1a5dffbfa3abb47fcd0f1579ac2e6f29fc

Yeah, I guess because I don't like implicit behavior (and deeply believed so far NixOS is not doing implicit things) I guess I don't like this behavior. But I don't have to like it though.
But it might make sense to document this behavior. I will make myself a note to do that, and close this issue. Thanks for your responses.


Cannot add connection

Since the update VSCode I cannot use the +-button on the right to add a new connection.
Version: 1.59.0 (user setup)
Commit: 379476f0e13988d90fab105c5c19e7abc8b1dea8
Date: 2021-08-04T23:13:12.822Z
Electron: 13.1.7
Chrome: 91.0.4472.124
Node.js: 14.16.0
V8: 9.1.269.36-electron.0
OS: Windows_NT x64 10.0.18363

Since the update VSCode I cannot use the +-button on the right to add a new connection.
Version: 1.59.0 (user setup)
Commit: 379476f0e13988d90fab105c5c19e7abc8b1dea8
Date: 2021-08-04T23:13:12.822Z
Electron: 13.1.7
Chrome: 91.0.4472.124
Node.js: 14.16.0
V8: 9.1.269.36-electron.0
OS: Windows_NT x64 10.0.18363

Me too...

Version: 1.59.0
Commit: 379476f0e13988d90fab105c5c19e7abc8b1dea8
Date: 2021-08-04T23:13:20.182Z
Electron: 13.1.7
Chrome: 91.0.4472.124
Node.js: 14.16.0
V8: 9.1.269.36-electron.0
OS: Linux x64 5.13.8-arch1-1
Activating extension 'formulahendry.vscode-mysql' failed: Cannot find module '/home/enzo/.vscode/extensions/formulahendry.vscode-mysql-0.4.1/node_modules/mysql/lib/Connection'.
$onExtensionActivationError @ mainThreadExtensionService.ts:95

Same here

Button + no work

Alguien lo pudo solucionar?

No,
El mié., 8 sept. 2021 18:39, ArianaAraneda @.***>
escribió:

Alguien lo pudo solucionar?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/formulahendry/vscode-mysql/issues/175#issuecomment-915394843,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AN2OHJ346H3ABHPO3DAQLATUA6GT7ANCNFSM5B6DWTBQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Button + no work on windows 10 pro

me too

Same here, os is macOS big sur 11.6...
vs code version is
Version: 1.60.2 (Universal)
Commit: 7f6ab5485bbc008386c4386d08766667e155244e
Date: 2021-09-22T11:59:27.195Z
Electron: 13.1.8
Chrome: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Darwin x64 20.6.0

apt install gnome-keyring
This helped for me. But sometimes clicking on + sign do nothing, restarting vs code hellps.

apt install gnome-keyring
This helped for me. But sometimes clicking on + sign do nothing, restarting vs code hellps.

Same problem, I'll uninstall and find an alternative


JsonSerializerInternalWriter.SerializeDictionary appears to be broken

A serialized dictionary entry should consist of a key and a value.
However, SerializeDictionary outputs the type of the key, not the value of the key:
		private void SerializeDictionary (JsonWriter writer, IDictionary values, JsonDictionaryContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
			{
			IWrappedDictionary wrappedDictionary = values as IWrappedDictionary;
			object underlyingDictionary = wrappedDictionary != null ? wrappedDictionary.UnderlyingDictionary : values;

			OnSerializing (writer, contract, underlyingDictionary);
			_serializeStack.Add (underlyingDictionary);

			WriteObjectStart (writer, underlyingDictionary, contract, member, collectionContract, containerProperty);

			if (contract.ItemContract == null)
				contract.ItemContract = Serializer._contractResolver.ResolveContract (contract.DictionaryValueType ?? typeof (object));

			if (contract.KeyContract == null)
				contract.KeyContract = Serializer._contractResolver.ResolveContract (contract.DictionaryKeyType ?? typeof (object));

			int initialDepth = writer.Top;

			foreach (DictionaryEntry entry in values)
				{
				bool escape;
				string propertyName = GetPropertyName (writer, entry.Key, contract.KeyContract, out escape);

				propertyName = (contract.DictionaryKeyResolver != null)
					 ? contract.DictionaryKeyResolver (propertyName)
					 : propertyName;

				try
					{
					object value = entry.Value;
					JsonContract valueContract = contract.FinalItemContract ?? GetContractSafe (value);

					if (ShouldWriteReference (value, null, valueContract, contract, member))
						{
						writer.WritePropertyName (propertyName, escape);
						WriteReference (writer, value);
						}
					else
						{
						if (!CheckForCircularReference (writer, value, null, valueContract, contract, member))
							continue;

						writer.WritePropertyName (propertyName, escape);

						SerializeValue (writer, value, valueContract, null, contract, member);
						}
					}
				catch (Exception ex)
					{
					if (IsErrorHandled (underlyingDictionary, contract, propertyName, null, writer.ContainerPath, ex))
						HandleError (writer, initialDepth);
					else
						throw;
					}
				}

			writer.WriteEndObject ();

			_serializeStack.RemoveAt (_serializeStack.Count - 1);

			OnSerialized (writer, contract, underlyingDictionary);
			}

It's working correctly. The documentation covers this.

Then why does it fail your own test:
        [Test]
        public void SerializeDictionaryWithStructKey()
        {
            string json = JsonConvert.SerializeObject(
                new Dictionary<Size, Size> { { new Size(1, 2), new Size(3, 4) } }
                );

            Assert.AreEqual(@"{""1, 2"":""3, 4""}", json);

            Dictionary<Size, Size> d = JsonConvert.DeserializeObject<Dictionary<Size, Size>>(json);

            Assert.AreEqual(new Size(1, 2), d.Keys.First());
            Assert.AreEqual(new Size(3, 4), d.Values.First());
        }

  Expected string length 15 but was 62. Strings differ at index 2.
  Expected: "{"1, 2":"3, 4"}"
  But was:  "{"System.Drawing.Size":{"Width":3,"Height":4,"IsEmpty":false}}"
  -------------^

Tests are green for me: https://ci.appveyor.com/project/JamesNewtonKing/newtonsoft-json/build/1.0.602

Very strange, because if you read the code, then the output shown above is exactly what would be expected.
Nowhere in that code is the value of the key output as part of the serialization process.

Also, SerializeValue for the value of the DictionaryEntry always outputs the property names as above, so how can the test ever pass, since it is expecting a serialized output with only values, and no property names.


Introduce framework to allow emitting promethues metrics from the plain provisioner

Controller-runtime already has a metrics server built in, so I'd push back a little on the need for an entire framework (since I think we already have most of one).
Also, I'm going to make yet another plug for kubebuilder/operator-sdk, which has a bunch of niceties around metrics (e.g. scaffolding that includes kube-rbac-proxy to protect the metrics endpoint and a ServiceMonitor object that can be used to automatically configure a prometheus-operator controlled prometheus instance to scrape our metrics)

@joelanford  if (read when) we want these metrics to be accessible outside the cluster via a Service, so that we can collect metrics from all the clusters running Bundles, I was under the impression that it's actually a security risk to expose the default port that the controller uses (which is where the controller-runtime metrics are published i.e /metrics). Which, I'm guessing, is probably why kubebuilder/sdk projects include kube-rbac-proxy.

Also, I'm going to make yet another plug for kubebuilder/operator-sdk

I feel like I'm missing some context here, but why wasn't any of kubebuilder/operator-sdk used again? Use cases like "automatically configure a prometheus-operator controlled prometheus instance to scrape our metrics" is going to come up when we go live in prod, and if we were to learn from OLM, we kept manually trying to keep up with these things, and therefore kept missing edge cases/introducing bugs etc.
Metrics is just one component, I'm guessing there will most definitely be other components that'll bring up the same discussion.
cc: @timflannagan @tylerslaton @exdx, in case you guys have the context I'm missing.

From the Kubebuilder metrics guide:

By default, controller-runtime builds a global prometheus registry and publishes a collection of performance metrics for each controller. These metrics are protected by kube-rbac-proxy by default if using kubebuilder. Kubebuilder v2.2.0+ scaffold a clusterrole which can be found at config/rbac/auth_proxy_client_clusterrole.yaml.

From the kube-rbac-proxy project:

Motivation
I developed this proxy in order to be able to protect Prometheus metrics endpoints. In a scenario, where an attacker might obtain full control over a Pod, that attacker would have the ability to discover a lot of information about the workload as well as the current load of the respective workload. This information could originate for example from the node-exporter and kube-state-metrics. Both of those metric sources can commonly be found in Prometheus monitoring stacks on Kubernetes.
This project was created to specifically solve the above problem, however, I felt there is a larger need for such a proxy in general.

Brief note from the upstream OLM working group: let's work towards a model where we invest in metrics sooner-rather-than-later. It sounded like we agreed that c-r metrics should be sufficient in the short term, assuming we back those metrics behind some RBAC/authz/etc. (e.g. kube-rbac-proxy). We can invest further in determine which metrics are needed a couple of metrics down the line once we iron out the API surface for the core rukpak APIs, and the behavior and functionality present in the plain provisioner implementation.

Now that some of the more focused on features have gotten into the RukPak project we should consider coming back to this. @anik120 was kind enough to open this issue as well as create a PR for it which is linked above. To get this work over the finish line we should re-evaulate that closed PR (which was closed mainly for age), rebase it, and think of what is needed to get it merged.

Brief note from the upstream OLM working group: let's work towards a model where we invest in metrics sooner-rather-than-later.

This should help us to focus on this point moving forward with the Deppy project as well since we can leverage the work done here there.


Fix: 1182 prevent used resources from deletion

Resolves https://github.com/Agenta-AI/agenta/issues/1182
Prevent deletion of variant, testsets and evaluators if used in an evaluation
Fixed BE to not crash if a linked resource is deleted

@MohammedMaaz Sorry I didn't find time to test this one. Can we fix conflicts so I have a look again!


Check final resolution of forecast data

Contemporary data are at 2.5 minutes, but forecast data source are coming in at 30 seconds. Does the src/data/prep-forecast-data.R convert to this lower resolution when doing the re-projection?

I just looked at code and some of the climate layers (contemporary and future). I think what you have above is correct -- everything now is at 2.5 min resolution (= orig resolution for contemporary data). The prep-forecast-data.R script does convert to lower resolution (lines 103-128), and the tifs saved in the ensemble ssp folders have 2.5 min resolution.  Didn't close comment in case you needed more info.


Any bug on RACE?

Hi Qingqing,
Thanks for releasing great code.
I have successfully trained sbert for SQuAD.  But when I train original bert on RACE, the loss can not convergence even the number of training dataset is 2 (=1 is ok).  Is there any bug about the RACE dataset?
If it is convenient, could you please provide me serveral sbert for RACE, such as sbert-s7/sbert-s9/sbert-s10?
Thanks, Looking forward to your reply!
Best,
Deming

After I shuffle the train.jsonl mannualy, it works.


Run tests in CI

In case it's useful! Happy to adapt this PR according to any feedback, desired behavior, etc.

Can you set the workflow to run on pull requests as well? Once that's done, I'll merge it!

@ikollipara updated!


fix typo in macrobenchmarks/lib/main

Description
Fix a typo.
Related Issues
N/A
Tests
N/A: not needed because no program logic change
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[X] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[X] I signed the CLA.
[X] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[X] I read the Tree Hygiene wiki page, which explains my responsibilities.
[X] I updated/added relevant documentation (doc comments with ///).
[X] All existing and new tests are passing.
[X] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[X] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Did any tests fail when you ran them? Please read Handling breaking changes.

[X] No, no existing tests failed, so this is not a breaking change.
[ ] Yes, this is a breaking change. If not, delete the remainder of this section.

(test exempt: unused variable naming)


File structure suggestions

One suggestion would be to store the templates in different folders based on the language, so the folder for cpp templates would be /public/templates/cpp/. You can also fetch the language a user is currently using when they click on the extension and only show these templates by default. A drop down to select other languages will also help.

Separate folders for languages sound like a good idea, will do.
Also, I'm planning to add platform-specific features like the one you suggested in future releases so let me know if you have other related suggestions.

when I am trying to install this from the instruction, brower is giving this error for brave
Manifest file is missing or unreadable
Could not load manifest.
what should I do now

#2 Made seperate folders for different languages

when I am trying to install this from the instruction, brower is giving this error for brave Manifest file is missing or unreadable Could not load manifest. what should I do now

@rahul07sharmars please make sure you downloaded & unzipped the correct file (neet-v*.zip)  from the release page.
if you still face any issues please consider opening a new GitHub issue.


nan loss values during training

I checked the other issues regarding this topic but I couldn't solve my problem. I am unable to train, all loss indicators during training are nan.
I'm training on a custom dataset with 4 classes with slightly modified train.py. Weights are converted according to the tutorial, I also tried to run the tutorial on VOC2012 and it worked fine.

Modified lines in train.py (I hope i didn't miss any):
# line 58
model = YoloV3(FLAGS.size, training=True, classes=4)

# line 95
model_pretrained = YoloV3(FLAGS.size, training=True, classes=80)

# line 123
loss = [YoloLoss(anchors[mask], classes=4) for mask in anchor_masks]

With these changes num_classes and weights_num_classes arguments should lose their meaning.
marks.names file:
BottomLeft
BottomRight
TopLeft
TopRight

Example of tfrecord:
features {
  feature {
    key: "image/encoded"
    value {
      bytes_list {
        value: "\211PNG\r\n\032\n\ <more_values>"
      }
    }
  }
  feature {
    key: "image/filename"
    value {
      bytes_list {
        value: "BL_0_TR_62_BR_60_TL_2_.png"
      }
    }
  }
  feature {
    key: "image/format"
    value {
      bytes_list {
        value: "png"
      }
    }
  }
  feature {
    key: "image/height"
    value {
      int64_list {
        value: 416
      }
    }
  }
  feature {
    key: "image/key/sha256"
    value {
      bytes_list {
        value: "dc834637dd97f6973f19a2855d05ac189211ab056c321829e20bc9704498997f"
      }
    }
  }
  feature {
    key: "image/object/bbox/xmax"
    value {
      float_list {
        value: 0.3461538553237915
        value: 0.8293269276618958
      }
    }
  }
  feature {
    key: "image/object/bbox/xmin"
    value {
      float_list {
        value: 0.19951923191547394
        value: 0.5504807829856873
      }
    }
  }
  feature {
    key: "image/object/bbox/ymax"
    value {
      float_list {
        value: 0.4110576808452606
        value: 0.41826921701431274
      }
    }
  }
  feature {
    key: "image/object/bbox/ymin"
    value {
      float_list {
        value: 0.26442307233810425
        value: 0.19230769574642181
      }
    }
  }
  feature {
    key: "image/object/class/label"
    value {
      int64_list {
        value: 0
        value: 3
      }
    }
  }
  feature {
    key: "image/object/class/text"
    value {
      bytes_list {
        value: "BottomLeft"
        value: "TopRIght"
      }
    }
  }
  feature {
    key: "image/source_id"
    value {
      bytes_list {
        value: "BL_0_TR_62_BR_60_TL_2_.png"
      }
    }
  }
  feature {
    key: "image/width"
    value {
      int64_list {
        value: 416
      }
    }
  }
}
features {
<another_image_annotation>
}
<more_images_annotations>

Callbacks during training:

After first epoch

[[{{node IteratorGetNext}}]]
2020-04-15 21:51:54.725860: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Out of range: End of sequence
	 [[{{node IteratorGetNext}}]]
	 [[IteratorGetNext/_8]]
2020-04-15 21:52:00.118671: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Out of range: End of sequence
	 [[{{node IteratorGetNext}}]]
2020-04-15 21:52:00.119143: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Out of range: End of sequence
	 [[{{node IteratorGetNext}}]]
	 [[IteratorGetNext/_2]]
C:\Users\dfice\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow_core\python\keras\callbacks.py:1916: RuntimeWarning: invalid value encountered in less
  self.monitor_op = lambda a, b: np.less(a, b - self.min_delta)

Epoch 00001: saving model to checkpoints/yolov3_train_1.tf
C:\Users\dfice\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow_core\python\keras\callbacks.py:1261: RuntimeWarning: invalid value encountered in less
  if self.monitor_op(current - self.min_delta, self.best):

After second epoch

2020-04-15 21:52:18.622583: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Out of range: End of sequence
	 [[{{node IteratorGetNext}}]]
2020-04-15 21:52:18.622966: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Out of range: End of sequence
	 [[{{node IteratorGetNext}}]]
	 [[IteratorGetNext/_2]]

Epoch 00002: saving model to checkpoints/yolov3_train_2.tf

23/23 [==============================] - 18s 779ms/step - loss: nan - yolo_output_0_loss: nan - yolo_output_1_loss: nan - yolo_output_2_loss: nan - val_loss: nan - val_yolo_output_0_loss: nan - val_yolo_output_1_loss: nan - val_yolo_output_2_loss: nan
WARNING:tensorflow:Unresolved object in checkpoint: (root).layer-8
W0415 21:52:22.357214  9088 util.py:144] Unresolved object in checkpoint: (root).layer-8
WARNING:tensorflow:Unresolved object in checkpoint: (root).layer-9
W0415 21:52:22.357214  9088 util.py:144] Unresolved object in checkpoint: (root).layer-9
WARNING:tensorflow:Unresolved object in checkpoint: (root).layer-10
W0415 21:52:22.357214  9088 util.py:144] Unresolved object in checkpoint: (root).layer-10
WARNING:tensorflow:Unresolved object in checkpoint: (root).layer-11
W0415 21:52:22.357214  9088 util.py:144] Unresolved object in checkpoint: (root).layer-11
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
W0415 21:52:22.357214  9088 util.py:152] A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.

Ok looks like I have a typo in my tfrecord file where "TopRIght" is supposed to be "TopRight". This solved my issue.


Scene Building vs. Runtime Instantiations

Currently key game tokens (e.g. the Ship) are instantiated at runtime (traditional method) instead of already existing in the scene. This means that there is no instance reference of them to easily drag & drop into the Inspector for other tokens that require them. Rather they are passed around as parameters, or registered, or "found",  as is normally done in non-Unity games.
Since this is a Unity game, perhaps it would be best to take advantage of its features, like editor scene building & reference dragging, rather than do things in the "traditional" non-Unity way!?
It would make some things easier, but it may also encourage bad programming habits.

So I guess one has to consider what the price for a certain solution really is. There are obviously many considerations to keep in mind. Each solution most likely place a constraint on the system. This constraint that developers typically aim to avoid are those that tend to create a "ball of mud", but there are others like harm toward reusability, that may be (for game dev) a secondary target to avoid.
Before you unleash the most complex solution, I guess it wouldn't do any harm thinking along the lines of KISS/YAGNI. If you can get by with a static registration/accessor, I guess you should just go for it. If it cause issues, figure out why and find a solution that solves that particular issue. If the game runs, what harm is there left to fix? How bad is it, really? In reality, when can you foresee your existing solution becoming a proper problem and how hard do you think it'll be to solve it later on?
// KISS variant
static class GameTokens
{
    // Impl. omitted, but easy to figure out...
    // Ofc, OnShipAvailable should callback immediately
    // if ship already has been flagged as available.
    public static void OnShipAvailable(Action<Ship> ship);
    public static void ShipAvailable(Ship ship);
}

And some usage...
public class Ship : MonoBehaviour {
    void Awake() {
        GameTokens.ShipAvailable(this);
    }
}

And some more...
public class ShipDependent : MonoBehaviour {
    Ship ship;

    void Awake() {
        DisableUntilShipIsAvailable();
    }

    void DisableUntilShipIsAvailable() {
        enabled = false;
        GameTokens.OnShipAvailable
        (s =>
            ship = s;
            enabled = true
        );
    }
}


Support python -m bifrost.version command

Just enables a simple command to print out version and copyright info:
% python -m bifrost.version
bifrost 0.9.0.dev+g2bc27cc
Copyright (c) 2016-2020, The Bifrost Authors. All rights reserved.
Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
License: BSD 3-Clause

I think it would be nice to offer many of the 'tool' scripts with this interface, e.g. python -m bifrost.telemetry --enable rather than (or I guess in addition to) a script file in bin/.

Codecov Report

Merging #163 (ad74b5e) into autoconf (54ed45b) will decrease coverage by 0.02%.
The diff coverage is 0.00%.

:exclamation: Current head ad74b5e differs from pull request most recent head 26207f2. Consider uploading reports for the commit 26207f2 to get more accurate results

@@             Coverage Diff              @@
##           autoconf     #163      +/-   ##
============================================
- Coverage     59.01%   58.99%   -0.03%
============================================
  Files            65       66       +1
  Lines          5673     5675       +2
============================================
  Hits           3348     3348
- Misses         2325     2327       +2

Impacted Files
Coverage Δ

python/bifrost/version/__main__.py
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 54ed45b...26207f2. Read the comment docs.

It looks like there is also a reference to version.py in bifrost/__init__.py.

I took a look through tools and telemetry is the only one that I see that would be a good candidate for the python -m ... treatment.

When you did the python -m treatment for LSL, you didn't encounter this message:
RuntimeWarning: 'BLAH.__main__' found in sys.modules after import of package 'BLAH',
but prior to execution of 'BLAH.__main__'; this may result in unpredictable behaviour

It looks like you still have just one module lsl.misc.telemetry that includes an if __main__ block. When I tried it that way for Bifrost telemetry (or version), I got the above warning. Getting around it seemed to mean using a separate __init__.py for the module definitions and then __main__.py for the program. And I think it's precisely because telemetry is imported everywhere (and version is imported by the bifrost main module). In general, if the module is not already imported before you try to run it, having an if __main__ block will be fine.
Or having the script live at bifrost.tools.telemetry but the module imported everywhere is bifrost.telemetry, which is less satisfying.

I didn't get that warning but I also have extensively tested my changes.

Ok, well, that's fixed in LSL.
Do you want to go ahead and expand this to cover bifrost.telemetry?  I could tackle that one if it would help.

@jaycedowell: I believe I have a "git stash" where -m bifrost.telemetry more-or-less works, so I could probably push that to the autoconf branch. However, for the purpose of building documentation (and possibly test suites etc.) I have a minor revision of telemetry.py on commit 6ad2617c1081651a6d832b35897a6e009b00217d that would be nice to include – or just address the conflict when I'm ready to merge #162... either way I guess.
If you think 6ad2617 is fine, I could cherry-pick that onto autoconf, fix its conflict with my stashed telemetry-main, and commit.

Update: I see you did something similar to telemetry in lwa-project/lsl@0d12197cfe7ef2589bbc05ce9f95c7f95594890a.

Yeah, I think that is fine.  The alternative to disabling the telemetry would be to generate a new _INSTALL_KEY each time and report with that.  I don't think that is necessary since this readonly condition seems like an edge case.

Thinking about your comment in #164, what about doing something like having python -m bifrost.version give you what it does now and then adding something like a python -m bifrost.version --config that gives you that, plus how the library is configured.  That might be helpful for people who are on a system where Bifrost is already installed and they might be curious how it is configured.

Yes! That looks very helpful. Small friendly amendment forthcoming… haha.

Does this need anything else or is it ready to merge?

Ready to go.


Small rewrite of #450

This is a rewrite designed to accomplish the following:

The interval function should not do anything besides update attributes if a transition is in progress.
Slightly rework detect_non_ha_changes to suit goal 1
Move the functions around to match, by order of top to bottom, the correct sequence of calls from _update_attr_and_maybe_adapt_lights all the way to the _adapt_light()

Based on #450
If the changes aren't liked, no worries! That's what #450 is for. #450 merges nicely with the original code.

@basnijholt I'm in love with your pytest: it's so thorough!

That is great! Let's write more of them with new features 😄
Have you been able to run the tests locally on your computer?

@basnijholt Unfortunately not, I have docker setup but the entire process was a bit beyond me. Tried using a project to run github actions locally, called nektos\act, but it ran into issues authenticating.

If you are able to run Docker images, I can probably make an image that one can use to run the tests with.

That would be fantastic!

@basnijholt what's the significance of None = None checks you've got going on here?
    async def _adapt_lights(
        self,
        lights: list[str],
        transition: int | None = None,
        force: bool = False,
        context: Context | None = None,
        adapt_brightness: bool | None = None,
        adapt_color: bool | None = None,
        prefer_rgb_color: bool | None = None,
    ) -> None:

e.g., in context: Context | None = None, the Context | None part means that the type can be Context or None and then we say that the default value is None.

Oh I see!

That would be fantastic!

See https://github.com/basnijholt/adaptive-lighting/tree/master/tests#developer-notes-for-the-tests-directory :)

BTW, I do not use Windows, the updated command might be
docker run -v %cd%:/app basnijholt/adaptive-lighting:latest

to run on Windows. Let me know if that works! (Or even better, update the tests/README.md :smile:)

@basnijholt
C:\Users\*****\Documents\GitHub\adaptive-lighting>docker run -v $(pwd):/app basnijholt/adaptive-lighting:latest
Unable to find image 'basnijholt/adaptive-lighting:latest' locally
docker: Error response from daemon: manifest for basnijholt/adaptive-lighting:latest not found: manifest unknown: manifest unknown.
See 'docker run --help'.

C:\Users\*****\Documents\GitHub\adaptive-lighting>docker run -v %cd%:/app basnijholt/adaptive-lighting:latest
Unable to find image 'basnijholt/adaptive-lighting:latest' locally
docker: Error response from daemon: manifest for basnijholt/adaptive-lighting:latest not found: manifest unknown: manifest unknown.
See 'docker run --help'.

C:\Users\*****\Documents\GitHub\adaptive-lighting>

Oh, that is because the image hasn't been uploaded yet. It should be done once https://github.com/basnijholt/adaptive-lighting/actions/runs/4527678249/jobs/7973752635 is done.
If you don't want to wait for that, you can try running:
docker build -t basnijholt/adaptive-lighting:latest --no-cache .

Unfortunately that doesn't work either :/
C:\Users\*****\Documents\GitHub\adaptive-lighting>docker build -t basnijholt/adaptive-lighting:latest --no-cache .
[+] Building 0.5s (2/2) FINISHED
 => [internal] load build definition from Dockerfile                                                               0.8s
 => => transferring dockerfile: 2B                                                                                0.1s
 => [internal] load .dockerignore                                                                                0.8s
 => => transferring context: 2B                                                                                0.1s
failed to solve with frontend dockerfile.v0: failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount3359422249/Dockerfile: no such file or directory

C:\Users\*****\Documents\GitHub\adaptive-lighting>

That is because you are probably on a branch where the Dockerfile doesn't exist. I just merged in the master branch to this branch. Pull and try again :)

Please don't commit here until the draft finishes I'm trying to locate a bug.

        """Check if the light has been 'on' and is now manually controlled."""
>       manual_control = self.manual_control.setdefault(light, False)
E       AttributeError: 'bool' object has no attribute 'setdefault'

What on earth?

So somehow self.manual_control became a boolean...

@basnijholt would you perhaps know what's happening here?
in init for AdaptiveSwitch
        self._manual_lights = {}
        all_lights = _expand_light_groups(self.hass, self._lights)
        for light in all_lights:
            self._manual_lights[light] = {"timer": 0}

the error I'm getting:
        self.turn_on_off_listener.last_service_data[light] = service_data
>       self._manual_lights[light]["timer"] = 0
E       KeyError: 'light.kitchen_lights'

homeassistant/components/adaptive_lighting/switch.py:1239: KeyError

Have you been able to run the tests with Docker now BTW?
The image finally builds successfully and showed up on https://hub.docker.com/r/basnijholt/adaptive-lighting/tags

@basnijholt I just tried it and it's finally grabbing it using the commands listed on your documentation.

Do you perhaps know where I can see my _LOGGER.debug messages using your docker image?

I believe that it should only show up when a test is failed.
So I take that it is working?
Did you have to use docker run -v %cd%:/app basnijholt/adaptive-lighting:latest or docker run -v $(pwd):/app basnijholt/adaptive-lighting:latest?

@basnijholt Both worked on my system.
The problem I'm having with your tests is my debug messages are printing - but I can't find the correlation between when they print and when the assert checks fail.

@basnijholt Hey thanks for getting me hooked up with that Docker image, this is going so much faster now :). The community appreciates you man!

About the many log messages.
You should look at the section of the failing test and then at the Captured log call section.
There will be all the log messages. Many messages from Home Assistant core will be in there too, which might or might not be helpful. I haven't figured out how to suppress those messages yet.
There are A LOT of messages in Captured stderr call which (I think) can be ignored.
One can pass extra arguments to pytest like:
Requires new image that has not been uploaded by CI yet (probably in 1 hour)):
docker run -v $(pwd):/app basnijholt/adaptive-lighting:latest --log-format="%(asctime)s %(levelname)s %(message)s" --log-date-format="%H:%M:%S" tests/components/adaptive_lighting/test_switch.py

The following command will supress the stderr section:
docker run -v $(pwd):/app basnijholt/adaptive-lighting:latest --show-capture=log tests/components/adaptive_lighting/

I think this will be the clearest:
docker run -v $(pwd):/app basnijholt/adaptive-lighting:latest --show-capture=log --log-format="%(asctime)s %(levelname)-8s %(name)s:%(filename)s:%(lineno)s %(message)s" --log-date-format="%H:%M:%S" tests/components/adaptive_lighting/

I think that if you do docker pull basnijholt/adaptive-lighting:latest now, you will get the latest updated version of the Docker image, allowing you to run the command above.

I have just added that as an example here https://github.com/basnijholt/adaptive-lighting/pull/483.

 Thank you too for all the energy and work you are putting into this! You definitely have made the most contributions of anyone so far 😄

For some reason when the color switch is off the integration is detecting it as on. I've no idea why, but it's the last error and I couldn't fix it tonight. Will take a look fresh tomorrow.

Andddddd it's back to the ATTR_SUPPORTED_FEATURES error.

Compiled list of errors that are suppressed in this version:
On startup:
ATTR_SUPPORTED_FEATURES not in 'state' in _supported_features
state is None in _supported_features
Elsewhere or not confirmed where:
switch._manual_lights[light]["timer"] has KeyError: light not in switch._manual_lights in the maybe_reset_manual_control func.
Suppressing all these errors still doesn't pass the tests. What's more interesting is all the tests run fine on my HASS, so I'm guessing they all have to do with startup. Maybe I'm running something asynchronously when I shouldn't be.
this roadblock is unfortunately too large, with the lack of helpful errors I've no idea what to do. Most of the errors are related to timeout, yet don't specify a function.
So this is just where the project ends unless someone decides to pick it up.


[analyzer] Add CTU On-demand analysis support

On-demand parsing of the ASTs can be now enabled if the relevant Phabricator patch lands. If enabled, on-the-fly analysis removes the need to create AST dumps, but in turn the compilation database must be provided to Clang. The CTU Index also changes if the option is enabled. The mapping form function name (USR) -> AST dump file becomes function name (USR) -> path to source file.

Do note, that this modification depends on the clang801 branch getting the pull request for supporting on-demand-parsing. This is currently not merged, so the ci may fail.

Renamed the feature to match the naming used in Clang.

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  28

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  28

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  1
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  25

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu/test_ctu.py  1
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  13

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu/test_ctu.py  1
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  13

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  13
- analyzer/tests/functional/ctu/test_ctu.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  13
- analyzer/tests/functional/ctu/test_ctu.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  13
- analyzer/tests/functional/ctu/test_ctu.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- analyzer/codechecker_analyzer/analyzers/clangsa/analyzer.py  1

Clones added
============
- analyzer/tests/functional/ctu/test_ctu.py  1
- analyzer/tests/functional/ctu_failure/test_ctu_failure.py  13

See the complete overview on Codacy


Flaky tests are failing on main

Flaky test CI is failing on main: https://github.com/meilisearch/meilisearch/actions/runs/3620651224
But does work on

release-v0.30.1 -> https://github.com/meilisearch/meilisearch/actions/runs/3622525160
release-v0.30.0 -> https://github.com/meilisearch/meilisearch/actions/runs/3622523742

Closed in favor of the new issue https://github.com/meilisearch/meilisearch/issues/3273


Add Self-Hosting section

Summary
I added a Self-Hosting section to the docs and included an example configuration for Nginx.
Reason
Even though there are great hosting solutions, some people, like me, like to host their Quartz website on their own server. The new sections could be a starting point for documenting self-hosting solutions with different web servers.

hey, could you run npm run format?


Package request: nim-2-0

Project description
We currently package Nim 2.2.0 since this PR. However, there are some glaring problems with Nim 2.2.x for now:

A lot of packages are outright broken due to changes to the type system (eg. sigui)
A lot of packages that do end up compiling have issues, like nimlsp completely maxing out the CPU.

I think we should package Nim 2.0 as a separate package for some time as the Nim team irons out these problems, similar to how there are multiple versions of NodeJS in nixpkgs currently.
Metadata

homepage URL: https://nim-lang.org
source URL: https://github.com/nim-lang/Nim
license: BSD
platforms: Linux and Darwin

@ehmry @Eveeifyeve
Add a :+1: reaction to issues you find important.

Project description
We currently package Nim 2.2.0 since this PR. However, there are some glaring problems with Nim 2.2.x for now:
* A lot of packages are outright broken due to changes to the type system (eg. [sigui](https://github.com/levovix0/sigui))

* A lot of packages that do end up compiling have issues, like [nimlsp](https://github.com/PMunch/nimlsp/issues/180) completely maxing out the CPU.

I think we should package Nim 2.0 as a separate package for some time as the Nim team irons out these problems, similar to how there are multiple versions of NodeJS in nixpkgs currently.
Metadata
* homepage URL: https://nim-lang.org

* source URL: https://github.com/nim-lang/Nim

* license: BSD

* platforms: Linux and Darwin

@ehmry @Eveeifyeve
Add a 👍 reaction to issues you find important.

Yeah I agree with this and I will look into making a pr that will fix this by separating nim 1 & nim 2.

But I think I might go the generic approach simular to how pnpm is doing this where there is two packages nim & nim_1

But also can you provide like a btop or something that shows that nim 2.0.0 is the problem.

But most likely I would wait for a response from https://github.com/PMunch/nimlsp/issues/180 to see if it's a nim problem and not a nix problem. I would highly suggest reporting the high cpu usage here that way nim can work on a fix for it.

I mean first of all the high cpu usage is not nix problem it's either nim_lsp or nim itself problem and two packages broken? Can you please send a hydra log of a package that is broken so I can deep dive into the issue

Nim-2.0.4 is still the default Nim-2 on the release-24.05 branch so we should sort this out before 24.11 (#344920). I would prefer just to fix everything that we have packaged so that it runs with 2.2.0.

And second of all this might be a blocker, but I can't confirm until you provide hydra logs.

And I would if I was you make an issue to sigui to tell them to update their types as there has been changes to types that they should be aware of.

And I would if I was you make an issue to sigui to tell them to update their types as there has been changes to types that they should be aware of.

https://github.com/nim-lang/Nim/blob/version-2-2/changelogs/changelog_2_2_0.md#changes-affecting-backward-compatibility

And I would if I was you make an issue to sigui to tell them to update their types as there has been changes to types that they should be aware of.

I've already notified the maintainer of it.

But also can you provide like a btop or something that shows that nim 2.2.* is the problem.

The thing is, this CPU usage problem became much more frequent after 2.2.x, so it might be the result of an undocumented change in an internal Nim component (my educated guess would be the GC).
It's best that we push out a Nim 2.0 package before 24.11 enters feature freeze.

And I would if I was you make an issue to sigui to tell them to update their types as there has been changes to types that they should be aware of.

I reported the issue with Sigui at Shady, because that is where the type that breaks for me originates: https://github.com/treeform/shady/issues/21

But also can you provide like a btop or something that shows that nim 2.2.* is the problem.

The thing is, this CPU usage problem became much more frequent after 2.2.x, so it might be the result of an undocumented change in an internal Nim component (my educated guess would be the GC).
It's best that we push out a Nim 2.0 package before 24.11 enters feature freeze.

I mean yes but a simple fix can be making an issue downing on the problem to ninlang or ninjas because it's not a nix problem but how nimlang or nim works.

I mean I will start working on package for nim 2_0 but I am going to add a throw to say it will no longer supported until the issue in here are fixed just as a temporary fix.


Video stops and doesn't auto play continue

I check some FLV links in my server, sometimes the video stops but it doesn't resume automatically, even though, I saw the buffer loaded. I have to seek video forward and it plays fine. I don't see any error log when video stop

I get similar issues. I do get errors if I monitor the various video element events (abort, stalled, etc.).
Unfortunately, this project is pretty much abandoned it seems.
Did you figure out some solution?


Tree shaking behaving differently for require

Tree shaking  behaves differently for the last usecase.
input: if ('a' === false) { console.log('a'); } else { console.log('b'); }
output:  console.log('b');
input: if(false === false) { console.log('a'); } else { console.log('b'); }
output:  console.log('a');
input: if(false === false) { require('./a.js'); } else { require('./b.js'); }
output: require('./a.js'); //contents of a
input:  if ('a' === false) { require('./a.js'); } else { require('./b.js'); }
output:  if ('a' === false) { require('./a.js'); } else { require('./b.js'); } // contents of a and b

@TheLarkInn

It looks like you just deleted/ignored our lovely crafted issue template. It was there for good reasons. Please help us solving your issue by answering the questions asked in this template. I'm closing this. Please open a new issue with filled issue template. Also make sure your issue is not a question. Questions should be posted on Stack Overflow.

@sokra apologies I unintentionally posted the issue in form of question and ignored the template. I have updated the issue using the issue template. Please help me understand the cause of the issue.


Praise

Well done for this library. I would like to use it. Do you have a plan to maintain operator list?

Yes, we will update this


Bug: Model cannot insertBatch when $useAutoIncrement is false

PHP Version
8.2
CodeIgniter4 Version
v4.3.3
CodeIgniter4 Installation Method
Composer (using codeigniter4/appstarter)
Which operating systems have you tested for this bug?
Windows
Which server did you use?
cli-server (PHP built-in webserver)
Database
No response
What happened?
Model cannot insert using insertBatch when $useAutoIncrement is set to false.
It's exactly the same as this issue:
https://forum.codeigniter.com/showthread.php?tid=84733
But it occurs in insertBatch.
Steps to Reproduce

Create a model, set $useAutoIncrement to false, and exclude $primaryKey from $allowedFields as mentioned in the docs.

Instantiate model, $model = model(MyModel::class);

Utilize the $model->insertBatch($data); method to insert data, including the primary key.

Expected Output
The data should be inserted into the database.
Anything else?
https://forum.codeigniter.com/showthread.php?tid=84733

I've confirmed this bug.

CodeIgniter\Database\Exceptions\DataException : There is no primary key defined when trying to make insertBatch.

--- a/tests/system/Models/InsertModelTest.php
+++ b/tests/system/Models/InsertModelTest.php
@@ -65,6 +65,26 @@ final class InsertModelTest extends LiveModelTestCase
         $this->seeInDatabase('job', ['name' => 'Cab Driver']);
     }

+    public function testInsertBatchUseAutoIncrementSetToFalse(): void
+    {
+        $insertData = [
+            [
+                'key'   => 'key1',
+                'value' => 'value1',
+            ],
+            [
+                'key'   => 'key2',
+                'value' => 'value2',
+            ],
+        ];
+
+        $this->createModel(WithoutAutoIncrementModel::class);
+        $this->model->insertBatch($insertData);
+
+        $this->seeInDatabase('without_auto_increment', ['key' => 'key1']);
+        $this->seeInDatabase('without_auto_increment', ['key' => 'key2']);
+    }
+
     public function testInsertBatchValidationFail(): void
     {
         $jobData = [

@MaulanaMalikS I sent #7759. Try it if you can.

@kenjis, yes, I've tried it and it works!


PCI Passthrough/Discrete Device Assignment for WSL2.

Is your feature request related to a problem? Please describe.
Related to #1788 but different. I'd like to pass through my AMD GPU in its entirety to the guest VM for ROCm workloads. The solution described only works on CUDA but not anyplace else.
Describe the solution you'd like
I'd like DDA/PCI passthrough for WSL2, this will allow any device to be passed through. Even if it doesn't stay connected to the host, this is exactly the behaviour I want. Since WSL2 is based on Hyper-V, and HyperV has passthrough support, and it's even present in Windows 10 not just Windows server according to some reports, this should be doable as a feature.
Describe alternatives you've considered
Passing through just ROCm won't work as there's no ROCm for Windows 10. In addition, I'd like nested virtualization (Docker/KVM) on the guest.

nested virtualization would be hard as WSL2 distros don't exactly use multiple VMs per distro, they use one common VM for everything that are namespaced by a custom init (see #994), this would also make PCI passthrough hard since you'll be sharing this for your entire VM, not just one distro, if you're interested how GPU support works, Canonical gave an overview.

Nested virtualization is available on Dev Channel builds of Windows 10. You have to set nestedVirtualization=true in .wslconfig, terminate any running WSL instances with wsl.exe --terminate, restart LxssManager, and then re-open Ubuntu. You can then check with lscpu or kvm-ok.
I have a guide to setting up a WSL environment for maximum KVM performance here.
Docker isn't virtualization, it's a container, and it's already working on WSL 2.

Thanks, but my main use is still with DDA.

Thanks, but my main use is still with DDA.

I don't get why you need DDA though @hameerabbasi, we already have a virtualized GPU via /dev/dxg. AFAIK ROCm support is in the works (AMD already has support via DirectML).

I want to run KVM as a hypervisor on WSL, and pass through the discrete GPU completely.
DieectML doesn’t help, as stuff compiled for ROCm wouldn’t work with it, neither would it aid those wishing to work on the ROCm stack.
ROCm support for Windows (if that’s what we’re talking about) was never officially announced. AFAICT it was just one engineer who said it was coming in an unofficial capacity.
If ROCm support for WSL is what we’re talking about, would you mind pointing me to a reference?

If you were in Build 2020, this was announced, if you weren't the highlights about it is available.

I'm looking to support a custom FPGA accelerator--exposed to the host as a PCIe device--within WSL2. It seems like DDA is the way to do that, but as far as I can tell it's 1) not supported in Windows 10, and 2) not supported for lightweight VMs like those used by WSL2, only full Hyper-V VMs. Is this correct? If so, it would be nice to address these issues.

I'm looking to support a custom FPGA accelerator--exposed to the host as a PCIe device--within WSL2. It seems like DDA is the way to do that, but as far as I can tell it's 1) not supported in Windows 10, and 2) not supported for lightweight VMs like those used by WSL2, only full Hyper-V VMs. Is this correct? If so, it would be nice to address these issues.

I have exactly the same issue.  Looking at other virtualization and even dual-booting but would way rather run in WSL2 if possible.

I'm looking to support a custom FPGA accelerator--exposed to the host as a PCIe device--within WSL2. It seems like DDA is the way to do that, but as far as I can tell it's 1) not supported in Windows 10, and 2) not supported for lightweight VMs like those used by WSL2, only full Hyper-V VMs. Is this correct? If so, it would be nice to address these issues.

I would also like to passthrough a custom FPGA.  Did you figure out a solution?

I'm looking to support a custom FPGA accelerator--exposed to the host as a PCIe device--within WSL2. It seems like DDA is the way to do that, but as far as I can tell it's 1) not supported in Windows 10, and 2) not supported for lightweight VMs like those used by WSL2, only full Hyper-V VMs. Is this correct? If so, it would be nice to address these issues.

I would also like to passthrough a custom FPGA. Did you figure out a solution?

I would like to piggyback off this also as this is exactly what I would like to accomplish also, specifically a thunderbolt attached Xilinx Alveo PCIe card.

I also would be interested in this being enabled - my use case is a Linux-only ADC card. I was able to get the module compiled and loaded in WSL2 running Debian 10, but without the device being accessible, it's useless.

C'mon Microsoft team, make this possible! We need full PCI GPU passthrough in WSL!

I also would like to access my smartnic on my machine via WSL. This is not going to be possible without PCIe passthrough too.

Would love access to this feature to test graphics heavy applications with a secondary GPU from within a WSL VM

Just chiming in here, to say that I also would like to access a custom fpga that is exposed to the host via PCIe from WSL, since a specific version of CentOS is needed to flash it. Dual boot seems to be the only alternative elsewise...

I would really like to have this feature for using a coral device on WSL.

I would like to play around with a bluefield dpu, being able to pass through arbitrary pcie devices would be really helpful here.

i also would very much like this


How Image Serve is handled from the backend?

Description
I wrote an api to get profile image and i generated the client

I am fetching the image but it is not showing when i create URL.createobjecturl() it is showing undefined or empty

Reproducible example or configuration
No response
OpenAPI specification (optional)
import { defineConfig } from "@hey-api/openapi-ts";
export default defineConfig({
client: {
name:"@hey-api/client-fetch",
bundle: true
},
input: "./openapi.json",
output: {
path: "./src/client",
format: "biome",
lint: "biome",
},
});
System information (optional)
No response

@sagardwivedi I think you'll want to modify your response type. Otherwise, you're getting JSON as specified and I am not sure what the issue is there. I'd need to see a running example to help further


Recovery E2E

NOTE: currently this does not check the expiration time of the trust bundle payload. I will work to add that while this is in PR. Shouldn't be a large change.
Added expiry check

Validated the expiration works.


Refactory upstream

Issues associated with this PR
Your PR should present related issues you want to solve.
Sign the CLA
Make sure you have signed the CLA
Solutions

Refatcory upstream

delete priorityset and some structures that not be used now
use hostset to manage subset
use types.Cluster instead of simpleInMemCluster struct, makes cluster can be extended.

UT result
Unit Test is needed if the code is changed, your unit test should cover boundary cases, corner cases, and some exceptional cases.
And you need to show the ut result.
Benchmark
If your code involves the processing of every request, you should give the Benchmark Result
Code Style

Make sure Goimports has run
Show Golint result

Codecov Report

Merging #710 into stable will decrease coverage by 0.25%.
The diff coverage is 77.84%.

@@            Coverage Diff             @@
##           stable     #710      +/-   ##
==========================================
- Coverage   51.01%   50.76%   -0.26%
==========================================
  Files         194      193       -1
  Lines       16929    16652     -277
==========================================
- Hits         8636     8453     -183
+ Misses       7436     7348      -88
+ Partials      857      851       -6

Impacted Files
Coverage Δ

pkg/filter/network/tcpproxy/proxy.go
15.23% <ø> (+0.14%)
:arrow_up:

pkg/types/upstream.go
59.37% <ø> (-10.4%)
:arrow_down:

pkg/proxy/downstream.go
34.57% <ø> (+0.09%)
:arrow_up:

pkg/upstream/cluster/stats.go
100% <ø> (ø)
:arrow_up:

pkg/istio/mixerclient/client.go
0% <0%> (ø)
:arrow_up:

pkg/server/handler.go
36.76% <0%> (ø)
:arrow_up:

pkg/istio/utils/utils.go
22.22% <0%> (ø)
:arrow_up:

pkg/upstream/healthcheck/healthchecker.go
80.8% <100%> (-0.39%)
:arrow_down:

pkg/upstream/cluster/cluster_adapter.go
12.5% <12.5%> (ø)

pkg/router/configutility.go
40.27% <50%> (ø)
:arrow_up:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb2e09b...acfb340. Read the comment docs.

Compare init hosts and update hosts performance data

(pprof) list GetOrCreateAddr
Total: 334511
ROUTINE ======================== sofastack.io/sofa-mosn/pkg/upstream/cluster.GetOrCreateAddr in Go/src/sofastack.io/sofa-mosn/pkg/upstream/cluster/host.go
         0      65537 (flat, cum) 19.59% of Total
         .          .    141:	addr, err := net.ResolveTCPAddr("tcp", addrstr)
         .          .    142:	if err != nil {
         .          .    143:		log.DefaultLogger.Errorf("[upstream] resolve addr %s failed: %v", addrstr, err)
         .          .    144:		return nil
         .          .    145:	}
         .      65537    146:	AddrStore.Store(addrstr, addr)
         .          .    147:	return addr
         .          .    148:}

LGTM


Optimize readability of .M cards

Before we release, I'd like to see if we want make some of the cards more readable. I have a few ideas of how we might go about doing so.

I forgot #245 was still open. This is a duplicate.


detect color changes

Color changes are not detected (and saved) I made a new commit in PR #100 addressing that, the event is fired many times, but I coud not find another one able to detect those changes.

Hi,
Could you also check that is my jsfiddle which is listening 'change' event doing the right thing to your issue as well, I commented this ticket https://github.com/angular-ui/ui-tinymce/pull/110 there is link to the jsfiddle which has 'change' event listerner added.
If that 'change' event is enough then 'nodeChange' listerner would not be needed.

Hi,
But if the 'change' event does the same thing then only that should be added, so that directive only listens relevant events and not all..
http://jsfiddle.net/RopoMen/swvcbfhm/
That jsfiddle contains 'change' event listner, but I just wanted to be sure that my 'change' event listener does what OP wanted :)

I do not think the library should bind to that event if it indeed does fire that often - that could be potentially problematic when you have a fast typer.
Because of potential performance & UX concerns, I believe this should be implemented at the user level.

@sevenbytes +1 for your solution
@wesleycho this solution is required to properly handle changes as text-colors, highlight colors, table cells and border propeties and font size. Without this fix some are not handled at all (font color for exemple) and others seems randomly handled (cell border properties for example)


Ruby example

I just did this for fun...
Everything is runs the same as the C version. I over-commented so it would be easy to understand.

Go look at #6 instead.


Events StreamId in multi tenant scenario

In my app I derive event stream ids from ids coming from an external source. On the external source IDs are unique within a tenant (that system is also multi-tenant), but there is no guarantee that IDs are globally unique.
Looking a the mt_events table
CONSTRAINT pk_mt_events_stream_and_version UNIQUE (stream_id, version),
currently stream_id is supposed to be globally unique. I would expect it to be unique only within a tenant, similar to how document ids are unique within a tenant. Otherwise I will have to include tenantid into  stream_id to ensure global uniqueness which I think defeats the purpose of built-in multi-tenancy.
Is this just an oversight\missing feature or am I using stream_ids incorrectly?

You're simply bumping into the fact that this is an OSS project where the primary author does not have infinite bandwidth. We know there are missing features for the event store + multi-tenancy, and we also know it'd help to have some level of sharding and probably more identity options. The event store is the place where I need to most help, so if you're volunteering....

I absolutely understand and accept that.
In terms of real usage I am new to almost everything here starting from document databases to event sourcing and to Marten itself. Sorry for asking obvious things. Just want to make sure I am not missing anything before working on potentially crazy workarounds.
With this specific issue - it seems to be pretty straightforward. Events\streams tables always include tenantid column, so it looks like it is just a matter of changing constraints to include tenantid and then updating mt_append_event function accordingly. Does it sound about right? I can try to create a PR for this.
Also so far I could not figure out how Migrations work or where to add a migration to update constraints and the function on existing databases

"With this specific issue - it seems to be pretty straightforward." -- it's not simple or straightforward at all if you look at all of the usage. Yes, changing the constraint would be easy, but you'd have to change a lot of internals in the event store to be tenant aware to effectively use composite keys. And most of the time it's not multi-tenanted.
I'm sorry, but this isn't likely to be changed any time soon and I'm going to close this just in the name of bookkeeping.


Something wrong about output, can not get the right string

Error: sequence item 0: expected str instance, list found

It is also wrong to directly pass text into clip.

I'm going to follow this post because I'm tired to even try making mine. It's called Anynode but any input or output failed. Every single workflow in the list failed, I feel more tired just thinking about it

I'm going to leave this one open because this is definitely a bug that others face.
So, Could you give me the prompts you guys are using?  I can't read it on your screenshots because they don't have enough resolution to read the small prompt text in the AnyNodes you are showing. Just paste the prompt text here please.
We will get this resolved, it seems to be simply that you are trying to feed the wrong input to the string list output... It will only either expect the type: string or list.  I don't know because I haven't used that one but here are some tips for getting the right output format for other nodes.
From what I can tell, you guys are having a mixup between what the input type actually is and what the output type is but I can't tell because I can't read your prompts. It also seems you are trying to use AnyNode as if it were directly an LLM.  If you make this misconception of AnyNode's place in your workflow, you will have problems with the output from your function being bad quality if it works at all.
Where does AnyNode fit in Text Processing?
Try to think of AnyNode as a "python coder that makes you a function". It is not directly taking your input text and using an LLM to process that text. It is taking the "type" of your input and making a function in python that handles it.  If you try to use it assuming that it will behave like other LLM nodes, it will not work.  Other LLM nodes can perform AI text manipulation, AnyNode can perform these functions, but only if you asked it to make you a function which uses these AI tools to perform the text manipulation.
Any tool has it's limitations. You can try to use a hammer to beat a screw into the wall, but it will not work the way you expected if you had just used a screwdriver.

In this Image: someone trying to use AnyNode as if it were Llava Captioner... doesn't work
What you should try to use other LLM Nodes For

AI text like extracting keywords,
Image comprehension
Text generation
Image or Video Generation
Audio Generation (I'd want you to use StableAudioSampler for that ^_^)

Again, you can try to get AnyNode to do things that you can do through the generative AI apis.  Think about your problem like you are talking to a python developer, asking them to make you an app. You are not directly asking the LLM to do anything beyond making that function. If you want it to do something with Generative AI you have to ask it to use openai in the function it is generating. How would a developer know which service you were trying to use to perform the above advanced capabilities?
Goal of AnyNode
The goal of AnyNode is to provide you the right tool in the right place in your workflow.  It is not to replace every other node. I am not trying to beat out the other nodes, and I respect other node developers.  I want it to assist you, not replace everything else.
Stuck? Try Using a Tool Built for your Task
There is a very large node ecosystem. Think about the work people put into their nodes developing them.  Think about how tested their nodes are, and how precise they are. Think of all the feedback their nodes got from users to help their node get better.  Those devs make amazing nodes that do things which you are trying to do. Why would you ask a junior dev to try to build that for you in one node with two inputs? I never said AnyNode doesn't have limits, I just said I don't really know where they are. I never said AnyNode replaces every node with just a "turn of phrase". You are asking a "genie" to know the hidden desires behind your wishes... but it's a genie, you have to be very specific of what you ask for, or it will try to grant your wish the best it knows how, and that might not be what you really want.

I hope this helps clear things up. Let me know if you have any questions. I will leave this thread open.

this is my prompt in the AnyNodes: "Get the keywords in the input and pass them to the output".
and i found this error the following(maybe is useful):

and i get the right method to use AnyNodes —— you must as much as possible clearly described the [input] and the [output], something like: you can tell AnyNodes what type from [input] and you want [output] what type is.

it's cool this Node, i hope this will be more strong!


Extend expected test time of SharedTsBlockQueueTest

On mac with jdk17, it could be time-consuming when loading some metric props. SharedTsBlockQueueTest fails frequently due to this:

As this test aims to test whether there's deadlock and the basic functions of SharedTsBlockQueue, not the performance of it, I extended the expected test time of SharedTsBlockQueueTest and this should fix the issue.

We're currently processing your upload.  This comment will be updated when the results are available.


Build MonitoringResource from detected platform

/cc @iantalarico

Thanks for getting this in so fast @chrisdunelm !


max-memory options

If there is an max-memory options will be very cool.

I think this is a somewhat valid option to have... At least in the form of maximum # of items in the cache, default being infinite. The implementation could easily be done using least recently used key algorithm, using for example lru-queue, and just add code to set, del and flushAll. Would a pull request for this be accepted at all? Changes go in CoffeeScript or Javascript source code or both?


creationDate and lastUpdateDate is not enforced/kept properly for edges

Currently creationDate for edges is only enforced when the edge is created from createEdge not when the edge is created by create-object calls (quite easy to fix).
What is more complicated to fix is the lastUpdateDate. As we currently recreate edges on an update-object call (if we update those edges at all), we would then get new creationDates for those edges if fixed in similar fashion as to when creating objects.
This needs to be changed if creationDate and lastUpdateDate are to behave as expected when used for queried etc..
One alternative would be to prevent edge updates from object updates, and only allow edges to be updated from actual edge update calls, which is currently not implemented.
@hartig @keski @rcapshaw

Leif, good point! Yes, these dates-related fields should not simply be ignored (and even left out) for edges that are created or updated inline within create-object or update-object operations.
As you say, the creationDate should be easy.
...and the lastUpdateDate is tricky. However, we can also take the following position: If a user updates the (outgoing) edges of a node via an update-object operation, then this is really like replacing all the existing edges with new ones, even if some of the replacing edges are equivalent to some of the replaced edges. Taking this position will simplify matters because we do not have to trouble ourselves with a need to keep edges and update their lastUpdateDate field (of course, the update-edge operations are a different matter).
Sounds good?

@hartig ,

...and the lastUpdateDate is tricky. However, we can also take the following position: If a user updates the (outgoing) edges of a node via an update-object operation, then this is really like replacing all the existing edges with new ones, even if some of the replacing edges are equivalent to some of the replaced edges. Taking this position will simplify matters because we do not have to trouble ourselves with a need to keep edges and update their lastUpdateDate field (of course, the update-edge operations are a different matter).

This is how it's done now (or technically after the correction of PR#66).
I must however question this approach as it allows the deletion of edges indirectly, which might not be desired. We could instead modify update to only allow the addition of new edges, and force all other edge modifications to be done through updateEdge and/or deleteEdge calls. This approach also raises the question if updateEdge should be able to change the source/target of an edge (and hence, modify the data of the source/target). Or might an redirectEdge function and corresponding api-call be relevant?
Either option has it's pros and cons.

I think you are right. The more I ponder on it, the more I think that we should actually disallow (and, in fact, take away) the option to update a node's (outgoing) edges via an update-object operation. This option has always been a huge source of misunderstanding and confusion. I will create a separate issue to discuss this question further and, ultimately, to plan the removal of this option.
To your other point ("the question if updateEdge should be able to change the source/target of an edge"), I would say no! Updates of edges should be concerned only with the annotations but not with moving edges around in the graph (i.e., changing what their head or their tail is). The latter would just open another can of worms and I don't see any clear use cases that justify putting effort into this now. Should we have a use case later, I would probably prefer your idea of adding a completely separate kind of operation (redirectEdge or moveEdge or something), but for the moment, let's simply ignore this.

@epii-1 now that we have decided to eventually remove the option to update a node's (outgoing) edges via an update-object operation (see #67), is there anything else left to do regarding this issue here?

As with the decision mentioned by Olaf above, this issue becomes obsolete and I will hence close it.


AO8/HotSpot was building with --variant openj9

Problematic job: https://ci.adoptopenjdk.net/job/openjdk8_build_s390x_linux
Removed --variant openj9

Fixed


Improve COPY FROM retry logic

Summary of the changes / Why this improves CrateDB
Resolves https://github.com/crate/crate/issues/13110
Checklist

[ ] Added an entry in CHANGES.txt for user facing changes
[ ] Updated documentation & sql_features table for user facing changes
[ ] Touched code is covered by tests
[ ] CLA is signed
[ ] This does not contain breaking changes, or if it does:

It is released within a major release
It is recorded in CHANGES.txt
It was marked as deprecated in an earlier release if possible
You've thought about the consequences and other components are adapted
(E.g. AdminUI)

Please bear with me If I miss something in the context.

The FileReadingIterator now, doesn't handle the Socket exceptions but instead rethrows the Exception and closes the reader? is this correct?
The BatchIteratorBackpressureExecutor which wraps the FileReadingIterator, now handles the Socket exceptions and will resume the execution after delay (given by the policy), but now the reader is already closed?
If this is the way to go, is there a way unify the getDelayInMs mechanism with the backoff policy (in case of socket exception)?

Thanks for reviewing @matriv,

The FileReadingIterator now, doesn't handle the Socket exceptions but instead rethrows the Exception and closes the reader? is this correct?
Yes that is correct (but not just closes, closes and re-initializes, which were existing logic).
The BatchIteratorBackpressureExecutor which wraps the FileReadingIterator, now handles the Socket exceptions and will resume the execution after delay (given by the policy), but now the reader is already closed?
Yes that is also correct but the reader is re-initialized such that another invocation of moveNext() would work smoothly.
If this is the way to go, is there a way unify the getDelayInMs mechanism with the backoff policy (in case of socket exception)?
So far was focused more on implementing non-blocking back-off policy. I will look into this Thank you.

Yes that is correct (but not just closes, closes and re-initializes, which were existing logic).

Apologies, I missed this.

Yes that is also correct but the reader is re-initialized such that another invocation of moveNext() can resume smoothly.

ok. thx!
I don't like that much (design-wise) that the BatchIteratorBackpressureExecutor needs to know about the Socket exception, and implement some concrete logic, although I don't have a concrete idea to somehow make this more generic in an elegant way.
Is there a way for the ShardUpsertExecutor to know that it's being called by COPY FROM so we can create some new class extending the BatchIteratorBackpressureExecutor, which also handles those SocketExceptions with a backoff policy?
Please wait for @seut review, as I'm not so familiar, and don't want to confuse/de-rail you.

@jeeminso

The FileReadingIterator now, doesn't handle the Socket exceptions but instead rethrows the Exception and closes the reader? is this correct?

Yes that is correct (but not just closes, closes and re-initializes, which were existing logic).

The BatchIteratorBackpressureExecutor which wraps the FileReadingIterator, now handles the Socket exceptions and will resume the execution after delay (given by the policy), but now the reader is already closed?

Yes that is also correct but the reader is re-initialized such that another invocation of moveNext() can resume smoothly.

Is this really working as expected given that both exception the BatchIteratorBackpressureExecutor should handle are IOExceptions which are catched by FileReadingIterator.moveNext() at https://github.com/crate/crate/blob/master/server/src/main/java/io/crate/execution/engine/collect/files/FileReadingIterator.java#L187 ?
Is your draft tested/validated somehow? If not, this would help validating your approach.
Otherwise the solution looks good to me.

I don't like that much (design-wise) that the BatchIteratorBackpressureExecutor needs to know about the Socket exception, and implement some concrete logic, although I don't have a concrete idea to somehow make this more generic in an elegant way.
Is there a way for the ShardUpsertExecutor to know that it's being called by COPY FROM so we can create some new class extending the BatchIteratorBackpressureExecutor, which also handles those SocketExceptions with a backoff policy?

I wonder if may could/should catch generic IOExceptions instead, this would also make it more generic and not bound to socket errors. @mfussenegger wdyt?

Another approach that came to mind is to have a dedicated RetryBatchIterator(BatchIterator<T> bi, Iterable<TimeValue> backoffPolicy, List<Class<?>> retryExceptions). (Or Predicate<Throwable>).
That would allow to add retry handling on top of any batchIterator. The advantage over extending BackpressureExecutor is that it doesn't change the interface, but is transparent to consumers. And it could be used independently. It's probably easier to test as well.

Hi @mfussenegger I have 5aa4b95 that is a rough draft of RetryBatchIterator but

That would allow to add retry handling on top of any batchIterator.

seems not possible since the source batchIterator is required to catch the exceptions and get ready to re-try.
For example, a source batchIterator may react to an exception by closing/releasing resources or subsequent calls to moveNext() may not resume from expected index.

status update:
Taking loadNextBatch() approach.
The draft implements retry with back-off policy and the only thing remaining should be testing. Currently we have a single test covering retries caused by socket exceptions, I will put a few more tests with socket exceptions combined with headers and multiple URIs, etc.

Hello @seut, it would be great if you could have a quick look at my new draft when you have time. Thanks!

@jeeminso I think the approach looks good.

retest this please: test_copy_from_with_fail_fast_with_single_node
java.lang.AssertionError:
Expecting actual throwable to be exactly an instance of:
  io.crate.exceptions.JobKilledException
but was:
  java.lang.IllegalStateException: BatchIterator is closed
	at io.crate.data.BatchIterator.<clinit>(BatchIterator.java:73)
	at io.crate.testing.TestingBatchIterators.range(TestingBatchIterators.java:41)
	at io.crate.action.sql.CursorTest.rows(CursorTest.java:640)
	...(41 remaining lines not displayed - this can be changed with Assertions.setMaxStackTraceElementsDisplayed)

I don't think the current changes will cause this flakyness which I cannot reproduce with several repetitions locally.

Is the current back-off delays good enough?
BackoffPolicy.exponentialBackoff(TimeValue.ZERO, MAX_SOCKET_TIMEOUT_RETRIES)

Is the current back-off delays good enough? BackoffPolicy.exponentialBackoff(TimeValue.ZERO, MAX_SOCKET_TIMEOUT_RETRIES)

Yes, we could add 10ms initial delay, but we also retried immediately with the old code. I think it's fine to do the first retry immediately.


Memory Usage by Component legend label

Hi,
After loading nmon file(s) into the tool I click on the "Memory Use" tab which draws the graph for "Memory Usage by Component".
On the left side of the graph the label shows "MB" - but I think this should be showing either "%" or "Percentage".   Do you agree?
Thanks
Dean

Yes, oops. It's an easy fix here which I will do.
Just to confirm, are these values always between 0 and 100? If not, I'd need to make another change to ensure the Y axis isn't limited to that range.


charmxi: mismatched module braces leads to confusing error messages.

Original author: Ralf Gunter Corrêa Carvalho
Original issue: https://charm.cs.illinois.edu/redmine/issues/979

The following code sample raises a very obscure error message:
mainmodule mod {   mainchare mchare {     entry mchare(CkArgMsg *m);   }
~/p/p/c/t/c/c/tmp ❯❯❯ ~/programming/ppl/charm/bin/charmxi tmp.ci tmp.ci:5:4-4: error: invalid construct

Original date: 2017-12-29 17:48:01

Reassign to Evan based on discussion with Eric B.  Confer on how important this work is to schedule in the near future.

Original date: 2018-01-03 22:38:04

The output of charmxi with the given test case is currently:
[PARSE-ERROR] Unexpected/missing token at line 5. Current token being parsed: ''.

This is better than the "invalid construct" message in the original report. It's still not the clearest but seems sufficient to get the point across.

Original date: 2018-01-16 23:09:08

The above message seems a good enough solution to me. I'm closing this. Feel free to re-open if there is disagreement.