refactor[next]: Change default datatype for tests to float

Currently most tests use int64 as the default dtype for fields:
IField: TypeAlias = common.Field[[IDim], np.int64]

This PR changes this to float for 2 reasons:

Most of our use cases are with fields of floats.
Using int literals, e.g. 1, 2 will break after https://github.com/GridTools/gt4py/pull/1255 has been merged as they will be int32 instead of int64.

@havogt Rico mentioned you suggested to use int when he refactored the tests. Please take a short look on the approach.

My reason for int is that you can do equality comparison, therefore if dtype doesn't matter for the test, I would argue it's better.

My reason for int is that you can do equality comparison, therefore if dtype doesn't matter for the test, I would argue it's better.

In that case we should change the default comparison in cases.verify to np.equal. It is currently np.allclose.

My reason for int is that you can do equality comparison, therefore if dtype doesn't matter for the test, I would argue it's better.

In that case we should change the default comparison in cases.verify to np.equal. It is currently np.allclose.

We could, but probably it doesn't matter, I'd expect np.allclose == np.equal for all ints (but I'd have to do the math). Then np.allclose would be better as it works for all types.

btw. for the second problem mentioned in the description: no problem changing the default to int32

We could, but probably it doesn't matter, I'd expect np.allclose == np.equal for all ints (but I'd have to do the math). Then np.allclose would be better as it works for all types.

Ok, that doesn't make sense... We should use equal then...

We could, but probably it doesn't matter, I'd expect np.allclose == np.equal for all ints (but I'd have to do the math). Then np.allclose would be better as it works for all types.

Ok, that doesn't make sense... We should use equal then...

Why doesn't it make sense? When using np.allclose() with ints, it will convert the values to floats first and then do the comparison, so if two int values are equal before the conversion, it will always work , right?

a = np.asarray([1e10], dtype=np.int64)
b = np.asarray([1e10+1], dtype=np.int64)
np.allclose(a,b)
True

I think I misunderstood the point. I agree it doesn't make sense to use int as default type, what I meant is that it should be ok to use np.allclose() for comparisons even with int arrays

a = np.asarray([1e10], dtype=np.int64)
b = np.asarray([1e10+1], dtype=np.int64)
np.allclose(a,b)
True

I think I misunderstood the point. I agree it doesn't make sense to use int as default type, what I meant is that it should be ok to use np.allclose() for comparisons even with int arrays

Now I am confused, I still think it makes sense to use int as default type and then do equality comparison, because it's a bit stricter than float with allclose (in the default case where we explicitly don't care about the values and the dtype).

Using int to allow for equality comparison makes sense. I switched to using int32 as the default type in the tests. The change only works properly together with https://github.com/GridTools/gt4py/pull/1255 so closing here. Changing the comparison function is outside of the scope of this PR. I have opend an issue here: https://github.com/GridTools/gt4py/issues/1269


Add position order for handlers

onexit function could be async and that would simplify the next/done part by just doing await, but I wanted to keep practically the same code for when it doesn't use positions.
Edit: it happened several times where I have multiple resources that I want to clean up but sometimes it's really important the order of terminating those resources, and they could be sparsed in different files, etc.
This reminds me of CSS property z-index but in reverse priority.

1.1.0


KEDA crashes when using cert-manager certificates and restricted secret access

When enabling restricted secret access feature and using cert-manager instead of self-managed certificates, KEDA operator and metrics-server fail to start.
A possible workaround for this is providing access to own-namespace secrets by creating Role and RoleBinding manually or via extraObjects in chart values.
Expected Behavior
KEDA chart renders necessary Role and RoleBinding under these conditions
OR
KEDA doesn't try to access any secrets
Actual Behavior
KEDA chart doesn't render Role and RoleBinding, leaving no access to secrets at all (which would otherwise be available through ClusterRole). KEDA components try to access secrets in own namespace, failing to do so due to missing access permissions, and crash.
Steps to Reproduce the Problem

Restrict global secret access via following Helm values:

permissions:
  metricServer:
    restrict:
      secret: true
  operator:
    restrict:
      secret: true

Enable cert-manager certificates instead of self-generated ones via following Helm values:

certificates:
  autoGenerated: false
  certManager:
    enabled: true

Deploy KEDA to cluster
KEDA metrics-server and operator crash with following error:

W0807 18:55:16.700808       1 reflector.go:424] k8s.io/client-go/informers/factory.go:150: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
E0807 18:55:16.700852       1 reflector.go:140] k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
W0807 18:55:18.950561       1 reflector.go:424] k8s.io/client-go/informers/factory.go:150: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
E0807 18:55:18.950596       1 reflector.go:140] k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
W0807 18:55:25.059600       1 reflector.go:424] k8s.io/client-go/informers/factory.go:150: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
E0807 18:55:25.059638       1 reflector.go:140] k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"

Specifications

KEDA Version: 2.11.0
Chart Version: 2.11.2

I think that this is an error in the chart, because the secret access has to be granted if restricted mode is set.
Are you willing to open a PR fixing it?

I am facing the same issue and would have time to open PR.
Any pointers?

I have opened #518 to attempt to fix this issue.
It seems to work in our test environment.


Improve config manager locking

Codecov Report

Merging #10 into master will decrease coverage by 0.86%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master     #10      +/-   ##
=========================================
- Coverage   43.36%   42.5%   -0.87%
=========================================
  Files           6       6
  Lines         196     200       +4
=========================================
  Hits           85      85
- Misses        109     113       +4
  Partials        2       2

Impacted Files
Coverage Δ

config/manager.go
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1188b7d...17ee678. Read the comment docs.


custom_components/multimatic/sensor.py:103 AttributeError: 'NoneType' object has no attribute 'unit'

Have been getting this error for some time now. Running multimatic component v1.6.0, HA Container v2021.5.0
Logger: homeassistant
Source: custom_components/multimatic/sensor.py:103
Integration: Multimatic (documentation, issues)
First occurred: 23 mei 2021 19:23:57 (66 occurrences)
Last logged: 16:32:20
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 127, in _handle_refresh_interval
await self._async_refresh(log_failures=True, scheduled=True)
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 259, in _async_refresh
update_callback()
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 319, in _handle_coordinator_update
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 334, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 378, in _async_write_ha_state
unit_of_measurement = self.unit_of_measurement
File "/config/custom_components/multimatic/sensor.py", line 103, in unit_of_measurement
return self.report.unit
AttributeError: 'NoneType' object has no attribute 'unit'

I also have a problem the sam.... and still no read infor from multimatic... some time read ok but sometime no read.
`Logger: homeassistant
Source: custom_components/multimatic/sensor.py:103
Integration: Multimatic (documentation, issues)
First occurred: 25 maja 2021, 17:46:10 (147 occurrences)
Last logged: 21:49:14
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 127, in _handle_refresh_interval
await self._async_refresh(log_failures=True, scheduled=True)
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 259, in _async_refresh
update_callback()
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 319, in _handle_coordinator_update
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 334, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 378, in _async_write_ha_state
unit_of_measurement = self.unit_of_measurement
File "/config/custom_components/multimatic/sensor.py", line 103, in unit_of_measurement
return self.report.unit
AttributeError: 'NoneType' object has no attribute 'unit'
`

same behaviour here.

Just released 1.7.0b0 see https://github.com/thomasgermain/vaillant-component/releases/tag/1.7.0b0
It should fix the issue (or at least reduce it). You can see it through HACS, if you enable beta
(I tested it for 2 days and I didn't have any error, expect around 3:30).

Upgraded to latest and greatest of HA DC (v2021.6.2) and multimatic (v1.7.0b0 via git clone) in the custom_components dir.
It resulted in all multimatic related entities being duplicated. I removed the duplicate multimatic entities which were not working, resulting in a set the full set of entities being operational.
Will keep an eye on the logs and revert back in case something pops up.
I enabled HACS beta versions but did not find any trace of multimatic or vaillant.

@Wiggum127 you can check 1.7.0b1, it should the issue

Upgraded to 1.7.0b1. Will keep an eye on things.

Do you still have the issue ?

I did not find the same error back for a couple of days running 1.7.0b3. I'm getting a lot of connection errors with the gateway from different .py files, but not this error.

Ok se closing it, feel free to open a new issue if you have still problem


Add gh-pages infrastructure

As a developer, I'd like to add gh-pages infrastructure, so we could migrate the current spring-cloud-stream-modules site as app-starters.

Review and update all the other project links

This is complete: http://cloud.spring.io/spring-cloud-stream-app-starters/


Can not reproduce results on the table

Great thanks for your work! I try exacy the same setting but I got different results on MMLU and BBH. The alpaca-tuned llama always perform worse than original llama(7B or 13B). Is there anything wrong with the loaded models?

Thanks for raising this issue, we are currently investigating it. Based on initial checking, it may be due to changed behavior of LlamaTokenizer when we upgraded transformer library version (git+https://github.com/huggingface/transformers.git@057e1d74733f52817dc05b673a340b4e3ebea08c to 4.28.1)

Thanks a lot! So can the provided transformers version(git+https://github.com/huggingface/transformers.git@057e1d74733f52817dc05b673a340b4e3ebea08c ) help me reproduce the correct results?

We are currently retesting the models, but it would be a great help if you could also try with the older transformers version (git+https://github.com/huggingface/transformers.git@057e1d74733f52817dc05b673a340b4e3ebea08c). If you can also reproduce the results, then we know the cause of the issue for sure, and we can revert to this transformers version in the short term. In the long term, we may need to debug the LlamaTokenizer in the newer library version

We have confirmed that the problem is due to transformers library version, this has been fixed in the latest commit. For example, the command python main.py mmlu --model_name llama --model_path chavinlo/alpaca-native gives a result of Average accuracy: 0.416. Would you mind trying on your end if it is fixed for you too?

Thanks for your reply. I've tried again and the result seems to be fine. Can you provide more details about the cause of this problem, so I won't have such problems as the version conflict in the future. I'd be very grateful!

No problem, we are still working to ensure that the issue is fully resolved in the newer transformer version, it is a subtle issue as the newer LlamaTokenizer tokenizes whitespace a bit differently

Hi, May I ask the update about this issue.


Pictures not showing up under some players

After setting the picture for some files, their files in GNOME Nautilus no longer displayed a thumbnail. Also, the files I did this to did not show their album art in my car's infotainment system.
Doing some manual troubleshooting, I was able to narrow this down to an issue with the picture mimetype not being set. Using a different tag editor (Kid3) to fill in the picture mimetype as "image/jpeg" allowed the image to show up in thumbnails and my car again.
Also, it seems that this picture mimetype field gets clobbered even when setting unrelated fields, but that's probably a separate issue.

I will take a look at this and make sure the mimetype is properly set.


COSTARS data entry page for Riley

This should do the following:

List all COSTARS contracts by COSTARS number
Basic data entry form:

COSTARS basic metadata
Some sort of way of handling variable numbers of line items, manufacturers, etc.

Let me know what you think @rileystewart

[ ] new form
[ ] new view
[ ] new template
[ ] line items (tricky)


git generates: warning: LF will be replaced by CRLF in src/graphql/generated/fragment-matcher.ts

Describe the bug
Whenever we create a new file with extension .graphql and run graphql-codegen to generate the required files for typescript we get the following error from git:

To Reproduce

Create a file test.graphql on a Windows machine
Run graphql-codegen
Add the newly generated files to git git add .

Notice line ending warnings

The GraphQL generated code can be found here.

My codegen.js config file:

/* eslint-disable @typescript-eslint/no-unsafe-member-access */
/* eslint-disable @typescript-eslint/no-var-requires */

const fs = require('fs')

const token = fs.readFileSync('./token.txt')

module.exports = {
  overwrite: true,
  schema: {
    [process.env.BACKEND_API_URI]: {
      headers: {
        Authorization: `Bearer ${token}`,
      },
    },
  },
  documents: 'src/**/*.graphql',
  hooks: {
    afterAllFileWrite: ['prettier --write'],
  },
  generates: {
    'src/graphql/generated/schema.json': {
      plugins: ['introspection'],
    },
    'src/graphql/generated/operations.ts': {
      plugins: [
        {
          add: {
            content: '/* eslint-disable */',
          },
        },
        'typescript',
        'typescript-operations',
        'typescript-vue-apollo',
      ],
      config: { withCompositionFunctions: true },
    },
    'src/graphql/generated/fragment-matcher.ts': ['fragment-matcher'],
  },
}

Expected behavior
No line ending warnings as this isn't happening for manually created files.
Environment:

OS: Windows Server 2016
@graphql-codegen/...:

    "@graphql-codegen/add": "^2.0.2",
    "@graphql-codegen/cli": "^1.19.4",
    "@graphql-codegen/fragment-matcher": "2.0.1",
    "@graphql-codegen/introspection": "1.18.1",
    "@graphql-codegen/typescript": "^1.19.0",
    "@graphql-codegen/typescript-operations": "^1.17.12",
    "@graphql-codegen/typescript-vue-apollo": "^2.3.1",

NodeJS: v12.18.1

@ardatan @kamilkisiela any idea how to solve that?

I don't think this is a critical error and actually it is not an error at all.
We can normalize the final content before saving to the disk by using the following package;
https://www.npmjs.com/package/eol

This might be creating some noise in git changes making it harder to recognize intentional vs unintentional pending changes.  When the content is otherwise identical, the change disappears when staged - I think this is because git is renormalizing the line endings at that time.  It isn't consistent though - if I undo the git change and build again, git does not recognize another change.  These generated files are excluded from prettier and linting.  I'm not sure why line endings would be plaguing this tool specifically.


Using FQDNs for meshNetworks gateways silently fails

Bug description
Specifying address: <IP_ADDRESS> in a gateway section works for us, that endpoint shows up appropriately when going cross-cluster.
Specifying address: some.fqdn.com does not. The gateway endpoints silently vanish.
Expected behavior
The address would resolve to the IP address.
We've verified that from the pilot discovery container host can resolve the hostname we're trying to use.
If there is an error in https://github.com/istio/istio/blob/1.1.2/pilot/pkg/networking/util/util.go#L258 I would expect it to be printed somewhere in the logs so we're able to debug, instead of it silently failing.
(Indeed, if it fails to resolve these addresses due to some temporary network issue during startup, that's going to cause serious problems with the mesh. It might be better to simply crash out if given invalid config, rather than starting up and breaking all cross-cluster traffic)
Steps to reproduce the bug
Try using a hostname for the gateway address
Version (include the output of istioctl version --remote and kubectl version)
client version: version.BuildInfo{Version:"1.1.3", GitRevision:"d19179769183541c5db473ae8d062ca899abb3be", User:"root", Host:"fbd493e1-5d72-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Clean", GitTag:"1.1.2-56-gd191797"}
citadel version: version.BuildInfo{Version:"1.1.2", GitRevision:"2b1331886076df103179e3da5dc9077fed59c989-dirty", User:"root", Host:"35adf5bb-5570-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.1"}
egressgateway version: version.BuildInfo{Version:"bdda7cfcf5ba1397e6e0e2629d53114c9ea8fc14", GitRevision:"bdda7cfcf5ba1397e6e0e2629d53114c9ea8fc14", User:"mjog", Host:"devinstance.c.mixologist-142215.internal", GolangVersion:"go1.10.1", DockerHub:"docker.io/istio", BuildStatus:"Clean", GitTag:"1.1.0-snapshot.4-592-gbdda7cf"}
galley version: version.BuildInfo{Version:"1.1.2", GitRevision:"2b1331886076df103179e3da5dc9077fed59c989-dirty", User:"root", Host:"35adf5bb-5570-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.1"}
ilbgateway version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
ingressgateway version: version.BuildInfo{Version:"bdda7cfcf5ba1397e6e0e2629d53114c9ea8fc14", GitRevision:"bdda7cfcf5ba1397e6e0e2629d53114c9ea8fc14", User:"mjog", Host:"devinstance.c.mixologist-142215.internal", GolangVersion:"go1.10.1", DockerHub:"docker.io/istio", BuildStatus:"Clean", GitTag:"1.1.0-snapshot.4-592-gbdda7cf"}
pilot version: version.BuildInfo{Version:"1.1.2", GitRevision:"2b1331886076df103179e3da5dc9077fed59c989-dirty", User:"root", Host:"35adf5bb-5570-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.1"}
policy version: version.BuildInfo{Version:"1.1.2", GitRevision:"2b1331886076df103179e3da5dc9077fed59c989-dirty", User:"root", Host:"35adf5bb-5570-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.1"}
sidecar-injector version: version.BuildInfo{Version:"1.1.2", GitRevision:"2b1331886076df103179e3da5dc9077fed59c989-dirty", User:"root", Host:"35adf5bb-5570-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.1"}
telemetry version: version.BuildInfo{Version:"1.1.2", GitRevision:"2b1331886076df103179e3da5dc9077fed59c989-dirty", User:"root", Host:"35adf5bb-5570-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.1"}

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.12", GitCommit:"c757b93cf034d49af3a3b8ecee3b9639a7a11df7", GitTreeState:"clean", BuildDate:"2018-12-19T11:16:52Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.12", GitCommit:"c757b93cf034d49af3a3b8ecee3b9639a7a11df7", GitTreeState:"clean", BuildDate:"2018-12-19T11:04:29Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

How was Istio installed?
Using the helm template command. Mesh network setup taken from https://istio.io/docs/examples/multicluster/split-horizon-eds/
Environment where bug was observed (cloud vendor, OS, etc)
AWS EKS
Additionally, please consider attaching a cluster state archive by attaching
the dump file to this issue.

I started trying to add some debug output for this, and I think I spotted the real bug - the host resolution is done on the wrong object which then gets overwritten. Opened #13582

Fix merged.


[🐞] Cannot find module 'bun:sqlite'

Which component is affected?
Qwik Runtime
Describe the bug
I'm trying to use use and deploy qwik using Bun and it's sqlite libray "bun:sqlite", without success.
After bun create qwik@latest and selecting Empty App, I replaced the src/routes/index.tsx file with the content below:
import { component$ } from<EMAIL_ADDRESS>import { server$ } from<EMAIL_ADDRESS>import { Database } from "bun:sqlite";

const servertest = server$((text: string) => {
  console.log(text);
  const db = new Database(":memory:");
  const query = db.query("create table foo;");
  query.run;
  return "hello from server";
});

export default component$(() => {
  return (
    <>

Hi 👋

      <button
        onClick$={async () => {
          const greeting = await servertest("hello from client");
          console.log(greeting);
        }}
      >
        greet
      </button>
    </>
  );
});

When I run bun start I get the error: [vite] Internal server error: Cannot find module 'bun:sqlite' imported from '/Users/me/test1/src/routes/index.tsx'
Trying to deploy using Bun didn't work either. This is what I tried.
First, added the integration: bun run qwik add bun
I also installed Bun's types with bun install -d bun-types and added "bun-types" to compilerOptions.types in tsconfig.js
And when I try to run bun run build I get this error: [vite]: Rollup failed to resolve import "bun:sqlite" from "/Users/henpa/dev/test1/src/routes/index.tsx". This is most likely unintended because it can break your application at runtime. If you do want to externalize this module explicitly add it to build.rollupOptions.external
Then I tried to add to vite.config.ts the following:
    build: {
      rollupOptions: {
        external: ["bun:sqlite"]
      }
    },

When I run bun run build again I get this error: Error [PLUGIN_ERROR]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. Received protocol 'bun:'
Is it possible to run and deploy to Bun and use bun:sqlite? Thanks!
Reproduction
https://github.com/henpa/test1
Steps to reproduce
bun create qwik@latest
bun start
bun run qwik add bun
System Info
System:
    OS: macOS 14.2.1
    CPU: (8) arm64 Apple M2
    Memory: 2.46 GB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.10.0 - /usr/local/bin/node
    npm: 10.2.3 - /usr/local/bin/npm
    pnpm: 8.10.5 - ~/Library/pnpm/pnpm
    bun: 1.0.18 - ~/.bun/bin/bun
  Browsers:
    Chrome: 120.0.6099.109
    Safari: 17.2.1
  npmPackages:
    @builder.io/qwik: ^1.3.1 => 1.3.1
    @builder.io/qwik-city: ^1.3.1 => 1.3.1
    undici: ^5.26.0 => 5.28.2
    vite: ^5.0.6 => 5.0.10

Additional Information
No response

Hi @henpa this issue https://github.com/vitejs/vite/issues/14351 is a similar one 🤔

@henpa Did you solve this issue?


"Keyword 'MISSION' not found." for some fits files

Problem description
Some Kepler light curve fits files downloaded with the wget scripts at https://archive.stsci.edu/kepler/publiclightcurves.html do not have the MISSION keyword in their headers (especially quarter 1 data for KIC 9821777 and above?). This missing keyword renders the data inaccessible to lightkurve.  I have not been able to set the mission attribute manually either. Can this attribute be None if the keyword is not in the header as is done for "Campaign" for Kepler data?
Example

import lightkurve
klf = lightkurve.KeplerLightCurveFile(path='kplr009821777-2009166043257_llc.fits')
print(klf.PDCSAP_FLUX)

Expected behavior
Should print the flux array, but instead returns KeyError: "Keyword 'MISSION' not found."
Environment:

platform (e.g. Linux, OSX, Windows): Linux
lightkurve version (e.g. 1.0b6): 1.0b8
installation method (e.g. pip, conda, source): pip

Hi @keatonb,
thanks a lot for investigating this!
However, I can't reproduce this error when I download the lightcurve file from the archive by doing:
>>> klf = KeplerLightCurveFile.from_archive(9821777, quarter=1)
>>> print(klf.PDCSAP_FLUX)
KeplerLightCurve(KIC: 9821777)
>>> klf.mission
'Kepler'

@barentsen what does this mean with respect to lightcurve files from here?

If you find this ever to be true in the future, Astropy has proposed a header-edit process that worked for me (I was just monkeying around to see if it would work):
http://www.astropy.org/astropy-tutorials/FITS-header.html
Obviously, you need to know what goes in the element you are adding.

I also found that the individual files on the MAST archive do have this header, but the files in the tarballs (downloaded ~1 month ago) don't always.

Okay, I found that particular FITS at
http://archive.stsci.edu/pub/kepler/lightcurves/0098/009821777/kplr009821777-2009166043257_llc.fits
and it seems intact:
import lightkurve
PATH="http://archive.stsci.edu/pub/kepler/lightcurves/0098/009821777/kplr009821777-2009166043257_llc.fits"
klf = lightkurve.KeplerLightCurveFile(path=PATH)
print(klf.PDCSAP_FLUX)

Here's the full header of the file I have:

BITPIX  =                    8 / array data type
NAXIS   =                    0 / number of array dimensions
EXTEND  =                    T / file contains extensions
NEXTEND =                    2 / number of standard extensions
EXTNAME = 'PRIMARY '           / name of extension
EXTVER  =                    1 / extension version number (not format version)
ORIGIN  = 'NASA/Ames'          / institution responsible for creating this file
DATE    = '2014-11-04'         / file creation date.
CREATOR = '585725 FluxExporter2PipelineModule' / pipeline job and program used t
PROCVER = 'svn+ssh://murzim/repo/soc/tags/release/9.2.22 r57227' / SW version
FILEVER = '5.0     '           / file format version
TIMVERSN= 'OGIP/93-003'        / OGIP memo number for file format
TELESCOP= 'Kepler  '           / telescope
INSTRUME= 'Kepler Photometer'  / detector type
OBJECT  = 'KIC 9821777'        / string version of KEPLERID
KEPLERID=              9821777 / unique Kepler target identifier
CHANNEL =                   39 / CCD channel
SKYGROUP=                   23 / roll-independent location of channel
MODULE  =                   12 / CCD module
OUTPUT  =                    3 / CCD output
QUARTER =                    1 / Observing quarter
SEASON  =                    3 / mission season during which data was collected
DATA_REL=                   24 / version of data release notes for this file
OBSMODE = 'long cadence'       / observing mode
RADESYS = 'ICRS    '           / reference frame of celestial coordinates
RA_OBJ  =           287.236781 / [deg] right ascension
DEC_OBJ =            46.614040 / [deg] declination
EQUINOX =               2000.0 / equinox of celestial coordinate system
PMRA    =               0.0000 / [arcsec/yr] RA proper motion
PMDEC   =               0.0000 / [arcsec/yr] Dec proper motion
PMTOTAL =               0.0000 / [arcsec/yr] total proper motion
PARALLAX=                      / [arcsec] parallax
GLON    =            77.331469 / [deg] galactic longitude
GLAT    =            16.542294 / [deg] galactic latitude
GMAG    =               14.833 / [mag] SDSS g band magnitude
RMAG    =               14.317 / [mag] SDSS r band magnitude
IMAG    =               14.151 / [mag] SDSS i band magnitude
ZMAG    =               14.093 / [mag] SDSS z band magnitude
D51MAG  =               14.622 / [mag] D51 magnitude,
JMAG    =               13.103 / [mag] J band magnitude from 2MASS
HMAG    =               12.773 / [mag] H band magnitude from 2MASS
KMAG    =               12.736 / [mag] K band magnitude from 2MASS
KEPMAG  =               14.356 / [mag] Kepler magnitude (Kp)
GRCOLOR =                0.516 / [mag] (g-r) color, SDSS bands
JKCOLOR =                0.367 / [mag] (J-K) color, 2MASS bands
GKCOLOR =                2.097 / [mag] (g-K) color, SDSS g - 2MASS K
TEFF    =                 5948 / [K] Effective temperature
LOGG    =                4.329 / [cm/s2] log10 surface gravity
FEH     =               -0.280 / [log10([Fe/H])]  metallicity
EBMINUSV=                0.117 / [mag] E(B-V) redenning
AV      =                0.361 / [mag] A_v extinction
RADIUS  =                1.088 / [solar radii] stellar radius
TMINDEX =            278003793 / unique 2MASS catalog ID
SCPID   =                      / unique SCP processing ID
CHECKSUM= '9A1KF61H9A1HC51H'   / HDU checksum updated 2014-11-04T22:03:44Z
END

Thanks for the reference.  Downloading.
Is there any interest in having the FITS files repaired?
MISSION=?
Any other hdu[n] elements need to be added/updated/deleted?

It looks like the lightcurves in the tarballs under lightcurves/tarfiles at MAST are not the final versions.  For example, file kplr100000926-2009166043257_llc.fits obtained from http://archive.stsci.edu/pub/kepler/lightcurves/tarfiles/Q1_public/public_Q1_long_6.tgz shows the following version info in the header:
DATE    = '2014-11-04'         / file creation date.
CREATOR = '585867 FluxExporter2PipelineModule' / pipeline job and program used t
PROCVER = 'svn+ssh://murzim/repo/soc/tags/release/9.2.22 r57227' / SW version
FILEVER = '5.0     '           / file format version

In contrast, the file obtained directly from https://archive.stsci.edu/missions/kepler/lightcurves/1000/100000926/kplr100000926-2009166043257_llc.fits shows the following version info:
DATE    = '2015-09-01'         / file creation date.
CREATOR = '796539 FluxExporter2PipelineModule' / pipeline job and program used t
PROCVER = 'svn+ssh://murzim/repo/soc/tags/release/9.3.22 r60269' / SW version
FILEVER = '6.1     '           / file format version

It looks like the tarballs at MAST must be re-created i.e. updated to reflect the final Kepler data release.
I am going to close this issue because this is not a lightkurve problem, but I will be asking @JeffLCoughlin and @mustaric to work with MAST to resolve this problem.

(Many thanks @keatonb for bringing this issue to our attention :+1: :+1: :+1: )

Yes many thanks @keatonb for finding this. I've notified our contact at MAST and will be sure to follow-up with them to fix it. Most of MAST is out today and tomorrow, so I expect a resolution sometime next week.

Is that why their domain is down?


FPS almost halved going from 4.4 dev3 to 4.4 dev 4.

Tested versions
Reproducible in Godot 4.4 dev 4.
System information
Godot v4.4.dev4 - Windows 10.0.22631 - Single-window, 2 monitors - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 4090 (NVIDIA; <IP_ADDRESS>14) - 13th Gen Intel(R) Core(TM) i7-13700K (24 threads)
Issue description
I seem to have a much lower fps in Godot 4.4 dev4 compared to Godot 4.3 dev3.
Here are some benchmarks within the same scenario.
Dev 3 editor: 383fps
Dev 4 editor: 200fps
Dev 3 exported: 402fps
Dev 4 exported: 240fps
This is using the Forward+ renderer for a 2D game. I also make heavy use of the RenderingServer. Sorry I have not had the chance to test this further.
Steps to reproduce
Try open a 2D game scene using the Forward+ renderer in Godot 4.4 dev3 and then also in Godot 4.4 dev4. (Please note, i have not tried this on a new empty scene)
Minimal reproduction project (MRP)
N/A

I can't reproduce with a simple 2D project I have available like https://github.com/johncoffee/ngj-2024/
With 4.3-dev3 and 4.3-dev4 I get around 1070-1100 FPS with --disable-vsync --print-fps in the main menu scene.
So we'll need more details about what your project is doing to pinpoint what causes the perceived performance regression.

I have taken some images of the debug profilers to see if it can help shed any light onto the issue.
Here is the Main Menu in dev 4.4.

The same Main Menu in dev 4.3.

Here are the profilers in dev 4.4 when in game.

And the same profilers in dev 4.3 when in game.

Please note that the remote scene tree is also extremely slow in dev 4.4 making it difficult to test anything. I did manage to 'hide' everything in the scene for dev 4.4 though and it didn't seem to make any difference.
The profilers above were checked one after the other, in the exact same situation. I just loaded up in dev 4.4, loaded the level, done the checks. Closed dev 4.4, loaded up dev 4.3 and done the same.
Was there any new settings or anything added to dev 4.4 that I could try turning off or that requires adjustments?

You can at least see that rendering point light shadows is taking twice as long on the CPU and Resources have decreased considerably.

I did mention the project makes heavy use of the RenderingServer, I am not sure what other information you require? Feel free to let me know and i will try and provide it.
Anyhow, I have done some more limited testing, though unfortunately i have ran out of time for now, but commenting out canvas_item_add_texture_rect and turning off shadows for lights ensure the project stays the same FPS in both dev3 and dev4. So I suspect its something to do with the RenderingServer.
RenderingServer.canvas_item_add_texture_rect(ci_rid, Rect2i(Vector2i(0,0), texture.get_size()), texture.get_rid(), true, c)
RenderingServer.canvas_light_set_shadow_enabled(ci_rid, false)
I'm not entirely sure if that will help all that much but when i have some more free time i will try and produce an MRP.

I've tested with the Isometric Demo which has many lights with shadows and I can't reproduce a performance drop between dev3 and dev4 and dev5.
I'm testing on Integrated so this demo is GPU-bottlenecked for me. If the regression is on the CPU-side then I might not see it.
dev5 does include a performance improvement to the CPU-cost of rendering shadows for PointLight2Ds. It might help alleviate the issue, but we should still find the source of the regression.
To do that, we really need an MRP so that we can test. Without one we are just blindly guessing.

Unfortunately I'm around 5 years deep into my project so unless i can pinpoint the actual issue, which i currently cannot, its not easy for me to produce an MRP. I'm also not too keen on sharing large chunks of my codebase publicly.
I have noticed that the issue is only when using Forward+. Compatibility keeps frames consistent across both dev 3 and dev 4 in all cases that i have tried (although when using lights in Compatibility the FPS is actually much lower than even dev 4 when using Forward+).
If anyone has any ideas on what i could test or check, if there's anything jumping out at you that could effect only the Forward+ renderer within the changes between dev 3 and 4 then please let me know.

The only PR I can think of is https://github.com/godotengine/godot/pull/98652 this one. It doesn't impact 2D lights specifically, but it does impact performance for 2D rendering generally. However, in all the testing we did, this improved performance. It introduced a new project setting though. You could try increasing the size of rendering/2d/batching/uniform_set_cache_size I would increase it to a around 2048 and see if it makes a difference

If you have the time, you can also try bisecting between dev 3 and dev 4, to locate the PR that caused it.

If you have the time, you can also try bisecting between dev 3 and dev 4, to locate the PR that caused it.

This is the commit from using this approach: 0d1d94572750b624e10aa6e655011e37028fa1c5
2D: Fix various issues and minor performance optimisations
FPS is not on par with dev 3 but I am assuming that's because I am not compiling for production use? However it is much better than in dev 4.

CC @stuartcarnie

Dev 3 = 380~ fps
Dev 4 = 254~ fps
Dev 4 adjusting uniform_set_cache_size to 2048 = 292~ fps (Changing this to 512 = 300~ fps)

The best FPS achieved using the bisecting method (uniform_set_cache_size = 256) = 335~ fps

☝ 256 is the default setting:
https://github.com/godotengine/godot/blob/6e9d31f602c2c31e047db9173e728fe17706185c/servers/rendering_server.cpp#L3589
I'm confused how Dev 4 = 245 fps (presumably using the default setting), and then also 335 fps set to 256.
That PR did a few issues, such that dev 3 was not rendering correctly, so we should try a version that was rendering correctly.
Could you try Godot 4.4 dev 2, which is prior to all canvas batching changes.

I'm confused how Dev 4 = 245 fps (presumably using the default setting), and then also 335 fps set to 256.

Yes, dev 4 was using the default of 256. The 335 was after I assume the bisecting removed your pr? It's the first time I have used this feature so I'm not that familiar with it.
Dev 2 gives me around 384~ or pretty much the same as dev 3.
Here is the scene I am rendering.

I am having the same problem. My game is also in 2D and I'm targeting mobile and I've had to downgrade from dev4 or dev5 back to dev3 since it's really noticeable on the exported APK with a drop of from around 80fps down to 40fps on a regular gameplay scene. However, my PC runs the game fine and I haven't seen any differences in the profiler so I haven't been able to figure out what is causing this.
My game is a simulation game and it features a bunch of sprites on screen, a good amount of area detectors and some whole screen shaders. I have no lighting at all so I can rule that out.

My game is a simulation game and it features a bunch of sprites on screen, a good amount of area detectors and some whole screen shaders. I have no lighting at all so I can rule that out.

Are your sprites in a single texture atlas or separate textures?

My game is a simulation game and it features a bunch of sprites on screen, a good amount of area detectors and some whole screen shaders. I have no lighting at all so I can rule that out.

Are your sprites in a single texture atlas or separate textures?

Separate textures for different families of objects.

Guests are an atlas (colors are applied via shaders), trees have an atlas, terrain textures have an atlas, each animal is a separate atlas. For that picture there are around 10 atlases, but a few hundred sprite intances in total. That's kind of a standard scene in my game. I have not yet tried using the rendering server.

I don't see anything that would cause a GPU performance regression. The only change was moving the specular_shininess from the instance buffer to the push constant.
The two most significant changes in that PR, affecting CPU, were to use the LRUCache for uniform sets and to cache the TextureInfo for a given TextureState when calling canvas_render_items.
Caching TextureInfo
TextureInfo contains the texture state required for the 2D shader, including the diffuse, specular and normal textures, the sampler the calculated specular shininess, etc:
https://github.com/godotengine/godot/blob/2b68c63a886f1442f7afdb91f6bae16e755325cc/servers/rendering/renderer_rd/renderer_canvas_render_rd.h#L437-L446
When building batches, in dev3, we would have to reconstruct this each time the texture state changed, by calling _prepare_batch_texture_info. One of the calls made is to TextureStorage to fetch the info:
https://github.com/godotengine/godot/blob/1c6aad660f73c8f08d3138c936e17b8393689dc4/servers/rendering/renderer_rd/renderer_canvas_render_rd.cpp#L3164-L3170
there is a fair bit going on in here, but notably it performs a hash table looking here:

@ptkstefano could you try setting the uniform_set_cache_size to 16384 or larger?

@Ptkstefano could you try setting the uniform_set_cache_size to 16384 or larger?

I tested it really quickly right now and the problem is still there. From 100fps to 50fps in the same scene between dev3 and dev5. I think that there there was no impact at all with the cache size changes, but I'm not 100% sure. I've noticed that the time for my scene load is a bit longer in dev5 also. My best guess is that it's something to do with my processing rather than the graphics rendering, but that is just a hunch.

I don't see anything that would cause a GPU performance regression. The only change was moving the specular_shininess from the instance buffer to the push constant.
The two most significant changes in that PR, affecting CPU, were to use the LRUCache for uniform sets and to cache the TextureInfo for a given TextureState when calling canvas_render_items.
Caching TextureInfo
TextureInfo contains the texture state required for the 2D shader, including the diffuse, specular and normal textures, the sampler the calculated specular shininess, etc:
https://github.com/godotengine/godot/blob/2b68c63a886f1442f7afdb91f6bae16e755325cc/servers/rendering/renderer_rd/renderer_canvas_render_rd.h#L437-L446
dev 3
When building batches, in dev3, we would have to reconstruct this each time the texture state changed, by calling _prepare_batch_texture_info. One of the calls made is to TextureStorage to fetch the info:
https://github.com/godotengine/godot/blob/1c6aad660f73c8f08d3138c936e17b8393689dc4/servers/rendering/renderer_rd/renderer_canvas_render_rd.cpp#L3164-L3170
there is a fair bit going on in here, but notably it performs a hash table lookup here:
https://github.com/godotengine/godot/blob/c7198dda64b2495c9de56e4a4aa1bc3c1f5d16f8/servers/rendering/renderer_rd/storage_rd/texture_storage.cpp#L653
and then returns a fairly large struct that would likely be pushed on the stack:
https://github.com/godotengine/godot/blob/c7198dda64b2495c9de56e4a4aa1bc3c1f5d16f8/servers/rendering/renderer_rd/storage_rd/texture_storage.cpp#L725-L733
then finally, back in _prepare_batch_texture_info, we build the TextureInfo struct, which involves a lot of conditional logic:
https://github.com/godotengine/godot/blob/1c6aad660f73c8f08d3138c936e17b8393689dc4/servers/rendering/renderer_rd/renderer_canvas_render_rd.cpp#L3183-L3198
dev 4
The change we made to remove all this work was to cache the TextureInfo in a local HashMap:
https://github.com/godotengine/godot/blob/2b68c63a886f1442f7afdb91f6bae16e755325cc/servers/rendering/renderer_rd/renderer_canvas_render_rd.h#L514
so that for the current call to canvas_render_items, we only have to call _prepare_batch_texture_info once:
https://github.com/godotengine/godot/blob/1c6aad660f73c8f08d3138c936e17b8393689dc4/servers/rendering/renderer_rd/renderer_canvas_render_rd.cpp#L2361-L2365
We do a hash lookup here, but this is no more than previous.
Conclusion
We are clearly doing less work with this change, so I don't believe this is causing the drop in FPS
UniformSetCache → LRUCache
The second most significant change was to switch from the UniformSetCache to the LRUCache.

[!NOTE]
This is the change that introduces the uniform_set_cache_size property, to adjust how many uniform sets were cached. The uniform set contains the texture state (diffuse, normal, specular) and instance data buffer

We made this change, as there was a notable amount of time spent looking up items in the UniformSetCache, due to how uniform sets are hashed:
https://github.com/godotengine/godot/blob/5a98845655900a7884bf5a73c5406318f1106d77/servers/rendering/renderer_rd/uniform_set_cache_rd.h#L161-L163
and _hash_args performs a hash for each Uniform:
https://github.com/godotengine/godot/blob/5a98845655900a7884bf5a73c5406318f1106d77/servers/rendering/renderer_rd/uniform_set_cache_rd.h#L61-L67
Our UniformSet has 5 uniforms:
https://github.com/godotengine/godot/blob/1c6aad660f73c8f08d3138c936e17b8393689dc4/servers/rendering/renderer_rd/renderer_canvas_render_rd.cpp#L2914-L2918
The hash function requires:

hash_murmur3_one_64
hash_murmur3_one_32
For each uniform (5 times)
hash_murmur3_one_32
hash_murmur3_one_32
hash_murmur3_one_64
hash_fmix32

The summary of hash functions being:

6 x hash_murmur3_one_64
11 x hash_murmur3_one_32
1 x hash_fmix32

The dev 4 change stores the cached uniform sets in the LRUCache:
https://github.com/godotengine/godot/blob/2b68c63a886f1442f7afdb91f6bae16e755325cc/servers/rendering/renderer_rd/renderer_canvas_render_rd.h#L479
keyed by the RIDSetKey. The hash for this is considerably less complex:
https://github.com/godotengine/godot/blob/2b68c63a886f1442f7afdb91f6bae16e755325cc/servers/rendering/renderer_rd/renderer_canvas_render_rd.h#L470-L472
and the TextureState::hash:
https://github.com/godotengine/godot/blob/2b68c63a886f1442f7afdb91f6bae16e755325cc/servers/rendering/renderer_rd/renderer_canvas_render_rd.h#L432-L433
Resulting in the following hash functions:

2 x hash_murmur3_one_64
1 x hash_murmur3_one_32
1 x hash_fmix32

The compare functions of the UniformSet is also considerably more complex than RIDSetKey.
If the UniformSet is not found, then it is created, which is the same cost for dev 3 and dev 4.
LRUCache::getptr
When performing a lookup in the LRUCache, if the item is found, the LRU list is updated:
https://github.com/godotengine/godot/blob/a8931f1aa9c9c388947f13c6d1a765dd11dc2b44/core/templates/lru.h#L128-L136
this moves the element back to the front of the LRU list, which involves updating a few pointers for the linked list:
https://github.com/godotengine/godot/blob/a47daa0a44900a4246cc8f71041593590c86a711/core/templates/list.h#L593
I don't believe this would result in a significant CPU increase, given the reductions we've made elsewhere.
Cache eviction
When an item is inserted into the LRUCache:
https://github.com/godotengine/godot/blob/1c6aad660f73c8f08d3138c936e17b8393689dc4/servers/rendering/renderer_rd/renderer_canvas_render_rd.cpp#L2923
it checks if the cache has hit the max size, and will then evict items:
https://github.com/godotengine/godot/blob/a8931f1aa9c9c388947f13c6d1a765dd11dc2b44/core/templates/lru.h#L76-L84

[!IMPORTANT]
The UniformSetCache never evicts items, so this is a change. If the value of uniform_set_cache_size is not large enough, items will be evicted and force them to be recreated, which would cause a regression in performance.

Other differences
The UniformSetCache uses a PagedAllocator to allocate cache entries:
https://github.com/godotengine/godot/blob/5a98845655900a7884bf5a73c5406318f1106d77/servers/rendering/renderer_rd/uniform_set_cache_rd.h#L52
Could this account for a difference?

@produno could you set the uniform_set_cache_size to a very large number, like 30000 or more. I'm trying to determine if you are seeing some thrashing.

I have tried with it set up to 50k and it doesn't move my fps above 300, which is the same as when i set it to 512.

By removing canvas_item_add_texture_rect, were you substituting those draws with something else, or are you rendering fewer objects?

Handling fewer objects. I tried removing stuff to see what caused the most effect on fps between versions.
On a new game where there is not many objects i have roughly the same fps between versions:

If I fill the space with walls, which uses regions from a sprite sheet via canvas_item_add_texture_rect_region, the fps stays roughly the same between versions:

Its only when there are more objects added via canvas_item_add_texture_rect or lights when the fps drastically changes between the two (Like the image in my previous comment).

@stuartcarnie I'm starting to wonder if this is just a worst-case scenario for batching. IIRC that benchmark project I shared had the same issue, CPU performance was decreased in the case where we had thousands of sprites that could not be batched together. Although, that doesn't explain why the FPS changes between dev3 and dev4.
@produno would you be willing to share the project privately with Stuart or I? We would be able to get to the root of this very quickly if you could.

I shall try get something arranged tomorrow.

Although, that doesn't explain why the FPS changes between dev3 and dev4.

My thoughts exactly – a couple of fixes were introduced, and in particular one for the lights, but it doesn't seem enough. The changes I walk through above should only result in CPU improvements and shouldn't affect GPU – I don't see how those could result in worse perf, as in both instances it is doing less work. The lookup in the UniformSetCache is a little simpler, but it wasn't showing up in profiles.

@produno do you notice any measurable CPU usage difference between dev 3 and dev 4?

@stuartcarnie I'm starting to wonder if this is just a worst-case scenario for batching. IIRC that benchmark project I shared had the same issue, CPU performance was decreased in the case where we had thousands of sprites that could not be batched together. Although, that doesn't explain why the FPS changes between dev3 and dev4.
@produno would you be willing to share the project privately with Stuart or I? We would be able to get to the root of this very quickly if you could.

My project had a substantial performance boost going from 4.3 to 4.4, but I'm also running into the performance problem going from dev 3 to the next ones, so it could very well be related to batching I think 🤔

@produno do you notice any measurable CPU usage difference between dev 3 and dev 4?

No, both are the same at around 8%~ with the game open and running.

OP shared their project with me and I have now tested it on Windows, Linux, and MacOS. On MacOS I can't measure a performance difference (there was a 1% increase actually, but that is within a margin of Error). On Linux I tried two combinations:

Integrated graphics (this was GPU bound, so no difference)
Dedicated GPU (no difference)

On Windows I can see a distinct increase in the amount of time it takes to render each PointLight2D on the CPU.
Timing are roughly as follows:
dev4: 175 FPS (5.7 mspf)
dev3: 200 FPS (5.0 mspf)
Coincidentally, the increase in point light 2D rendering time in the Visual Profiler is 0.7 ms.
What makes this so strange is that the function to update a PointLight2D hasn't changed. I also checked and the number of PointLight2D's being processed per frame is the same, and the number of total occluders is the same.
Since the issue is only noticeable in the RD backend, it points to the regression being the RD API. I suspect that it might be due to the thread guards introduced in https://github.com/godotengine/godot/pull/90400 (CC @DarioSamo). On MacOS I can see them taking quite a bit of time, but the cost is offset by the time that they save. On Windows, they may be relatively more expensive. I need to profile more deeply to find out.

Thread guards are in theory not expensive as they're merely a check for the thread's current ID to match, although it is unknown to me if Godot does anything else that might result in an expensive OS call there.
What is more likely is that there's an extra hashing cost associated to finding a pipeline now. I've discussed this with others before, but if we find a significant chunk of the time being eaten by the hashing, we should really just try switching to an algorithm like XXH3, which we already got in the codebase as a thirdparty dependency of something else. Murmur3 is significantly slower and we should avoid using it.

What is more likely is that there's an extra hashing cost associated to finding a pipeline now. I've discussed this with others before, but if we find a significant chunk of the time being eaten by the hashing, we should really just try switching to an algorithm like XXH3, which we already got in the codebase as a thirdparty dependency of something else. Murmur3 is significantly slower and we should avoid using it.

I don't think so as the code that slowed down is in light_update_shadow which doesn't use the same code as canvas rendering. It pulls the pipeline from a flat array with 3 elements depending on the cull mode. So there is no hashing needed to find the pipeline.

I came back to this today with 2 changes to my approach:

I re-profiled everything at 1024x768 resolution with gameplay paused and with no zoom (previously I profiled zoomed in all the way at the smallest res I could get to rule out a GPU bottleneck)
I bought Superluminal and used it to capture traces

Superluminal showed that the only performance difference was in the _render_batch() function. Further it showed that the cost was mostly from creating uniform sets and inserting them into the cache. This is a strong indicator that the uniform set cache was being thrashed. Accordingly, I increased the rendering/2d/batching/uniform_set_cache_size to 16384 and the performance went away
My results were as follows:
dev3: 100-105 FPS
dev4: 90 FPS
dev4 (with large uniform set cache): 103 FPS
Conclusion: I think this is just uniform set thrashing, and indicates that we should increase the default size. Since we basically need at least one uniform set for each unique texture, I think something in the range of a few thousand would be a better default.

My last post
I think the visual profiler showing an increase in PointLight2D cost was a red herring. The visual profiler is not super accurate and profiling tools should be given more weight.
Further, I was testing with an extremely small window size which had the benefit of culling out most CanvasItems, so shadows ended up being the primary cost.
The experience has shown me that we have a lot of low hanging fruit we can do to optimize shadow rendering.

My thoughts on produno's bisect
I think the bisect correctly pinpointed the regression. Additionally, the performance difference remaining makes some sense since it might not have been using a build with all optimizations enabled.

Adjusting uniform_set_cache_size for me does not solve the issue.
Running the game maximised on a 2560x1440 monitor, paused with the camera fully zoomed out is as follows:
Dev 3 = 390fps~
Dev 4 (uniform_set_cache_size to 512+) = 300fps~
Dev 4 (uniform_set_cache_size set to default) = 256fps~
Note that changing that setting above 512 makes no difference to the fps for me. Incidentally whilst bisecting and compiling the engine with no optimisations with the uniform_set_cache_size at default, I was getting 335fps, which is still more than adjusting the cache size in dev 4.
Whilst it does seem to make a difference to the fps, as can be seen above, there definitely seems to be something else affecting the performance. I still get almost 25% better performance in dev 3 compared to dev 4 with the cache set to 512+, which is quite a large performance gap.

After further testing, setting `uniform_set_cache_size' to 512 and setting light shadows to false will give me roughly the same fps between both builds. So this definitely seems to be pointing towards an issue with shadows.
With shadows set to true, I get the results posted above.

I mentioned this to produno on Discord already. But I think the last bit of performance difference on their hardware might be down to the cost of reading the thread guards. The cost was visible on both my Windows machine and my Macbook, but it was outweighed by other improvements (most likely by not using mutexes anymore). its possible that on produno's hardware the cost of the thread guard is higher relatively and so the cost is more visible.
The cost is likely to manifest in point light shadow rendering for two reasons:

It uses a lot of RD API calls, (many of which didn't have mutexes previously)
This scene renders a lot of shadows (around 20,000) so any increase in cost to rendering shadows will manifest itself quite plainly.

Accordingly, I think the way to move forward with this issue is to batch together API calls when rendering shadows to reduce the overhead.

Another notable change in dev4 was the introduction of detecting intersections between draw lists in https://github.com/godotengine/godot/pull/98701. I have made a synthetic benchmark with just a bunch of PointLight2Ds and occluders and _check_command_intersection() is showing up pretty visibly in the profiler.
Since the changes in _check_command_intersection() are very valuable elsewhere, I think this highlights again the need to reduce the number of draw lists that shadows create. We are unnecessarily creating 4 draw lists per light. So when you have a lot of lights, the cost really adds up

I had already sent this to Clayjohn but just for completeness. Testing on other hardware is as follows:
dedicated NVIDIA GeForce GTX 1080 Ti (NVIDIA; <IP_ADDRESS>14) - Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz (12 threads)
Dev4 uniform_set_cache_size 512 = 126fps
Dev3 = 172fps
Could be specific to NVIDIA or Intel?

@produno I ahve made a PR that I think should fully resolve your remaining performance regression! https://github.com/godotengine/godot/pull/100302
Please test it if you can and confirm

@produno I ahve made a PR that I think should fully resolve your remaining performance regression! #100302
Please test it if you can and confirm

Awesome, nice work! I am not entirely sure how to compile with optimisations so its on par with the current dev builds, but previously when I was bisecting I reached 335 fps. Now I am getting 345fps, so that seems like a win to me.
I will know for sure once dev 7 is released and I can directly compare.

I have tested this on Dev7 and weirdly i seem to get lower FPS than when i compiled your own branch @clayjohn. Below are what i currently get with the same scene i have been using from the start.
Dev 3: ~390fps
Dev 4: ~260fps
Dev 7: ~295
Meaning i currently get the same in Dev 7 as i did in Dev 4 with uniform set cache size at 512. (I currently have it set at the new default 4096)
I also pulled from master and recompiled a couple of days ago and there i get ~355fps. That is currently 20 pull requests behind as of me typing this.

Well, that is very strange, Its kind of hard to believe actually. There have been no changes that would impact performance between https://github.com/godotengine/godot/pull/100302 being merged and dev 7.
Where did you download dev 7 from by the way? It hasn't officially released yet

From the Godot builds repo. Well, I am just as confused as you are. Is there some difference in the way its being compiled? Could something else have gone wrong on my end? You can see below the compiled version of master i used and then currently in Dev 7.

From the Godot builds repo. Well, I am just as confused as you are. Is there some difference in the way its being compiled?

There, but I don't expect it to cause a significant performance regression, normally it trends in the other direction. Dev 7 is the first snapshot where Windows builds are made on Fedora 41 instead of Fedora 40, so using MinGW 12.0.0 / GCC 14.2.1 instead of MinGW 11.0.1 / GCC 14.1.1.
I could make a test build of dev 7 with the previous build images for comparison.

Could something else have gone wrong on my end? You can see below the compiled version of master i used and then currently in Dev 7.

Can you try to compile the commit of dev 7 the same way you did master previously, to compare with similar toolchains and optimizations?
It may just be that the way you do your own master builds is somehow better optimized for this specific code path than what official toolchains and build options achieve.

Can you try to compile the commit of dev 7 the same way you did master previously, to compare with similar toolchains and optimizations?

Yep no worries, ill give this a go tomorrow.

Here is the result of compiling myself just using scons platform=windows.

I've done some testing myself and saw that with dev 7 and vsync enabled, the FPS would start to become unstable/inconsistent with it sometimes being at over 160+ FPS. Restarting the scene would cause it to get halved, But alt-tabbing would weirdly cause the FPS to increase to normal? VSync being disabled also keeps it from getting halved.


Fix resizing

Uses a resizing method less prone to aliasing. You can read about in this blogpost here: https://chadrick-kwag.net/cv2-resize-interpolation-methods/

thanks


add Answer page

This PR will add code for the design of the Answers page and also the 404 page
In this PR, 421 additions 169 deletions lines of code

some files contain only 5 to 6 lines of code
some files are of Images, so you can avoid them

Please review this PR and give your feedback

Live URL

Thank you for reviewing,


[shared_preferences]Shared preferences: add default value capability

fix https://github.com/flutter/flutter/issues/22806

pub global run flutter_plugin_tools format --plugins shared_preferences give me this error on my Mac:
Downloading Google Java Format...
Formatting all .dart files...
Unhandled exception:
ProcessException: No such file or directory
  Command: flutter format /Users/jaumard/StudioProjects/plugins/packages/shared_preferences/test/shared_preferences_test.dart /Users/jaumard/StudioProjects/plugins/packages/shared_preferences/example/lib/main.dart /Users/jaumard/StudioProjects/plugins/packages/shared_preferences/lib/shared_preferences.dart
#0      runAndStream (package:flutter_plugin_tools/src/common.dart:171:7)
<asynchronous suspension>
#1      FormatCommand._formatDart (package:flutter_plugin_tools/src/format_command.dart:98:11)
<asynchronous suspension>
#2      FormatCommand.run (package:flutter_plugin_tools/src/format_command.dart:39:11)
<asynchronous suspension>
#3      CommandRunner.runCommand (package:args/command_runner.dart:194:27)
<asynchronous suspension>
#4      CommandRunner.run.<anonymous closure> (package:args/command_runner.dart:109:29)
#5      new Future.sync (dart:async/future.dart:224:31)
#6      CommandRunner.run (package:args/command_runner.dart:109:11)
#7      main (package:flutter_plugin_tools/src/main.dart:41:17)
#8      _startIsolate.<anonymous closure> (dart:isolate/runtime/libisolate_patch.dart:287:32)
#9      _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)

what am I doing wrong ?

All good :) I formatted file manually as pub global run flutter_plugin_tools doesn't work for me

Can this be merged?

@mclark4386 Can this be merged now? This is a useful addition to shared preferences plugin, I'd love to use this in my project when it's released. Thanks in advance.

@acherkashyn It's fine with me, but I don't have merge permissions^^;

@collinjackson @cyanglaz Could you please take a look at this PR? It's been open since October, everything is already implemented and ready to merge. This is useful feature that makes the API more developer friendly.

Thank you for providing the PR.
I don't think this is really necessary. People can just use ?? to have the same behavior. E.g.
Bool myBool = getBool('myKey') ?? false;

Closing the PR now.

I think even if you can use ?? it's still nice to have a clean API that offer you clear way of doing things :)

I agree that it's always nice to have a clean API. However, in this case, I don't see the benefit of passing an object in the parameter and to let the method return the same object as default. I understand that you might not want to have ?? all of the places in your code. In that case, you can create your own helper methods to wrap around the original methods that provided in the plugin. Will that help?

I know I can have my own wrapper but when I did the PR I considered that a lot of people will have to do that wrapper, so why not have it integrated to the plugin directly, even more when the android native sharedPreferences have those default values. And I don't see the point of create a third party package just for that =D
My point is just about readability getString('myKey', defaultValue: 'ok'); is more readable than just getBool('myKey') ?? false; even more when you have new people / junior on the team.
If you consider that people should do their wrapper or use ?? fine by me :)


Implement parameters parser for crypto service

Let me know when your code is in a good place to merge.


[QUESTION] iterating with ElasticSearch?

Hi all!
I've stumbled upon dedupe.io, and I find the library pretty amazing, it's exactly what I was looking for!
I can see in the examples there is one describing how to use MySQL to free the memory and don't process millions of records directly in-memory, and that's what I'm aiming for.
I can see that all the processing stuff is done with the CURSORS of mysql package... but now I have an issue: I don't have a SQL database in my structure, and all my profiles right now are stored in an ElasticSearch database.
I don't want to keep a MySQL instance in-sync with ES, so I was asking myself if everything can be done with ES, and its SCROLL functionality instead of cursors... Is it viable? Will the execution be equally efficient?
thank you,
Giacomo.

good question, but not really relevant to this repo. If you figure this out, would be nice to see a PR with an example


Orientation Change Bug

When  maiViewController (Child of KWDrawer) is rotate when viewDidLoad() of maiViewController is call again?

I don't know what you mean.


Fixing font-family

Consider changing the font-family to sans-serif, for purely aesthetic reasons.

Nevermind, already done

Actually, font-family set for paragraphs on demo page, but not on popup itself. Think it looks much better with sans-serif, will send PR


chore: add Jintao Zhang to Committer

Signed-off-by: Jintao Zhang<EMAIL_ADDRESS>Changes:

add Jintao Zhang to Committer.

Merged, congratulations!

Thanks!


2022.9.x feature record revisions

Please take a look.
Only last 2 commits are related to revisions. Will move them out of the branch when finished.

Codecov Report

:exclamation: No coverage uploaded for pull request base (2022.9.x@aae7152). Click here to learn what that means.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             2022.9.x     #394   +/-   ##
===========================================
  Coverage            ?   38.76%
===========================================
  Files               ?      803
  Lines               ?    80987
  Branches            ?        0
===========================================
  Hits                ?    31396
  Misses              ?    45096
  Partials            ?     4495

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aae7152...8e6bcac. Read the comment docs.


Unable to build due to: cannot find type 'MLTensor' in Scope

The error came from the source package: "swift-transformers", updated to the latest preview branch.

main branch works

Yes @xavi0007, the preview branch is meant to be used with macOS 15, which should be released soon.


Please add Max Model Size limit on the table showing the limitations of each SKU/Capacity.

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 18013a6f-b02d-30d4-3b6d-93295624f352
Version Independent ID: e307e1d7-e565-8742-e563-86d596d7d1e1
Content: What is Power BI Premium? - Power BI
Content Source: powerbi-docs/enterprise/service-premium-what-is.md
Service: powerbi
Sub-service: powerbi-premium
GitHub Login: @KesemSharabi
Microsoft Alias: kesharab

Thanks for your comment, @kr77025. This issue has been assigned to the doc author for review.


Use generator expressions in performance.py and add tests

Currently the mspray.apps.main.views.performance module does, among other things, calculations such that look like:
"not_sprayed_total": sum(
                [i["not_sprayed_total"] for i in serializer.data]
            )

See https://github.com/onaio/mspray/blob/master/mspray/apps/main/views/performance.py#L118 for an example.
We can make this a little better by using generator expressions instead of list comprehensions in order to get the sum(s).
Additionally, once this is done we should add tests for those parts of the code that we change.

Fixed by #488


Update CODEOWNERS to Amplify UI team

Description of changes
This PR updates the CodeOwners to Amplify UI team

Issue #, if available

Description of how you validated changes
Checklist

[ ] PR description included
[ ] yarn test passes
[ ] Tests are [changed or added]
[ ] Relevant documentation is changed or added (and PR referenced)

Codecov Report

Merging #227 (f770525) into main (cc18f8c) will not change coverage.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@           Coverage Diff           @@
##             main     #227   +/-   ##
=======================================
  Coverage   43.99%   43.99%
=======================================
  Files          16       16
  Lines        1273     1273
  Branches      172      172
=======================================
  Hits          560      560
  Misses        713      713

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Add instance Semigroupoid Fold

Fixes #110

Oh, looking at folds now i notice two things: We might want a strict pair as the state, and I forgot an INLINE o. What do you think regarding strict/non-strict pair?

@lspitzner: Yeah, this should use a strict pair.  It's probably not necessary in newer versions of GHC where the strictness analyzer has gotten much better, but I still use the strict pair as a precaution

What does this instance do, semantically?
> Control.Foldl.fold (Control.Foldl.mconcat &>> Control.Foldl.mconcat) [[1],[10],[100],[1000]]
[1,1,10,1,10,100,1,10,100,1000]

This is quite a senseless result, I cannot think of a situation where this is useful? Is this the only lawful instance and if yes, what operation does it represent?

I don't know what the &>> operator in your example is from or what it is doing


Refactor Ingestion/Processing code

Refactor the python (God) script

break it down into multiple python modules
write unit tests
Use YAML with JINJA template as substitute for construction of the JSON files (CSV to JSON conversion)

This issue will be addressed in cQube v5.0.

This is implemented


Admin panel issue on production

Package
filament/filament
Package Version
v3.2
Laravel Version
v11.9
Livewire Version
No response
PHP Version
8.3
Problem description
not able to step up production-ready site , admin is not working
Expected behavior
after login 403, and once the user model is updated still issue persist with 500 error
Steps to reproduce
fresh setup laravel and filament , create user and try login
Reproduction repository (issue will be closed if this is not valid)
na
Relevant log output
No response

Donate 💰 to fund this issue

You can donate funding to this issue. We receive the money once the issue is completed & confirmed by you.
100% of the funding will be distributed between the Filament core team to run all aspects of the project.
Thank you in advance for helping us make maintenance sustainable!

https://task.devdata.uz/admin
<EMAIL_ADDRESS>:  12345678


Mobile First Support for top panel

Hi
Can you please add some css for top panel, so it stacks on lover resolutions ?

Hi @mmizera! Apologies for a very-very-very late reply! This should be fixed in the next major release (v3). Sample demo can be seen here: https://codesandbox.io/s/react-bs-datatable-3-zq7xe

I'll try to backport it to v2 when I have the  chance. Let me know if you have other concerns!


AttributeError partially initialized module 'sofar' has no attribute 'list_conventions'

I created a .py file to test this module:
sofar.py:
import sofar as sf
sf.list_conventions()

Results:
python3 ./sofar.py
Traceback (most recent call last):
  File "/Volumes/Storage 11 - Seagate Backup Plus Drive 10/Tutorials/Binaural Audio/sofar/./sofar.py", line 1, in <module>
	import sofar as sf
  File "/Volumes/Storage 11 - Seagate Backup Plus Drive 10/Tutorials/Binaural Audio/sofar/sofar.py", line 2, in <module>
	sf.list_conventions()
	^^^^^^^^^^^^^^^^^^^
	AttributeError: partially initialized module 'sofar' has no attribute 'list_conventions' (most likely due to a circular import)

I exported site-packages but that didn't help:
export PYTHONPATH="${PYTHONPATH}:/usr/local/lib/python3.11/site-packages"
I read at python - How to fix AttributeError: partially initialized module? - Stack Overflow:

"Most likely due to a circular import" refers to a file (module) which has a dependency on something else and is trying to be imported while it's already been imported.

'sofar.list_conventions' exists in 'sofa.py'.
So why is this a 'circular import' and what can be done about it?
Python 3.11.3 on OS X Ventura

Hm, this is odd. It's working for me with the same Python version and a fresh pip install sofar on Windows. I also added tests for 3.11 in #64 that finish without errors on Linux. How did you install sofar?

I tried pip install sofar and pip3 install sofar after uninstalling it.

thanks for the info @tluebeck @mberz do you have a couple of minutes to test on your Mac's?

I already tried that. I uninstalled the system version and installed it in a 'venv' with virtualenv 20.23.1. I got the exact same results.

Not sure how we can help if we cant replicate the issue...


Incorrect Shelly SSID "shelly3em"

Hi, i just found this git repo handling a Shelly Energy Meter by default from Venus OS.
I think this feature is not in the current release version. Therefore, i just pulled this repo and installed the driver according to the manual installation of a driver in Venus OS.
Since nothing happend i reviewed the code and found that the SSID of the three phase shelly "shelly3em-" in line 14 of mdsn.py is incorrect and a three phase Shelly connot be found.
Because the SSID of a Shelly cannot be changed i corrected the line 14 in mdsn.py to
NAMES = ('shellyem-', 'shellyem3-')
After starting the driver manually, another error occured.
Error : ValueError: Invalid bus name 'com.victronenergy.grid.MAC-Address': a digit may not follow '.' except in a unique name starting with ':'
pointing to line 91 in the code of meter.py
Simply exchaning
servicename = "com.victronenergy.{}.{}".format(role, name)
to "com.victronenergy.grid.ttyUSB0_di32_mb1"
of function VeDbusService in line 91 in meter.py
resolves this error and gives a messy solution.
Everything seems to work fine.

And I know that is driver is still under development and i don't want to appear super clever but i just wanted to give you this hint.

Thanks @LukasAusserhuber , the initial implementation was done with a Shelly-EM, and since the documentation says the hostname is always shelly<model> I made the initial assumption that once my test hardware shows up it would reflect the 3EM naming. It is already corrected, so I will close this.


Implement install

I'll work on this.

Have you made any progress on this?

I'm assuming that no progress has been made since 2014, and am now working on this.

closing for now


It's impossible to aquire ownership of incoming MyData

Since NetworkData does not implement Clone, it is impossible to acquire ownership of inner as one has only access through EventReader::iter() which hides e behind a reference. Thus, into_inner cannot be called.
fn handle(mut incoming: EventReader<NetworkData<MyData>>){
    for e in incoming.iter(){ // e:&NetworkData<MyData>
        // I want to retrieve e (and use it elsewhere)
        let data: MyData = e.into_inner(); // errors: cannot move
        let data: MyData = e.inner; // errors: inner is private
    }
}

I would propose making inner public.
edit: What I want is to clone inner :)

Then I seem to have missed something :), Could you provide an example?^.^*

Never mind
let data:MyData = (e as &MyData).clone();

does the trick :+1:

If you want to, you could implement AsRef for NetworkData instead of Deref (which is not a good impl in this case actually), as well as adding an inner method, which would allow you to borrow the data (and to which AsRef simply defers).


Improve link mechanism and order

Roadmap

[x] Support adjust and rewrite links order between add_deps, add_packages and add_links
[x] Support links group, -Wl,--start-group
[x] Support whole links, -Wl,--whole-archive
[x] Explicitly specify the linked file name

Related issues

#1240
#566
#1417
#1293
#1144
#1110
#1002
#605
#1650
#4234

Adjust the order of each configuration type
-- -la -ld -lb -lc -le -ls1 -ls2
add_syslinks("s1", "s2")
add_links("a")
add_links("b", "c", {order = 2})
add_links("d", {order = 1})
add_links("e")

-- -la -ld -lb -lc -le
add_ldflags("-la")
add_ldflags("-lb", "-lc", {order = 2})
add_ldflags("-ld", {order = 1})
add_ldflags("-le")

and add_includedirs, add_xxx etc, it supports all configurations
Adjust the link order between packages, deps, and options

Instead of repeating package, option, dependency using the "package::name" and setting the link order will it be better introduce "link_order" option for packages, options and dependencies?
Before,
add_links("package::e", {order = 1})
add_links("option::f", {order = 2})
add_links("dep::g")
add_packages("e")
add_options("f")
add_deps("g")

After,
add_packages("e", {link_order = 1})
add_options("f", {link_order = 2})
add_deps("g")

Instead of using a separate group flag, wouldn't it be better to use the "order" flag with two elements to denote the group? e.g.
add_syslinks("s1", "s2")
add_links("a")
add_links("b", "c", {order = 4})
add_links("d", {order = {5, 1}})
add_packages("e", {link_order = {1, 1}})
add_options("f", {link_order = {1, 2}})
add_deps("g", {link_order = 3, link_whole = true})
add_links("h", {order = {5, 2}})

This will end up with the command line,
-Wl,--start-group -le -lf -Wl,--end-group -Wl,--whole-archive -lg -Wl,--no-whole-archive -lb -lc -Wl,--start-group -ld -lh -Wl,--end-group -la -ls1 -ls2

Instead of repeating package, option, dependency using the "package::name" and setting the link order will it be better introduce "link_order" option for packages, options and dependencies?

I still want to consider other configurations in the support package, such as add_includedirs("package::xxx"). Maybe there will be ldflags, cflags in the future. .

Instead of using a separate group flag, wouldn't it be better to use the "order" flag with two elements to denote the group? e.g.

This is a bit too complicated, or the same order, we can put it in a group without using two elements.
add_links("a", "b", {order = 5})
add_links("c", {order = 5})

It's a bad idea using numbers to specify link orders. What matters is only the link order of some specific links and obviously a tree structure is more precise and robust.

It's a bad idea using numbers to specify link orders. What matters is only the link order of some specific links and obviously a tree structure is more precise and robust.

The main reason is that the order of links between add_package/add_options/add_links cannot be controlled. Are there any other better ideas?

@xq114 How to define the tree structure? should it be a single command like,
set_link_order("e", "f", {"a", "d", "c"}, {"deps::f", "options::b"}, "k")

This does not conflict with tree inheritance, I just intend to control the modification of the link order between options/packages/links in the current target, and support link grouping.
target("test")
    -- need not `dep::xxx`, we only adjust the link order between options/packages/links in the current target
    add_linkorders("a", "b", "package::c", "option::d")
    add_linkorders("a", "b", "package::c", "option::d", {group = true})

about whole link
target("test")
    add_deps("a", {wholelink = true}) -- only for dep("xxx"), -lxxx
    add_links("a", "b", "c", {wholelink = true})
    add_options("a", "b", "c", {wholelink = true})
    add_packages("a", "b", "c", {wholelink = true})

@xq114 How to define the tree structure? should it be a single command like,
set_link_order("e", "f", {"a", "d", "c"}, {"deps::f", "options::b"}, "k")

You're right, I mean, we can use a couple of add_linkorders command to specify the link dependency tree. For example, the following code will generate a tree a->b->c->d c->e b->f
add_linkorders("a", "package::b", "c", "option::d")
add_linkorders("c", "package::e")
add_linkorders("b", "package::f)

If I want to modify the structure by adding g->a, just a modification to the first line would work, without rearranging the numbers denoting the position of linking.
Furthermore, it's better if a graph structure is supported, i.e. a->b->d a->c->d. The actual link order could be either a->b->c->d or a->c->b->d depending on implementation.

Does it respect package add_deps() by default?
package("foo")
    add_deps("bar")

Will the link order will be?
-lbar -lfoo

Does it respect package add_deps() by default?
package("foo")
    add_deps("bar")

Will the link order will be?
-lfoor -lbar

yeah

Another solution comes from cmake and some other build systems: links can be treated as targets with dependency hierarchy. So the order of links can be derived from that.
add_link_deps("a", {"b", "c", "d"}) or add_link_deps("a", "b", "c", "d")
-- cmake: add_library(a IMPORTED); target_link_libraries(a b c d); target_link_libraries(<target> a)

Since dependency is specified, a simple topological sort can be applied to find the actual link order (and find circular dependencies). The implementation is similar to calculating the build order of targets.
For group links, another mechanism must be applied. General solution is
add_link_group or add_links(,{group=true})

a link group can be simplified as a node and topological sort still applies. A more tricky mechanism is to treat circular dependencies as link group (which means they require symbols from each other), e.g.
add_link_deps("a", "b")
add_link_deps("b", "a")
-- equal to add_links("a", "b", {group=true})

This is the solution of most build systems; if this kind of mechanism is chosen, a Tarjan's algorithm can be used to find the strong components for links. A simpler implementation is just to link these libraries multiple times: create a copy of the link every time it's required, and take all copies into account when calculating the link order, which reduced the usage of group link.
As for whole archive, it has nothing to do with link orders (in my opinion). We can just add whole archive flags for every links specified with {whole=true}, and remove all -Wl,--no-whole-archive -Wl,--whole-archive substrings from the final command.

I would also love the ability to link a single library statically -- this can be achieved on clang/GCC using -Wl,-Bstatic -lsome_static_lib -Wl,-Bdynamic -lmy_other_lib -lmy_other_lib_2. :)
Maybe a syntax like add_links("some_static_lib", { static = true })?

I would also love the ability to link a single library statically -- this can be achieved on clang/GCC using -Wl,-Bstatic -lsome_static_lib -Wl,-Bdynamic -lmy_other_lib -lmy_other_lib_2. :)
Maybe a syntax like add_links("some_static_lib", { static = true })?

I will consider it

Now I have improved it. https://github.com/xmake-io/xmake/pull/4250
link group
--  -Wl,--start-group -la -lb -Wl,--end-group
add_linkgroups("a", "b")

link group (whole archive)
--  -Wl,--whole-archive -la -lb -Wl,--no-whole-archive
add_linkgroups("a", "b", {whole = true})

modify links orders
It will create graph and use topological sort to sort links.
and it will check link cycle in add_linkorders() and show warnings.
add_links("a", "b", "c", "d", "e")
-- e -> b -> a
add_linkorders("e", "b", "a")
-- e -> d
add_linkorders("e", "d")

sort links and linkgroups
add_links("a", "b", "c", "d", "e")
add_linkgroups("c", "d", {name = "foo"})
add_linkorders("e", "linkgroup::foo")

sort links and frameworks
add_links("a", "b", "c", "d", "e")
add_frameworks("Foundation", "CoreFoundation")
add_linkorders("e", "framework::CoreFoundation")

examples
add_rules("mode.debug", "mode.release")

add_requires("libpng")

target("bar")
    set_kind("shared")
    add_files("src/foo.cpp")
    add_linkgroups("m", "pthread", {whole = true})

target("foo")
    set_kind("static")
    add_files("src/foo.cpp")
    add_packages("libpng", {public = true})

target("demo")
    set_kind("binary")
    add_deps("foo")
    add_files("src/main.cpp")
    if is_plat("linux", "macosx") then
        add_syslinks("pthread", "m", "dl")
    end
    if is_plat("macosx") then
        add_frameworks("Foundation", "CoreFoundation")
    end
    add_linkorders("framework::Foundation", "png16", "foo")
    add_linkorders("dl", "linkgroup::syslib")
    add_linkgroups("m", "pthread", {name = "syslib"})

add_linkgroups("a", "b", {whole = true})如果同时需要-Wl,--start-group ，就有点歧义的感觉了，add_links("a", "b", {whole = true，group = true})或者 增加add_linkwhole("a", "b")？

add_linkgroups("a", "b", {whole = true})如果同时需要-Wl,--start-group ，就有点歧义的感觉了，add_links("a", "b", {whole = true，group = true})或者 增加add_linkwhole("a", "b")？

I improved it.
add_linkgroups("a", "b", {group = true})
add_linkgroups("a", "b", {whole = true})
add_linkgroups("a", "b", {group = true, whole = true})

I would also love the ability to link a single library statically -- this can be achieved on clang/GCC using -Wl,-Bstatic -lsome_static_lib -Wl,-Bdynamic -lmy_other_lib -lmy_other_lib_2. :)
Maybe a syntax like add_links("some_static_lib", { static = true })?

we can use add_linkgroups("a", "b", {static = true}) now.

xmake update -s dev

在 package 中，报这个错误：
[root@70f6b8a8951d xmake-repo]# xmake l scripts/test.lua --shallow -vD spdk
error: ./packages/s/spdk/xmake.lua:35: global 'add_linkgroups' is not callable (a nil value)
stack traceback:
    [./packages/s/spdk/xmake.lua:35]: in main chunk

在 package 中，报这个错误：
[root@70f6b8a8951d xmake-repo]# xmake l scripts/test.lua --shallow -vD spdk
error: ./packages/s/spdk/xmake.lua:35: global 'add_linkgroups' is not callable (a nil value)
stack traceback:
    [./packages/s/spdk/xmake.lua:35]: in main chunk

暂不支持 package ，仅用于 target 下调整

在 package 中，报这个错误：
[root@70f6b8a8951d xmake-repo]# xmake l scripts/test.lua --shallow -vD spdk
error: ./packages/s/spdk/xmake.lua:35: global 'add_linkgroups' is not callable (a nil value)
stack traceback:
    [./packages/s/spdk/xmake.lua:35]: in main chunk

暂不支持 package ，仅用于 target 下调整

好的，请问这个有计划支持吗？期待一下。

在 package 中，报这个错误：
[root@70f6b8a8951d xmake-repo]# xmake l scripts/test.lua --shallow -vD spdk
error: ./packages/s/spdk/xmake.lua:35: global 'add_linkgroups' is not callable (a nil value)
stack traceback:
    [./packages/s/spdk/xmake.lua:35]: in main chunk

暂不支持 package ，仅用于 target 下调整

好的，请问这个有计划支持吗？期待一下。

I have supported it. https://github.com/xmake-io/xmake/pull/4314
we can also use add_linkorders to sort package links and add linkgroups/wholearchives.
package("libpng")
    add_linkorders("png16", "png", "linkgroup::foo")
    add_linkgroups("dl", {name = "foo", group = true})


[Registry Preview] Two values with the same name.

Microsoft PowerToys version
0.70
Installation method
GitHub
Running as admin
Yes
Area(s) with issue?
Registry Preview
Steps to reproduce
Create a registry entries file with the following content:
Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\.test]
"test"=dword:00000000
"test"=dword:00000000

Select the .test subkey in the right pane.
✔ Expected Behavior
A single line should appear with the name test.
❌ Actual Behavior
Two lines with the same content appear. This should not happen, since having two same names in a single key is impossible and after running the file, only one will be added (of course).

Other Software
No response

Reminds me of similar ask in the past, to support multiple Keys of the same path/name in the same file: the Editor may be able to rationalize it, because it's actually importing the values from top to bottom, but for a Preview of a file that's "presumes" that the REG file was exported from the Editor, making this type of duplicate entry an impossible scenario?  Definitely a bug a low bug on the stack, IMHO.
A reoccurring theme for me is that this was built for a Preview of the file, rather than an authoring tool.  If I ignore that lingering opinion, I keep thinking that if I wanted to make it an authoring tool, I would have designed it differently.  Right now, to check for existing values, I have to traverse the tree, then each of the values: small task for a small file but as the file grows, having to recheck the whole tree and then the attached table with each new line it becomes... painful?  That aside, then the next question is the recommended behavior: last one read wins?  Show an error for a single entry, calling out the issue?  Still show two, with the previous versions as "error" and keep the last one?
It's an interesting error.


[RFC][base-ui][Switch] The input component should be opt-in

What's the problem?
The docs and demos related to the Switch component include the hidden checkbox input. This is not a common pattern when building Switch components, and it might be confusing.
What are the requirements?
Simplify the switch component and hook for the base use case.
What are our options?

Keep it as is
Make the input opt-out
Make the input opt-in

Proposed solution
The input component should be opt-in, meaning:

the useSwitch hook should work without it. The input component is helpful for form submission so that I wouldn't remove it, but it's not a common enough use case to have it by default.
The Switch component shouldn't include the input by default
The docs and demos shouldn't include the input. There should be a separate demo for adding the input for the specified use case

Resources and benchmarks

https://www.w3.org/WAI/ARIA/apg/patterns/switch/
https://www.radix-ui.com/primitives/docs/components/switch
https://react-spectrum.adobe.com/react-aria/Switch.html

Search keywords:

hey ! i would like to contribute to this but i am unable to reproduce and check on my local system to verify the changes i have made . Can you guide me on this. Thanks!

Hi @Nikhilh26, thanks for the interest. This is an RFC without a decision, so the following steps are unclear. In other words, the issue is not ready to be worked on. I recommend looking for issues with the ready to take label instead.

We used to have a similar discussion around the Select. At first, the hidden input was rendered only when the name prop was present (to participate in form submission), but we ended up needing it for more than this (built-in browser form validation, autofill), so now it's always rendered.
I think the Switch should behave the same way. The checkbox doesn't necessarily have to be styled, but IMO it should be present.

Fair enough. I did a proper benchmark, and other unstyled libraries have it as well:

In Radix the input is added only when the switch is inside a form. The switch role is on the button: https://www.radix-ui.com/primitives/docs/components/switch#root
In React Aria, it seems it is always added. The switch role is on the input: https://react-spectrum.adobe.com/react-aria/Switch.html#example

In any case we're missing the switch role, but that's the topic for https://github.com/mui/material-ui/issues/40615 so I'm closing this.


root: 6.28.10 -> 6.30.02

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

cc @ShamrockLee

Add a :+1: reaction to pull requests you find important.

Congratulations for making it build!
As for the TMVA change, I originally planned to learn TMVA the time I switched the CMake option ON. After changing the direction of research, I no longer needed to learn it, so I'm unable to test if it works.

The pymva flag seems to provide an interface to python ML libraries, I've disabled it because it wouldn't work right away – needed external dependencies, perhaps.


use actual file exists check

This lets the plugin work on Windows, where files keys use backslashes e.g. "data\\global.yaml", but user assumes support for forward-slash node-style paths, e.g. "data/global.yaml".

It's based off of the key provided from metalsmith, from the source directory. File exists doesn't make sense for this, since it's based off what's in the Metalsmith source database.

@RobLoach I don't recall details of the issue, nor have I confirmed whether it's still the case.
It might be considered an upstream bug in Metalsmith - at some point the files object i loaded with entries based on paths in the file system, and the keys were not normalized to Unix-style slashes on Windows.
Thoughts?

Make sure the .yaml/.json file is in your source directory, and is being read. Otherwise, it won't be in the Metalsmith database.


mkl: add scalapack libraries to output

Motivation for this change
Add the scalapack extensions to the output.
Things done
Tested compilation with an external program (bagel quantum chemistry). Seems to work fine.

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @bhipple

@smaret Not sure it would make sense to unpack all RPMs. There is a lot of stuff that is distributed of a bunch directories that may not be required. @bhipple may be able to give more insight here.


[BUG] (type-safe-api) array responses produce broken code

Describe the bug
Returning an array as the top level response from an operation causes code to be generated which does not compile.
Expected Behavior
Working code.
Current Behavior
Broken code eg:
    const marshal = (statusCode: number, responseBody: any): string => {
        let marshalledBody = responseBody;
        switch(statusCode) {
            case 200:
                marshalledBody = JSON.stringify(Array<AssetResponse>ToJSON(marshalledBody));
                break;

Reproduction Steps
Use spec from https://github.com/aws/aws-pdk/issues/863
Possible Solution
No response
Additional Information/Context
No response
PDK version used
0.23.70
What languages are you seeing this issue on?
Typescript, Java, Python
Environment details (OS name and version, etc.)
osx

This is still an issue


Title card markup

The layout and design of the card are complete across all breakpoints and platforms

[ ] Desktop
[ ] Mobile

moving this to the backlog until we know if we're using a reactive title card or static title card.

@brmayes, Once we figure out the animation approach, can you work on the final design?


.prettierc overrides are ignored

It seems the plugin ignores prettier's overrides options. E.g. the following .prettierc fails for the directory build_tools:
{
  "trailingComma": "all",
  "overrides": [
    {
      "files": "build_tools/*.js",
      "options": {
        "trailingComma": "es5"
      }
    }
  ]
}

Apologies, this was not addressed to this repository.


Event loop integration makes bad recommendation

It suggests the following as event loop integration for ReactPHP:
$loop = React\EventLoop\Factory::create();
$loop->addPeriodicTimer(0, [$queue, 'run']);

That will basically be an infinite loop that always calls [$queue, 'run'] and hogs the CPU until the entire thing is done. A better way would be a short timer like 10 milliseconds. It's not as responsive, but at least it doesn't consume 100% CPU.

Can you create a PR?

No, sorry. I don't have enough time currently.


change stelligent-nu repo to a template

See https://docs.github.com/en/free-pro-team@latest/github/creating-cloning-and-archiving-repositories/creating-a-repository-from-a-template:

Creating a repository from a template is similar to forking a repository, but there are important differences:

A new fork includes the entire commit history of the parent repository, while a repository created from a template starts with a single commit.
Commits to a fork don't appear in your contributions graph, while commits to a repository created from a template do appear in your contribution graph.
A fork can be a temporary way to contribute code to an existing project, while creating a repository from a template starts a new project quickly.

stelligent-nu was marked as a template. It encourages the behavior we want for people going through the program and steers them away from the actions a repo contributor would take to copy the repo.

Agreed and done.


Scoreboard

Adds the /stats command and resolves #17

I've got a copy of Ze Kaiser's scoreboard in JSON format from Arken. I'll use a script to port it over