S can have negative eigenvalues in pyscf/pbc

I discovered a bug where a certain orientation of lattice vectors will result in a non-positive definite S that will crash the first call to eigh in the SCF cycle. The issue arises when I use a FCC lattice with vectors defined in the quantum-espresso convention (all x-coordinates are negative, i.e. a flip in the yz plane) with a supercell where the atoms are placed using a more 'conventional' choice of lattice vectors (all vectors in the positive octant), as I show in more detail in the MWE attached.
negS_mwe.txt
I have not exhaustively tested the scenarios in which S shows a negative eigenvalue, but here are some scenarios:

333 supercell and larger (smaller ones are OK)
Also shows in diamond
GTH-DZVP, but not GTH-SZV

While I'm not sure if the error in the calculation of S is as intended or something more difficult to track down, I would suggest that a check on the positive-definiteness of S be implemented prior to the initialization of any DF tensors, as I was notified of this error only after pyscf spent a day building a GDF tensor!
@verena-neufeld and @hongzhouye helped in this investigation, thanks!

Adding to what Bryan said, lib.cond, which is currently used by the SCF module to check the condition number for the AO overlap matrix (this line), cannot detect negative eigenvalues. It calculates the cond number roughly as abs(ev).max() / abs(ev).min() afaict where ev is the array of eigenvalues of the AO overlap matrix.

You need to make sure that all atoms are within the cell, i.e., the cell defined by cell.a.

You need to make sure that all atoms are within the cell, i.e., the cell defined by cell.a.

Should this not be checked by the initialization routine...?

I agree this should be checked on startup and the atoms could be
translated into the unit cell (not sure if this would mess up any user's
effort to do visualization after the fact).
On Wed, Jul 14, 2021 at 3:19 PM Susi Lehtola @.***>
wrote:

You need to make sure that all atoms are within the cell, i.e., the cell
defined by cell.a.
Should this not be checked by the initialization routine...?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pyscf/pyscf/issues/1017#issuecomment-880146641, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADGAJC7CMAHEE6JA7BTANFTTXXPM3ANCNFSM5AJSAOVQ
.

Does the translation of atoms into the unit cell actually make a difference
from the computational point of view? The actual origin of the unit cell is
never used (or I don't see why it should ever appear). The lattice vectors,
as far as I know, are only used to generate lattice translations, never to
assign a basis function or atom to a particular unit cell.
Sandeep.
On Wed, Jul 14, 2021 at 1:23 PM Timothy Berkelbach @.***>
wrote:

I agree this should be checked on startup and the atoms could be
translated into the unit cell (not sure if this would mess up any user's
effort to do visualization after the fact).
On Wed, Jul 14, 2021 at 3:19 PM Susi Lehtola @.***>
wrote:

You need to make sure that all atoms are within the cell, i.e., the cell
defined by cell.a.
Should this not be checked by the initialization routine...?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pyscf/pyscf/issues/1017#issuecomment-880146641, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ADGAJC7CMAHEE6JA7BTANFTTXXPM3ANCNFSM5AJSAOVQ
.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pyscf/pyscf/issues/1017#issuecomment-880148767, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AABVW4BDUJ6ZBHJAO4ETWG3TXXP2XANCNFSM5AJSAOVQ
.

cell.get_lattice_Ls will be affected depending on whether the atoms are inside the unit cell or not.

Bug fixed in PR 1044


Could the StoryBlok instance provide an isPreview flag?

Description
Unless I've missed it, there doesn't seem to be a way to detect if the site is being loaded within the StoryBlok editor from the StoryBlok instance.
My current use case is that my whole site is SSRed. In my mind, this should mean that I could run the editor and the front-end from a single deployment of the site.
The current documentation (link) seems built around the assumption that you'd want the editor SSRed and the front-end built out statically. The solution there is to have two separate identical deployments but with a single env var changed between the two to flag the story version to load.
Suggested solution or improvement
If the SKD provided a flag for if the current instance is loaded within the editor or not then you could have something like this:
---
//  index.astro
import { useStoryblokApi } from "@storyblok/astro";
import StoryblokComponent from<EMAIL_ADDRESS>import BaseLayout from "../layouts/BaseLayout.astro";

const storyblokApi = useStoryblokApi();
const { data } = await storyblokApi.get("cdn/stories/home", {
  version: storyblokApi.isPreview ? 'draft' : 'published',
});

const story = data.story;
---

<BaseLayout>
  <StoryblokComponent blok={story.content} />
</BaseLayout>

Additional context
No response
Validations

[X] Follow our Code of Conduct

Hi @nathobson
If you're using SSR, you can easily determine if your site is inside Storyblok by following these steps:
const inStoryBlokEditor =
  Astro.url.searchParams.has('_storyblok') &&
  Astro.url.searchParams.has('_storyblok_c')

You can console.log(Astro.url) to see the numerous query parameters added when you're in the visual editor. Additionally, you can refer to this guide on how to verify the preview query parameters of the visual editor if needed.
Although if you're using SSR and can handle the draft and published version from one deployment, our tutorial demonstrates two different deployments because it provides more control. Even if you're using full SSR, you may find adding caching on top of this single deployment a bit troublesome. However, if you want to do it, you can determine if you're in the Storyblok visual editor as I shared.

Closing this issue for now but please feel free to re-open the issue if you still have further questions/issues when you test the solution from above.


[docs] Updates to API reference content

Addressing the following Github issues:

Add content and sample for uploading file objects directly to workbook publish endpoint
https://github.com/tableau/server-client-python/issues/1007
https://github.com/tableau/server-client-python/issues/848

Fixed the duplicate issue and now merging


DDoS-resistant CloudFront CDN Website Construct

We need to build a Construct which deploys a website to AWS CloudFront that is DDoS resistant. The developer should be able to point at an arbitrary folder containing a compiled website and have it uploaded to AWS.
It's super important to be DDoS resistant in preparation for any raiding from malicious actors looking to kill a coin or crypto project.

Basic, Mutable:

S3.Asset pointing to local folder
S3 Bucket Deployment to CFN fronted bucket with Prune mode on.

Advanced, Immutable:

S3.Asset pointing to local folder
S3 Bucket Deployment to CFN fronted bucket with unique ID.
Update SSM parameter with unique ID
Website root is resolved from either API Gateway Lambda or Cloud Front resource fronted by Lambda@Edge to resolve the current immutable ID.

Existing code bases for reference. Both basic react sites with some backend. Nothing too special for the frontend, but hard to tell the hosting without going deeper. TODO: understand existing hosting strategies.
https://github.com/spartacus-finance/Spartacus-Landing
https://github.com/Wonderland-Money/wonderland-frontend

https://docs.ipfs.io/ IPFS instead of or along side CDN?

https://docs.ipfs.io/ IPFS instead of or along side CDN?

I know of projects that use IPFS to store things like videos or songs for building decentralized services like YouTube and Spotify. I wonder what the latency would be like. I don't think it's strictly necessary but it would be a useful Construct - an Asset which stores on IPFS!

Found IPFS when looking at the ETH dApp docs, suggesting the frontend be hosted on it
https://ethereum.org/en/developers/docs/dapps/#definition-of-a-dapp
Looks fairly inexpensive to start with a shadow of the content on IPFS and CloufFront, then we could measure performance. IPFS relies on peer to peer, but claims performance.

Can speed up the web when you're far away or disconnected. If you can retrieve a file from someone nearby instead of hundreds or thousands of miles away, you can often get it faster. This is especially valuable if your community is networked locally but doesn't have a good connection to the wider internet. (Well-funded organizations with technical expertise do this today by using multiple data centers or CDNs — content distribution networks (opens new window). IPFS hopes to make this possible for everyone.)

Cloud Front and S3 are dirt cheap (need to run numbers), but "free" is cheaper: https://aws.amazon.com/cloudfront/pricing/

Future interop with Filecoin (paid): https://filecoin.io/ https://docs.ipfs.io/concepts/faq/#ipfs-and-filecoin
We can claim more or have the option for more decentralization (at a cost, like latency).

Agree with IPFS and FileCoin integration. cdk3 can perhaps provide an abstraction over centralized and decentralized alternatives. Another example would be using Akash instead of Fargate: https://akash.network/


Replace hardcoded local gateway paths

Currently, the local gateways have paths that include a hardcoded path key. This hardcoded value should be replaced with a random uuid, which is the current configuration for AWS and GCP gateways. Slack discussion here.
For example, the local HTTP gateway path is defined as:
https://github.com/api3dao/airnode/blob/3472e6b8b5fccfce0104bd26c95e4c094fe5002c/packages/airnode-node/src/workers/local-gateways/server.ts#L126
with the hardcoded value from here:
https://github.com/api3dao/airnode/blob/3472e6b8b5fccfce0104bd26c95e4c094fe5002c/packages/airnode-node/src/workers/local-gateways/server.ts#L26
In contrast, AWS and GCP use random_uuid:
https://github.com/api3dao/airnode/blob/624b737766be6aed63ca7f6135f9a931a1cf8ecf/packages/airnode-deployer/terraform/aws/variables.tf#L10

Note that as of #1869, implementing this will require the E2E test to capture the non-hardcoded gateway URLs during local deployment because currently DEFAULT_PATH_KEY is used:
https://github.com/api3dao/airnode/blob/2a9831794a78e27256b3c1420d897d7e8a688240/packages/airnode-examples/test/e2e/coingecko-local.feature.ts#L71


Add supported car for 2019 Toyota Altis Hybrid

Fingerprint
Toyota-Altis Hybrid-2019
Car support
Toyota-Altis Hybrid-2019
https://my.comma.ai/5ceff72287a5c86c/1571534312818/1571537369543
Cabana
https://my.comma.ai/cabana/?route=5ceff72287a5c86c|2019-10-20--09-18-33&url=https%3A%2F%2Fchffrprivate-vzn.azureedge.net%2Fchffrprivate3%2Fv2%2F5ceff72287a5c86c%2F633c5011c172cc6c8f67084e2903ce1f_2019-10-20--09-18-33&seekTime=0

update other links
https://my.comma.ai/5ceff72287a5c86c/1571453941876/1571457352486
https://my.comma.ai/cabana/?route=5ceff72287a5c86c|2019-10-19--10-59-02&url=https%3A%2F%2Fchffrprivate-vzn.azureedge.net%2Fchffrprivate3%2Fv2%2F5ceff72287a5c86c%2F23574dab99ad8a81d33b9ee1e0b223e1_2019-10-19--10-59-02&seekTime=992

Thanks! My understanding is that the Altis is the same as the Corolla in the US market., correct? If yes, I would prefer to call it Corolla Hybrid (COROLLAH_TSS2) for consistency with the US market.

Also, fingerprint looks good. If you can change the name of the car, then it's good to be merged.

Hi rbiasini,
corolla_tss2 and altish_tss2 have different dbc as below.
CAR.COROLLA_TSS2: dbc_dict('toyota_nodsu_pt_generated', 'toyota_tss2_adas'),
CAR.ALTISH_TSS2: dbc_dict('toyota_nodsu_hybrid_pt_generated', 'toyota_tss2_adas'),
I try to add fingerprint in corolla_tss2 but get an "Communication Issue between processes" error.
I think the different is about the gear. AltisH gear is "P R N D B"

Ah, good catch! Can we just call it Toyota Hybrid and add it to the Readme (see my diffs)? Then it's good to go.

Sure. Thanks for help.
so, should I close this request?

Oh, why closing it? I mean, if you change the car name and add it to the readme I'll merge it in.

Got it. :)
I had changed the car name and add it to the readme.
Thanks.

Almost. Sorry, I'm not being clear, my fault. See 2 comments in PR

It's ok.
Please check it.
Thanks.

looks good!


InstanceType<(typeof this)["somefield"]> type issue

Hi, for inheritance purposes i need to type methods depending on class returned by getter, but i get  'is not assignable' error.
Is this a bug?
class S1 {}
class S2 extends S1 {}

class A  {
   get x() {
    return S1
  }

  f(): InstanceType<(typeof this)["x"]> {
   return new this.x as InstanceType<(typeof this)["x"]> // no error
  }

  fError(): InstanceType<(typeof this)["x"]> {
   return new this.x // got error
  }
}

class B extends A {
  get x() {
    return S2
  }

  f1(){
    return this.f();
  }
}

const a = new A;
const s1test = a.f() // S1 type

const b = new B;
const s2test = b.f1() // S2 type

https://www.typescriptlang.org/play?#code/MYGwhgzhAEDKCM0DeBfAUKSNYCZoFMAPAF3wDsATbRVNOzKaAQWmTVegHN9jpCAKAJRJ2HAE48ArmLJx4o9KIBmQgFzQAkmQjEwZYPgAqATwAO+ADzEAFgEsIAbQBEhJwF0AfG3FSZ0MvgA7tA29gB0hNCQmtq6+kZmlqGOLu5eAPTp-gD2BGJi2WIKdKxKAKL5hWoxOnoGJuZWdimunt6sEsTSsgHByRHQmVzZvPiVRazoihjgjABCBCTkVMzt3LwCgu0dvrK4xcrwQiIc0J3dIc1hKoIA3MXTwNmxUdAAvP5BzPdPLxDwpB07yi1yEgyyCBCiToGGeQIARsDetA5j84bwIDhAbwPvDrkctkNcFDzEA

There are too many things wrong with this code but I'll just leave a good solution
class S1 {}
class S2 extends S1 {}

class A<X extends new (...args: any[]) => any> {
  constructor(public cls: X) {}

  get x(): X {
    return this.cls
  }

  f(): X {
    const o = new this.x()
    return o;
  }
}

class B<X extends abstract new (...args: any[]) => unknown> extends A<X> {
  f1(){
    return this.f();
  }
}

const a = new A(S1);
const s1test = a.f() // S1 type
//    ^?

const b = new B(S2);
const s2test = b.f1() // S2 type
//    ^?

demo

thanks, but based on current architecture (including dependency injector) i can't accept your solution.
Also, your code have error (in extends of B) and produce types, not instances (s1test/s2test)
The question is why it not work as expected

thanks, but based on current architecture (including dependency injector) i can't accept your solution. Also, your code have error (in extends of B) and produce types, not instances (s1test/s2test) The question is why it not work as expected

@imsamurai
You are correct
Here is the one that correctly returns the type
class S1 {}
class S2 extends S1 {}

class A<X extends new (...args: any[]) => any> {
  constructor(public cls: X) {}

  get x(): X {
    return this.cls
  }

  f(): InstanceType<X> {
    return new this.x();
  }
}

class B<X extends new (...args: any[]) => any> extends A<X> {
  f1(){
    return this.f();
  }
}

const a = new A(S1);
const s1test = a.f() // S1 type
//    ^?

const b = new B(S2);
const s2test = b.f1() // S2 type
//    ^?

(demo)[https://www.typescriptlang.org/play?ts=4.9.0-dev.20220930#code/MYGwhgzhAEDKCM0DeBfAUKSNYCZoFMAPAF3wDsATbRVNDcKaAQQB4ANAk8q6M-Ad2gAKAHRiwAJwDmEAFzQwZAJ4BtALoBKaAF4AfAuX6kaaNGAB7MhGISArsGLmJQgA62ARiACWwMyDnQbFq0JtBS+MTQhEIa8hzGpqYSEbYSZNDEABZeECKgEKHooQBmMfIAklbEisD4ACpKLvjsRqFJKWm8AhnZudEaANyFaEWYjABC7JyklDB8gqLi0gGKqpo6+qv6RDM8rGytpsXwMQmJycSp6Vk5IqWDw0UYltYKOl2CTEIIDxZV0BB4KRXtoFHcYtAAPSQuCIYiNfBoaGJaAAPQA-HQ-q93O95tBxt8cL8XpEIDhgZFQe47ictMjcBkEUiYYkMUA]


Format code using 'cargo fmt'

Great idea!
Would it be possible to do it with max_width set to 120? There were several lines that ended up getting wrapped in the PR that I hope can be kept on one.

@tkaitchuck sorry I missed this. Yes, max_width can be set to 120. I have updated the PR, please review and merge.


Upgrade leak_tracker to 5.0.0

Fixes https://github.com/flutter/flutter/issues/126259
Updated pubspec:
https://github.com/flutter/flutter/blob/345f0bffbfda31fab7d4a740f65b3d1610d20a99/packages/flutter/pubspec.yaml

Looks like the checks are unhappy, though.

So this is breaking analysis because package:vm_service added package:protobuf and package:fixnum to the transitive closure in https://github.com/dart-lang/sdk/commit/6c3c34560e6fdac211e362078557fdda254bafdd#diff-e8b526145c30754d3e9fe773ac388cb9c9c30a4210628dae2f534eee7ef6c63f

cc @derekxu16

I will push a change to this PR pinning package:vm_service.

Ok, the version of vm_service introducing protobuf was retracted, so I rolled back that dep in https://github.com/flutter/flutter/pull/126367/commits/d049ebc1fee43a42224b54dd2f02d0d801b68865, let's see if CI passes.

Looks I might have pulled in a new dep that has another build failure:
[        ] A problem occurred evaluating project ':camera_android'.
[        ] > Could not set unknown property 'baseline' for task ':camera_android:lint' of type com.android.build.gradle.tasks.LintGlobalTask.
[        ] * Try:
[        ] > Run with --debug option to get more log output.
[        ] > Run with --scan to get full insights.
[        ] * Exception is:
[        ] org.gradle.api.GradleScriptException: A problem occurred evaluating project ':camera_android'.

pinned camera_android per https://github.com/flutter/flutter/issues/126710


SILGen: Copy the block before detaching a task for async methods called from ObjC.

The block needs to survive long enough for the task to get scheduled. Fixes rdar://76871310.

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - acac14e757ca7e678421a5e7a4171e86c4e9fe87

@swift-ci Please test


Add ability to make History checkpoints

The main issue here is that we need to save the coupler accumulators and counters. This will be very useful for monthly means

Fixed by #277


Performance problems when lots of time steps loaded

@leighorf reports that he can load 3400 files. But once loaded a window resize takes an unreasonable amount of time. Possibly other operations are expensive as well. This should be tested more thoroughly.

Note: We'll probably need to synthesize a data set to reproduce this problem. Leigh's was on the order of terabytes..

Leigh has all his data in the scratch space. Do we need to move them to a
permanent place?
On Thu, Dec 20, 2018, 4:57 PM clyne<EMAIL_ADDRESS>wrote:

Note: We'll probably need to synthesize a data set to reproduce this
problem. Leigh's was on the order of terabytes..
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/VAPOR/issues/1130#issuecomment-449181525, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAxvJSZEq1GeaerzUJuNSGP4BA-5lOq9ks5u7CPrgaJpZM4ZdHIU
.

We should copy it to CISL’s new  “campaign storage”. Would you care to attempt that?
thanks - jc

On Dec 20, 2018, at 7:45 PM, Samuel Li<EMAIL_ADDRESS>wrote:
Leigh has all his data in the scratch space. Do we need to move them to a
permanent place?
On Thu, Dec 20, 2018, 4:57 PM clyne<EMAIL_ADDRESS>wrote:

Note: We'll probably need to synthesize a data set to reproduce this
problem. Leigh's was on the order of terabytes..
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/VAPOR/issues/1130#issuecomment-449181525, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAxvJSZEq1GeaerzUJuNSGP4BA-5lOq9ks5u7CPrgaJpZM4ZdHIU
.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

John Clyne
National Center for Atmospheric Research
303.497.1236 (w),<PHONE_NUMBER> (c)
<EMAIL_ADDRESS>

Sure; let me figure out how to make use of this “campaign storage.”
On Fri, Dec 21, 2018 at 11:57 AM clyne<EMAIL_ADDRESS>wrote:

We should copy it to CISL’s new “campaign storage”. Would you care to
attempt that?
thanks - jc

On Dec 20, 2018, at 7:45 PM, Samuel Li<EMAIL_ADDRESS>wrote:
Leigh has all his data in the scratch space. Do we need to move them to a
permanent place?
On Thu, Dec 20, 2018, 4:57 PM clyne<EMAIL_ADDRESS>wrote:

Note: We'll probably need to synthesize a data set to reproduce this
problem. Leigh's was on the order of terabytes..
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/VAPOR/issues/1130#issuecomment-449181525,
or mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AAxvJSZEq1GeaerzUJuNSGP4BA-5lOq9ks5u7CPrgaJpZM4ZdHIU

.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

John Clyne
National Center for Atmospheric Research
303.497.1236 (w),<PHONE_NUMBER> (c)
<EMAIL_ADDRESS>—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/VAPOR/issues/1130#issuecomment-449470487, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAxvJa54XYFqdK-b7bv_ai3_xd8PqFeVks5u7S8cgaJpZM4ZdHIU
.

Update: the Leigh's data is all in the campaign storage space now.
On Fri, Dec 21, 2018 at 12:02 PM Samuel Li<EMAIL_ADDRESS>wrote:

Sure; let me figure out how to make use of this “campaign storage.”
On Fri, Dec 21, 2018 at 11:57 AM clyne<EMAIL_ADDRESS>wrote:

We should copy it to CISL’s new “campaign storage”. Would you care to
attempt that?
thanks - jc

On Dec 20, 2018, at 7:45 PM, Samuel Li<EMAIL_ADDRESS>wrote:
Leigh has all his data in the scratch space. Do we need to move them to
a
permanent place?
On Thu, Dec 20, 2018, 4:57 PM clyne<EMAIL_ADDRESS>wrote:

Note: We'll probably need to synthesize a data set to reproduce this
problem. Leigh's was on the order of terabytes..
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/VAPOR/issues/1130#issuecomment-449181525,
or mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AAxvJSZEq1GeaerzUJuNSGP4BA-5lOq9ks5u7CPrgaJpZM4ZdHIU

.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

John Clyne
National Center for Atmospheric Research
303.497.1236 (w),<PHONE_NUMBER> (c)
<EMAIL_ADDRESS>—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/VAPOR/issues/1130#issuecomment-449470487, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAxvJa54XYFqdK-b7bv_ai3_xd8PqFeVks5u7S8cgaJpZM4ZdHIU
.

Terrific. Thanks, Sam!
jc

On Dec 21, 2018, at 10:57 PM, Samuel Li<EMAIL_ADDRESS>wrote:
Update: the Leigh's data is all in the campaign storage space now.
On Fri, Dec 21, 2018 at 12:02 PM Samuel Li<EMAIL_ADDRESS>wrote:

Sure; let me figure out how to make use of this “campaign storage.”
On Fri, Dec 21, 2018 at 11:57 AM clyne<EMAIL_ADDRESS>wrote:

We should copy it to CISL’s new “campaign storage”. Would you care to
attempt that?
thanks - jc

On Dec 20, 2018, at 7:45 PM, Samuel Li<EMAIL_ADDRESS>wrote:
Leigh has all his data in the scratch space. Do we need to move them to
a
permanent place?
On Thu, Dec 20, 2018, 4:57 PM clyne<EMAIL_ADDRESS>wrote:

Note: We'll probably need to synthesize a data set to reproduce this
problem. Leigh's was on the order of terabytes..
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/VAPOR/issues/1130#issuecomment-449181525,
or mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AAxvJSZEq1GeaerzUJuNSGP4BA-5lOq9ks5u7CPrgaJpZM4ZdHIU

.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

John Clyne
National Center for Atmospheric Research
303.497.1236 (w),<PHONE_NUMBER> (c)
<EMAIL_ADDRESS>—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/VAPOR/issues/1130#issuecomment-449470487, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAxvJa54XYFqdK-b7bv_ai3_xd8PqFeVks5u7S8cgaJpZM4ZdHIU
.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

John Clyne
National Center for Atmospheric Research
303.497.1236 (w),<PHONE_NUMBER> (c)
<EMAIL_ADDRESS>


[FEATURE] wash dev

To help tighten the inner dev loop, we should add a command to wash (we've been also discussing a new experiments subcommand to put these under e.g. wash experiments dev ) that watches for changes, runs wash build and then wash deploy . This could even leverage something like wash spy (see #599).
Basic psuedocode

on wash dev

wash up if not up yet (make sure to enable FS upload)
run wash build
if actor is already running

wash ctl stop <actor>

wash ctl start <file>
watch fs for changes
on change run wash build

So an example flow would look like this:
$ wash dev
Building `echo`
Pushing `echo`
Deployed `echo`
Watching for changes...

Received invocation # wash spy part
{ "method": "GET", ...}

src/lib.rs changed, rebuilding
Building `echo`
Pushing `echo`
Deployed `echo`

@vados-cosmonic said he was going to work on this

Yep, thanks @thomastaylor312  -- open to input as well on ergonomics, so please feel free to comment in this issue if you have any feedback/wants!

implemented in https://github.com/wasmCloud/wash/pull/610


Upgrade to RadioLib 6.0.0

Hi
I found issue when using RadioLib 6.0.0 which is latest lib version
`//Farm-Data-Relay-System/src/fdrs_gateway_lora.h: In function 'void begin_lora()':
/Arduino/libraries/Farm-Data-Relay-System/src/fdrs_gateway_lora.h:293:30: error: no matching function for call to 'SX1278::setDio0Action(void (&)())'
   radio.setDio0Action(setFlag);
                              ^
In file included from /Arduino/libraries/RadioLib/src/modules/RFM9x/RFM95.h:9:0,
                 from /Arduino/libraries/RadioLib/src/RadioLib.h:77,
                 from /Arduino/libraries/Farm-Data-Relay-System/src/fdrs_gateway_lora.h:1,
                 from /Arduino/libraries/Farm-Data-Relay-System/src/fdrs_gateway.h:76,
                 from /.git/MedjuglaviceAtPalm/RemoteGateway/RemoteGateway.ino:9:
Arduino/libraries/RadioLib/src/modules/RFM9x/../SX127x/SX127x.h:700:10: note: candidate: void SX127x::setDio0Action(void (*)(), uint32_t)
     void setDio0Action(void (*func)(void), uint32_t dir);
          ^
Arduino/libraries/RadioLib/src/modules/RFM9x/../SX127x/SX127x.h:700:10: note:   candidate expects 2 arguments, 1 provided

exit status 1`

when using 5.7.0 everything works just fine. So project should be updated or restriction to use 5.7.0 specified in readme.

Thanks! I just saw the RadioLib release notes about this yesterday, and fixing it is on my big board of things to do. It's kind of urgent, so I'm going to have to get into it this tomorrow along with a whole backlog of code updates.

I added the extra parameter in a commit last night, so I think everything should work okay now. I'm going to leave this open until I'm a bit more confident about the other RL changes.
It also still works with RL v5.7, which is a relief to me.

Issue still exist when using
#define RADIOLIB_MODULE SX1262
#define USE_SX126X
I'm not sure how to fix that.
Compiler output:
c:\users\xxx\documents\arduino\libraries\radiolib\src\modules\sx126x\SX126x.h:568:10: note: candidate: 'virtual void SX126x::setDio1Action(void (*)())'
void setDio1Action(void (*func)(void));
^~~~~~~~~~~~~
c:\users\xxx\documents\arduino\libraries\radiolib\src\modules\sx126x\SX126x.h:568:10: note:   candidate expects 1 argument, 2 provided
exit status 1
Compilation error: exit status 1

Very good point. It looks like the new thing is only applicable to SX127x. I've now reversed the change for that chip.

@aviateur17 Could you maybe help explain what this second point means? If I'm understanding right, it's not an issue unless we're working with a pin value that was returned by RadioLib or assigning a pin based using a variable (which we never do).

Looking at the commit (https://github.com/jgromes/RadioLib/commit/9a68a3c9012e1ffe5dc94cb4d116cbe71ad3d3ea)  all they really do is change uint8_t to uint32_t so the only thing would be to check our types to make sure they are uint32_t otherwise there may be a compiler warning.  I don't see a need to adjust any pin names or values just because of this change.

Very good point. It looks like the new thing is only applicable to SX127x. I've now reversed the change for that chip.

And it works ;)


Usability improvements in wrapper libs - change the attribute name mapping in array()

A little bit of background: our data contains a lot of nested structures and hard-to-remember branch paths, on top of that, there are classes with tons of attributes which are reused for different things, while a set of attributes are just noise, which we would like to hide from the users. For example there is a generic Hit class which is used for MC hits and also for DAQ hits, but only a subset of the attributes are valid in the MC context and another (partly overlapping subset) for the DAQ context.
I maintain a wrapper library for uproot which is tailored to our experiment and offers a few classes so that it's possible to do things like:
r = WhateverReader("file.root")
r.events.hits.pmt_id   # returns a jagged array of PMT IDs for each event
r.events[2:5].hits.dom_id  # a jagged array of PMT IDs for a slice of events
r.events[:10].hits.dom_id[:5]  # a jagged array of the the first 5 DOM IDs of the first 10 events

Behind the scenes, .events maps to specific branch with a cryptic path and .hits to a subbranch, while dom_id and pmt_id etc. are also subbranches and .lazyarray() or .array() are called behind the scenes.
The indexing magic is done by propagating all the indices to the actual Jagged-array, so that these are equivalent:
r.events[0].hits.pmt_id[:5] == r.events.hits[0].pmt_id[:5] == r.events.pmt_id[0][:5] # etc.

Such a call translates then to something like
fobj["/a/long/cryptic/path/to/UglyPMT_ID_Naming"].lazyarray(interpretation, options)[0, :5]

For some branches I need to use custom interpretations and even some conversion to awkward1 etc. But you get the idea, a lot of happening behind the scenes 😉 On top of that, there is tab completion which shows only meaningful attributes. To achieve all this, I introduced (mapper) configurations for each branch which can be passed to a generic class.
Alright, sorry, long story short, in uproot4 everything is much easier since the Awkward1 arrays are working really nicely, so I can get rid of many optimisations and special hooks under the hood. But the basic problems remain and I think that many uproot users encounter these and create libraries around to make them more user-friendly, so I thought it would be useful to dump this somewhere, in case these are already solved in a much better way or if there is interest in providing a more general user API for these presumably repeated tasks when building upon uproot:

inconvenient, hard-to-remember branch addresses
"noise" in API (field naming, hiding unused fields etc.)
grouping things which belong together but are stored in different branches
indexing magic (see example above) which spans over multiple branches and propagates down to the data

Anyways, this issue is mostly about renaming fields, so let me come to the point 🙈
Given an example file from our experiment, you can see some events which consist of a jagged array of hits. I took this example because it almost works and only needs some adjustments in the field names:
>>> import uproot4

>>> from skhep_testdata import data_path

>>> f = uproot4.open(data_path("uproot-issue431b.root"))

>>> f["E/Evt/hits"].show()
name                 | typename                 | interpretation
---------------------+--------------------------+-------------------------------
hits                 | vector<Hit>              | AsGroup(<TBranchElement 'hits'hits.id              | int32_t[]                | AsJagged(AsDtype('>i4'))
hits.dom_id          | int32_t[]                | AsJagged(AsDtype('>i4'))
hits.channel_id      | uint32_t[]               | AsJagged(AsDtype('>u4'))
hits.tdc             | uint32_t[]               | AsJagged(AsDtype('>u4'))
hits.tot             | uint32_t[]               | AsJagged(AsDtype('>u4'))
hits.trig            | int32_t[]                | AsJagged(AsDtype('>i4'))
hits.pmt_id          | int32_t[]                | AsJagged(AsDtype('>i4'))
hits.t               | double[]                 | AsJagged(AsDtype('>f8'))
hits.a               | double[]                 | AsJagged(AsDtype('>f8'))
hits.pos.x           | double[]                 | AsJagged(AsDtype('>f8'))
hits.pos.y           | double[]                 | AsJagged(AsDtype('>f8'))
hits.pos.z           | double[]                 | AsJagged(AsDtype('>f8'))
hits.dir.x           | double[]                 | AsJagged(AsDtype('>f8'))
hits.dir.y           | double[]                 | AsJagged(AsDtype('>f8'))
hits.dir.z           | double[]                 | AsJagged(AsDtype('>f8'))
hits.pure_t          | double[]                 | AsJagged(AsDtype('>f8'))
hits.pure_a          | double[]                 | AsJagged(AsDtype('>f8'))
hits.type            | int32_t[]                | AsJagged(AsDtype('>i4'))
hits.origin          | int32_t[]                | AsJagged(AsDtype('>i4'))
hits.pattern_flags   | uint32_t[]               | AsJagged(AsDtype('>u4'))

In this case, hits is already accessible as an awkward1.Array but you can also read the branches separately. For this particular example however, only the following fields are of interest
dom_id, channel_id, t, tot, trig, everything else does not make sense in a specific context (due to the general Hit class which is unfortunately reused for everything, but this is what we have to live with).
Now in the first step I of course don't want to create (and allocate) a new array just to rename those fields, but directly modify the interpretation. We can also not change the binary structure of the dtype for obvious reasons. I tried something like this:
>>> interpretation = AsGrouped(f["E/Evt/hits"], {'id': AsJagged(AsDtype('>i4')), 'dom_id': AsJagged(AsDtype('>i4')), 'channel_id': AsJagged(AsDtype('>u4')), 'tdc': AsJagged(AsDtype('>u4')),
...  'tot': AsJagged(AsDtype('>u4')), 'trig': AsJagged(AsDtype('>i4')), 'pmt_id': AsJagged(AsDtype('>i4')), 'time': AsJagged(AsDtype('>f8')), 'charge': AsJagged(AsDtype('>f8')), 'pos_x': As
... Jagged(AsDtype('>f8')), 'pos_y': AsJagged(AsDtype('>f8')), 'pos_z': AsJagged(AsDtype('>f8')), 'dir_x': AsJagged(AsDtype('>f8')), 'dir_y': AsJagged(AsDtype('>f8')), 'dir_z': AsJagged(AsD
... type('>f8')), 'pure_time': AsJagged(AsDtype('>f8')), 'pure_charge': AsJagged(AsDtype('>f8')), 'type': AsJagged(AsDtype('>i4')), 'origin': AsJagged(AsDtype('>i4')), 'pattern_flags': AsJa
... gged(AsDtype('>u4'))})

but this will fail since those field names are stored also deeper in the structure, in a keys-dictionary (the dtype names are coupled to the branch names):
KeyInFileError: not found: 'id'

    Available keys: 'hits.id', 'hits.tdc', 'hits.t', 'hits.a', 'hits.tot', 'hits.dir.x', 'hits.dir.y', 'hits.dir.z', 'hits.dom_id', 'hits.trig', 'hits.pmt_id', 'hits.type', 'hits.pos.x', 'hits.pos.y', 'hits.pos.z', 'hits.pure_t'...

I followed the stack trace and found myself in HasBranches where I came to the conclusion that it's not so trivial to hook into the attribute naming and remap those. I though I could introduce a field_remap dictionary (e.g. {"hits.dom_id": "dom_id", hits.t: "time", ...} etc.) somewhere, but it's not easy to see where it would fit the best to not mess around with low level mechanics and break other stuff (iterate, caching etc.), so clearly I need some advise from an expert (Jim 😉 ).
My next idea was to try to see what I can do on the awkward1.Array side since that feels more accessible and much safer, however, I could not figure out how to change field names after array initialisation. Maybe it's better to do this in uproot directly, no idea. In the end, it should also work with iterate() etc. where you already can select just a subset of the attributes, which is the way to hide the "noisy fields".
So maybe the most straightforward soulution is ff iterate() and array() would provide a rename_fields option, but as written above, I guess it will have to be propagated to quite a low level part of the code.
Sorry for the long issue, but I am sure you have some good ideas about how to deal with this and maybe I am not alone with these problems ;)

Two things that might be relevant for this: the arrays/iterate/etc. functions can take aliases. If the TTree has built-in fAliases, then these are used first, but you can also provide them as a user.
>>> import uproot4
>>> import skhep_testdata
>>> t = uproot4.open(skhep_testdata.data_path("uproot-Zmumu.root"))["events"]
>>> t.arrays(["p1", "p2"])
Traceback (most recent call last):
...
KeyError: 'p1'
>>> t.arrays(["p1", "p2"], aliases={
...     "p1": "sqrt(px1**2 + py1**2 + pz1**2)",
...     "p2": "sqrt(px2**2 + py2**2 + pz2**2)"
... })
<Array [{p1: 82.2, p2: 60.6}, ... p2: 171}] type='2304 * {"p1": float64, "p2": f...'>

(There's no magic here: this is just applying the string-based formula as Python code after loading the necessary arrays, but if you build up a dict of aliases, you can hide a lot of formulae in a package.)
The second thing is NanoEvents: @nsmith- has been writing a wrapper like yours for CMS data, which renames and restructures the TTree without reading it (i.e. lazy reading). It was originally designed for ROOT files with a NanoAOD structure, but it has been generalized to be parameterized by a YAML file, such that two different ROOT file structures have successfully been read with it.
I doubt the NanoEvents infrastructure can apply custom Interpretations, but it might not be a huge extension.
For something like this, I'd rather build on top of Uproot than include it in Uproot, although there is this "rename_fields" like behavior through aliases, and ideally, Uproot should be coming up with the right Interpretations on its own, though I know you have some strange files.

It's a shame that I have not thought about using the alias functionality, thanks!
I will also look into NanoEvents, it sounds like I can borrow from there, very promising.
Given that the core functionality of renaming fields is there, I definitely agree that building upon instead of integrating it is the better choice.

Jim linked to the awkward0 version of NanoEvents.

Yikes! I changed the link above to the new NanoEvents so that no one clicks through by mistake.

Alright, many thanks for both of you, I think I a have all the tools and inspiration to move forward.
I'll close the issue now.

@tamasgal any chance you can re-use NanoEvents for your case? Happy to collaborate on it.

I agree that generalizing NanoEvents across experiments would be a great thing. It provides something unique that should sit at a layer above Awkward Array and Uproot, but still not be tied to a specific experiment (or TTree format, which it's already broken free of).

Indeed, I think that's a good idea in general. I just need to understand how NanoEvents works and how it maps to our use-case. We already have lots of code based on the current API and if it's possible to replace the inner parts, that would of course be great.
Do you think that the nanoevents part of coffea could be extracted, so that we can minimise dependencies?

For sure in the long run I would like to (plus a name change...). In the mean time, I would imagine your existing dependencies mostly overlap https://github.com/CoffeaTeam/coffea/blob/master/setup.py#L58-L74 but if those look too onerous do let me know (and which ones! we could try to slim a bit)

Ah ok, yes that seems quite huge compared to our low level I/O https://git.km3net.de/km3py/km3io/-/blob/master/requirements/install.txt 😉
The library is really just a thin wrapper, in the sense that for a specific type of ROOT file, we only define the mappings for a branch and subbranches using a BranchMapper class (https://git.km3net.de/km3py/km3io/-/blob/6f893c07beae7addacb6b2db62e4e37391b04653/km3io/rootio.py#L13)  seen in action here:  https://git.km3net.de/km3py/km3io/-/blob/6f893c07beae7addacb6b2db62e4e37391b04653/km3io/offline.py#L25
And after then initialse a subclass of Branch in our *Reader classes, like here https://git.km3net.de/km3py/km3io/-/blob/6f893c07beae7addacb6b2db62e4e37391b04653/km3io/offline.py#L205
That will then take over the mapping, tab-completion and index propagation as shown in the above example, by manipulating via getattr, getitem, getkey and alike, as you can see here https://git.km3net.de/km3py/km3io/-/blob/6f893c07beae7addacb6b2db62e4e37391b04653/km3io/rootio.py#L169
The indices are all collected and finally unfolded and passed to the actual branch, directly coming from uproot (vai unfold_indices: https://git.km3net.de/km3py/km3io/-/blob/6f893c07beae7addacb6b2db62e4e37391b04653/km3io/tools.py#L31)
This is basically what then provides the above mentioned API:
r = WhateverReader("file.root")
r.events.hits.pmt_id   # returns a jagged array of PMT IDs for each event
r.events[2:5].hits.dom_id  # a jagged array of PMT IDs for a slice of events
r.events[:10].hits.dom_id[:5]  # a jagged array of the the first 5 DOM IDs of the first 10 events

So in principle, everything is fairly simple and you see a lot of traces for some hacks, like the forced conversion to awkward arrays from uproot3 for specific branches (https://git.km3net.de/km3py/km3io/-/blob/6f893c07beae7addacb6b2db62e4e37391b04653/km3io/rootio.py#L37) or other cache/lazy related optimisations. Many of them are obsolete when transitioning to uproot4.
If I understood correctly, in nanoevents the mapper thing is basically the TreeMakerSchema (https://github.com/CoffeaTeam/coffea/blob/a5583401173859878b52dea44b14ed6c613aea81/coffea/nanoevents/schemas.py#L297) and I also see some mappings in NanoAODSchema: https://github.com/CoffeaTeam/coffea/blob/a5583401173859878b52dea44b14ed6c613aea81/coffea/nanoevents/schemas.py#L132 which are very similar to those we need, I guess.
I am trying  to wrap my head around and find out how I could implement a scheme for our files.
Here is a type of a ROOT file which does not use custom interpretations, so this should be I think easy to map in NanoEvents. This is how the file looks like in uproot4 (I shortened the list of keys, for the sake of simplicity):
>>> import uproot4
>>> from km3net_testdata import data_path
>>> f = uproot4.open(data_path("offline/km3net_offline.root"))
>>> f["E/Evt"].keys()
['AAObject',
 'AAObject/TObject',
 'AAObject/TObject/fUniqueID',
 'AAObject/TObject/fBits',
 'AAObject/usr_data',
 'AAObject/usr_data/usr_data.holder',
 'AAObject/usr_names',
 'id',
 'det_id',
 'mc_id',
 'run_id',
 'mc_run_id',
 'frame_index',
 'trigger_mask',
 'trigger_counter',
 'overlays',
 't',
 't/t.fSec',
 't/t.fNanoSec',
 'hits',
 'hits/hits.id',
 'hits/hits.dom_id',
 'hits/hits.channel_id',
 'hits/hits.tdc',
 'hits/hits.tot',
 'hits/hits.trig',
 'hits/hits.pmt_id',
 'hits/hits.t',
 'hits/hits.a',
 'hits/hits.pos.x',
 'hits/hits.pos.y',
 'hits/hits.pos.z',
 'hits/hits.dir.x',
 'hits/hits.dir.y',
 'hits/hits.dir.z',
 'hits/hits.pure_t',
 'hits/hits.pure_a',
 'hits/hits.type',
 'hits/hits.origin',
 'hits/hits.pattern_flags',
 'trks',
 'trks/trks.fUniqueID',
 'trks/trks.fBits',
 'trks/trks.usr_data',
 'trks/trks.usr_names',
 'trks/trks.id',
 'trks/trks.pos.x',
 'trks/trks.pos.y',
 'trks/trks.pos.z',
 'trks/trks.dir.x',
 'trks/trks.dir.y',
 'trks/trks.dir.z',
 'trks/trks.t',
 'trks/trks.E',
 'trks/trks.len',
 'trks/trks.lik',
 'trks/trks.mother_id',
 'trks/trks.fitinf',
 'trks/trks.hit_ids',
 'trks/trks.error_matrix',
 'w',
 'index',
 'flags']

So, the entry at "E/Evt" is our vector of events, which consist of hits and trks and other parameters. That is what we translate to f.events and f.events.hits.dir_x etc (which points to f["E/Evt/hits/hits.dir.x"].lazyarray(options...) etc.).
This allows to do the fancy indexing like mentioned above and other stuff like handy iteration:
for event in f.events:
    calib_hits = calibrate(event.hits, calibration)
    ...

As you can see, it's all about grouping things together and offering a user-friendly interface (tab completion,  hiding noisy stuff etc.):

Do you think that's easy to do with NanoEvents?

I just realised, it's a flood of source code links, sorry for that 🙈 but maybe it's easier to communicate the needs and implementations.


Mouse teleporting while adjusting values on blender

Hyprland Version
v0.29.1-56-ged51fe7b
Bug or Regression?
Bug
Description
Mouse teleports to the center of the screen when dragging value to adjust it
How to reproduce
Hold and drag value
Crash reports, logs, images, videos
https://github.com/hyprwm/Hyprland/assets/31055847/1d050641-081f-438a-ad6e-2db05c38fd31
No response

patch.txt
try zis

Does not work, cursor still warps to centre.

hm. Will have to install that hunk of garbage and test then. I hope this happens on the normal packaged release?

normal packaged release

You mean stable blender release? Yes.

I have the same issue (on the stable blender release)

I had the same issue when I tried to update to 0.29.0, and again with 0.29.1. The cursor would warp to the top-left corner of the window every time I try to click into or adjust a numeric value. hyprland-git presents a similar issue, except the cursor warps to the center of the window rectangle. 0.28.0 compiled from source, however, didn't have this issue.
Of particular note is the fact that this only happens with certain UI elements.

The numeric input field actually hides the cursor on a mouse-down event because it also responds to horizontal mouse movement, and unhides it on mouse-up.
Resizing UI areas in Blender presents the same issue, but only on mouse-up.
Splitting an area in half, or joining two adjacent areas, by dragging on the corners where they meet presents the same issue.
Dragging or resizing nodes, or connecting wires in the node editor.
Using a sculpting, painting, or drawing tool.

This does NOT happen in many other cases:

Non-numeric inputs
Using selection (rect, circle, lasso) tools
Dragging curve points in the graph editor
Dragging keyframes in the dope sheet

This is 0.29.1 from the Arch package.
https://github.com/hyprwm/Hyprland/assets/37093630/dbabb812-5658-4527-b080-0ed0b53beb79

god forgive me for I have sinned

this doesn't help.

patch.txt
try this patch

try this patch

Much better. It fixed most of the occurences, except the numeric inputs. Both dragging and clicking into them snaps the mouse to the window's center.

I pushed the patch. For the numeric inputs, can I get a video?

I can't record it right now, but it's almost the exact same as the video in my previous comment. The first thing I show there is the numeric inputs - dragging first, clicking second. The only difference now is that the cursor moves to the center instead of the corner.

this doesn't help.

Blender in the Arch repo depends on the open shading language, which has a hard dependency on llvm15, causing a segfault.
testing repo works: https://archive.archlinux.org/packages/b/blender/blender-17%3A3.6.2-6-x86_64.pkg.tar.zst

please check #4889

Tested and its working correctly for me
Im not sure if this is relevant for this issue but i dont use any scaling

You may ignore the issue if you deselect the Continues Grab setting.


Unmap on Windows fails

package main

import (
	"log"

	mmap "github.com/edsrzf/mmap-go"
)

func main() {
	const size = 1024 * 1024
	mem, err := mmap.MapRegion(nil, size, mmap.RDWR, mmap.ANON, 0)
	if err != nil {
		log.Fatalf("mmap: failed to allocate memory for buffer: %v", err)
	}
	log.Printf("memory %p", mem)
	err = mem.Unmap()
	if err != nil {
		log.Fatalf("mmap: failed to unmap memory for buffer: %v", err)
	}

}

This program works fine on linux, but under windows it prints this
X:\>go run mmap.go
2019/01/23 21:41:00 memory 0x31ba0000
2019/01/23 21:41:00 mmap: failed to unmap memory for buffer: FlushFileBuffers: The handle is invalid.
exit status 1

This is because FlushFileBuffers is being called on an anonymous handle.
This fixes it
--- a/vendor/github.com/edsrzf/mmap-go/mmap_windows.go
+++ b/vendor/github.com/edsrzf/mmap-go/mmap_windows.go
@@ -101,7 +101,7 @@ func (m MMap) flush() error {
 		return errors.New("unknown base address")
 	}

-	if handle.writable {
+	if handle.writable && handle.file != windows.Handle(^uintptr(0)) {
 		if err := windows.FlushFileBuffers(handle.file); err != nil {
 			return os.NewSyscallError("FlushFileBuffers", err)
 		}

Which I'll supply as a pull request in a moment

As this is fixed, this issue should be closed!


How to run guvicorn as a service?

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the FastAPI documentation, with the integrated search.
[X] I already searched in Google "How to X in FastAPI" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to FastAPI but to Pydantic.
[X] I already checked if it is not related to FastAPI but to Swagger UI.
[X] I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code
gunicorn main:app --workers 4 --worker-class uvicorn.workers.UvicornWorker --bind <IP_ADDRESS>:80

Description
Although the documentation (which btw is excellent) refers to deployment with Docker, and after online search, i cannot find how i will run guvicorn command:
gunicorn main:app --workers 4 --worker-class uvicorn.workers.UvicornWorker --bind <IP_ADDRESS>:80 as a service (without Docker).
Is there a recommended way for production? I found that in linux i can create a systemd service and run it always and after a crash. How i can to the same in windows?
And if i do that, do i need a reverse proxy (i like caddy) to run on top?
Operating System
Linux, Windows, macOS, Other
Operating System Details
raspberry pi os
FastAPI Version
0.79.0
Python Version
Python 3.9.12
Additional Context
No response

You can't run Gunicorn in windows.

found that in linux i can create a systemd service and run it always and after a crash. How i can to the same in windows?

I found that windows runs gunicorn only under Docker.
But in linux i should create my own systemd service and then run reverse proxy like caddy on top?

Is there a recommended way for production? I found that in linux i can create a systemd service and run it always and after a crash. How i can to the same in windows?

i use Linux or Mac as dev enviroment. but if you searching for auto restart after exception, you should consider supervisor, which is a python process monitor tool which guarateens your app wont stop.

I would prefer systemd as there are no dependencies and they exist by default in linux. There is no way to make this work?

there are no dependencies and they exist by default in linux

This is unlikely to be a true statement, regardless you can do as you described and create a systemd service to run that command.  Whether you need to run a reverse proxy in front depends entirely on your application.

i finally manage to make it with systemd for linux

If you managed to make a systemd file for your service then describe how you've done it. If someone stumbles upon this issue they will also know how to solve it.

i did something similar to this

I think it's good to post the solution in the issue. Linking to external sites poses the risk of site going down and making the solution disappear. After you post the solution you can also close the issue.

Another possibility I see, using msys2 or cygwin on windows, Grab the gunicorn source source code and compile it with simple command:
python setup.py install
and then run gunicorn application happily on windows

Thanks for the help here everyone! 👏  🙇
If that solves the original problem, then you can close this issue @ligantx  ✔

I think it's good to post the solution in the issue. Linking to external sites poses the risk of site going down and making the solution disappear. After you post the solution you can also close the issue.

You are right. So, here is my approach on how i created the systemd (spending a day of reading documentations etc. :P) with some additional useful info. I wish it will help other newcomers.
So, i wrote it the best i could. Please notify if you find any error, as i translated the instructions from my native language.


add cpp dir

Can you squash this (amend to previous is a weird name for a commit) and then, merge yourself.


Consolidate rdev explorer+portal deployment

TODOs:

[x] Remove happy code from single-cell-infra
[x] Move the relevant happy code from single-cell-infra to single-cell-explorer
[ ] Create a GHA workflow that does rdev deployment for both jobs (@elopez will do this)
[ ] Test the system
[ ] Update docs here and here

Blocked by a need for the Happy team to deploy two repos to the same rDev environment.

From last week's engineering meeting, there was some appetite from the engineers to get this addressed, since it would help simplify their pre-merge testing processes. Next step is to contact Happy team to ensure the necessary work is being tracked by them and then link this issue to their work.

I can keep working on this if the team thinks it's useful. It will require a few weeks as the Happy team will probably have to make a few changes to make this work.

Discussed with Emanuele and closing this since there is currently no need for consolidated deployment. A new issue will be created when/if the need arises.

Discussed with Emanuele and closing this since there is currently no need for consolidated deployment. A new issue will be created when/if the need arises.

Can you elaborate on "no need for consolidated deployment"? In light of our CI/CD plans, won't we need to be able to test Explorer<>Portal interactions in RDev? Are we just reasoning that we haven't need to test this recently?

Given how slow the explorer deployment is, I don't think we can justify doing this now. Happy/ECS didn't introduce any new tool that would help with this, so this is heavy lift work still. It's possible that Happy/EKS could simplify the work, but since we didn't adopt it, I don't think this can be scheduled anytime soon.

Thanks! I suppose we'll have to (re-)roll our own solution for Explorer<>Portal rdev testing when needed. Most importantly, each stack will continue to independently deployed, which is probably the right level of control for testing needs.


Label blocks found by FutureBit devices

Got a request from jstefanop of FutureBit to label blocks found by FutureBit's Apollo devices.
The devices mine solo. They put out between 2 and 9 TH/s depending on the device and settings but supposedly there are >100,000 of these things out there.
See example of coinbase tag here.

thanks you can close this, opened PR below
https://github.com/mempool/mining-pools/pull/51


Expand the current __toString behaviour

Python example:
a = dict()
a['a'] = 3
a['b'] = 2
a['c'] = 1

print a

{'a': 3, 'c': 1, 'b': 2}

I had this implemented very very early when php-ds was still private, where the structures would echo something similar to JSON. Values that could not be converted to strings had default fallbacks like object(stdClass)#1 and resource(type)#1.
I imagine the above equivalent would echo something like:
{'a': 3, 'b': 2, 'c': 1}

Does that mean that __toString() will be a debugging handler? I don't think that matches usual semantics, and we already have var_dump().

It's just a different level of verbosity, but you're right, it's not expected behaviour / usual semantics. 😢

@nikic to be honest usual semantics of php in this regard is quite inconvenient. Would it be that bad to change it for a more convenient approach by the cost of consistency with the language?

__toString is either an appropriate representation (think PSR-7 response body) or a debug handler. There is no default handler so I'm not even sure what "usual semantics of PHP" are in this context.
Instead of some custom string representation (mentioned above), what about having all structures just convert to their JSON representation, such that echo $structure is equivalent to echo json_encode($structure). That's both a appropriate representation and a debug handler at the same time.
FWIW, structures currently convert to this format: "object(Ds\Map)"


Avoid dividing by zero

Thanks @kemchenj, this is great!
Another way we might solve the divide by zero is to add a helper function:
        const normalize = (x: number, min: number, max: number) => {
            return (max - min < 0.0001) ? 0 : (x - min) / (max - min);
        };

And then modify the calls to use it:
            this.position[i] = pack111011(
                normalize(x[i], px.min, px.max),
                normalize(y[i], py.min, py.max),
                normalize(z[i], pz.min, pz.max)
            );

Because if max === min we probably don't want to perform a divide at all!


fix: use easycodingstandard instead of php_codesniffer, resolves #6

Codecov Report

Merging #8 (5c1b98a) into main (919dc8a) will not change coverage.
Report is 4 commits behind head on main.
The diff coverage is 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@             Coverage Diff             @@
##                main        #8   +/-   ##
===========================================
  Coverage     100.00%   100.00%
  Complexity        59        59
===========================================
  Files             10        10
  Lines            213       215    +2
===========================================
+ Hits             213       215    +2

Files
Coverage Δ

src/BlueskyApi.php
100.00% <100.00%> (ø)

src/BlueskyPostService.php
100.00% <100.00%> (ø)

src/Feed/Post.php
100.00% <ø> (ø)

src/HttpComponentsManager.php
100.00% <ø> (ø)

src/Response/ResponseTrait.php
100.00% <100.00%> (ø)

src/Richtext/AbstractFacet.php
100.00% <100.00%> (ø)

src/Richtext/FacetLink.php
100.00% <100.00%> (ø)

src/Richtext/FacetMention.php
100.00% <100.00%> (ø)


Enable development in Codespaces

Benefits of Codespaces:

All configuration is in source code and ready to be used - no set up needed
No need for any software to be installed and maintained locally
Can develop on any machine (including an iPad) - all that’s needed is a browser

Update after successfully starting the development environment on Codespaces:
In 0e7822c we upped the test timeout because the tests were running slowly on the remote (non Codespaces) devcontainer.
Good news is that they run much faster in Codespaces - c. 20 seconds instead of c. 50 seconds locally and c. 100 seconds on the remote (non Codespaces) container :-)


TII-271 - TurnitinOC - NextRetryTime is ignored when submitting a paper in status 4

If you submit a paper with an error that's recoverable (for instance, if you're working locally, disconnect your internet and run the queue job), you may find multiple submission attempts occur in quick succession, and your retry count may shoot up to the maximum.

The query in the implementation of crqs.getNextItemInQueueToSubmit is:
Criteria c = sessionFactory.getCurrentSession()
.createCriteria(ContentReviewItem.class)
.add(Restrictions.eq("providerId", providerId))
.add(Restrictions.in("status", new Long[]{ContentReviewConstants.CONTENT_REVIEW_NOT_SUBMITTED_CODE, ContentReviewConstants.CONTENT_REVIEW_SUBMISSION_ERROR_RETRY_CODE}))
.add(Restrictions.lt("nextRetryTime", calendar.getTime()))
.setMaxResults(1);
So this PR is redundant


The adds displaying

The adds displaying are not that catchy and the footer is way big than the page, I am a new contributor to GSSOC i can work on it, please assign this to me.

Please check i have made a PR, " Updated adds and main page"

hey, will my contribution count for leaderboard, please consider it, this is my first contribution, they said it starts from 7 pm onwards, can u do anything about it please.
can i make a pr again after 7 with same changes and can u merge it please

Sir can u assign me again, the adds became clumsy and main page is missing


T3.4 Presubmission check

Parent task: #4

Call the Leader Election contract before authorizing the submission of a batch.


[Bug] an exception occurred when restore job from a snapshot which previously sync multiple tables and set scan.incremental.close-idle-reader.enabled=true

Search before asking

[X] I searched in the issues and found nothing similar.

Flink version
1.7.1
Flink CDC version
2.4.0
Database and its version
MySQL 5.7
Minimal reproduce step

start a cdc job wich syncs multiple tables and has a parallelism more than 1
stop with a savepoint
restart the job with savepoint, add a new table and do not change the previous parallelism
an exception occurred:Can not restore vertex Source: MySQL Source -> Process -> (test: Writer -> test: Committer, test2: Writer -> test2: Committer)(cbc357ccb763df2852fee8c4fc7d55f2) which contain mixed operator finished state: [ALL_RUNNING, PARTIALLY_FINISHED]

What did you expect to see?
The job syncs all the tables, including newly added tables.
What did you see instead?
An Exception occurred:
org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster.
at org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.CompletionException: org.apache.flink.util.FlinkRuntimeException: Can not restore vertex Source: MySQL Source -> Process -> (test: Writer -> test: Committer, test2: Writer -> test2: Committer)(cbc357ccb763df2852fee8c4fc7d55f2) which contain mixed operator finished state: [ALL_RUNNING, PARTIALLY_FINISHED]
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1592)
... 3 more
Caused by: org.apache.flink.util.FlinkRuntimeException: Can not restore vertex Source: MySQL Source -> Process -> (test: Writer -> test: Committer, test2: Writer -> test2: Committer)(cbc357ccb763df2852fee8c4fc7d55f2) which contain mixed operator finished state: [ALL_RUNNING, PARTIALLY_FINISHED]
at org.apache.flink.runtime.checkpoint.VertexFinishedStateChecker$VerticesFinishedStatusCache.calculateFinishedState(VertexFinishedStateChecker.java:190)
at org.apache.flink.runtime.checkpoint.VertexFinishedStateChecker$VerticesFinishedStatusCache.lambda$getOrUpdate$0(VertexFinishedStateChecker.java:171)
at java.util.HashMap.computeIfAbsent(HashMap.java:1126)
at org.apache.flink.runtime.checkpoint.VertexFinishedStateChecker$VerticesFinishedStatusCache.getOrUpdate(VertexFinishedStateChecker.java:169)
at org.apache.flink.runtime.checkpoint.VertexFinishedStateChecker.validateOperatorsFinishedState(VertexFinishedStateChecker.java:70)
at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.restoreLatestCheckpointedStateInternal(CheckpointCoordinator.java:1737)
at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.restoreSavepoint(CheckpointCoordinator.java:1873)
at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.tryRestoreExecutionGraphFromSavepoint(DefaultExecutionGraphFactory.java:223)
at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:198)
at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:365)
at org.apache.flink.runtime.scheduler.SchedulerBase.(SchedulerBase.java:210)
at org.apache.flink.runtime.scheduler.DefaultScheduler.(DefaultScheduler.java:136)
at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:152)
at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:119)
at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:371)
at org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:348)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:123)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:95)
at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
... 3 more
Anything else?
中文描述：
从现象上看，使用scan.incremental.close-idle-reader.enabled=true参数，作业会自动释放资源，但是释放资源后，就不能再返回最初的多并行度状态了。而实际的生产需求往往是：高并行度同步一批表 -> 释放多余资源，恢复并行度1  -> 新增同步一批表，高并行度再次同步完 -> 释放多余资源，恢复并行度1   -> 循环此过程...
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

I carefully followed through this problem, and found the solution to avoid this exception. Here is the step-to-step solution:

After snapshot read done, the idle resource will be automatically released.
Stop the job with a savepoint
Modify scan.incremental.close-idle-reader.enabled = false, and restart the job with latest savepoint.
After at least one checkpoint done, stop the job with a savepoint.
Then you can add new tables in sync list. However, you ned to make sure scan.incremental.close-idle-reader.enabled=false. If it is set to true, you will find that the new added tables skip the snapshot reading step(you will not find keyword like: snapshot step 1 /step 2 /step 3 in taskmanager.log), and directly read the binlog.

@cygnusdark Thanks for reporting. We will take a look at this.

@cygnusdark This close idle readers feature is only supported in the flink 1.14 or newer version.
But I found another bug when testing.
Please update the flink version and use the new PR to test. Thanks~


Preserve exit code in bash hook

Preserve the exit code from the previous command the user ran when
exiting from the bash hook. This allows other commands in
PROMPT_COMMAND to handle the exit code from the original user command.

Thanks for this pull request. This fixes the problem when fasd is the only thing modifying the prompt command. But if there are more prompt commands then each one has to do such exit code passing. Is there a common agreed upon way to do this in the Bash community / users?

I'm not sure there is a common way to do this. I did some research and didn't find anything too helpful. Every prompt command does needs to do this to preserve the exit code, there's no other way. I did update my own handler the same way.

I add fasd to the precmd array provided to me by bash-preexec and in fact they do something very similar as you can see from the link. This commit seems more appropriate for a third party bash lib like bash-preexec or if you had a library loading themes etc it would and should have a function for installing to the PROMPT_COMMAND. I know bash-git-prompt provides one, so ... just fyi.

I roll my own prompt and don't use any bash frameworks right now.  There were two commands in my  PROMPT_COMMAND (fasd and one of my own), and I needed this fix in both places to properly handle the previous command's return code.
I'm not sure it's reasonable to ask fasd users to install bash-preexec just to work around this issue.

@clvv howdy! I'm one of the maintainers of https://github.com/Bash-it/bash-it/. As noted in the linked PR and commit above, we have to work around this issue in fasd in order to get the correct exit status to work with inside themes and plugins and such. I'd really like to see this merged so we can simplify our plugin.


Please remove synchronized from ECCurve.getMultiplier()

Hi,
I've noticed recently while working on a performance issue that must of the times the threads of my application were blocked waiting on ECCurve.getMultiplier(). All the threads of my application were using the same ECCurve - acquired via CustomNamedCurves.getByName - so the ECCurve instance was basically a singleton so all my threads were synchronizing on a singleton, albeit only for a very short time.
ECCurve.getMultiplier() has the synchronized keyword because it creates the multiplier lazily.
I can see two solutions to this problem:

create the multiplier eagerly
or remove the synchronised keyword from ECCurve.getMultiplier(). This would mean that if ECCurve.getMultiplier() is called from multiple threads at the same time when the multiplier is still null then they could all create a multiplier object, which is a bit wasteful, however all threads would create an instance of the same class with the same configuration, and the multiplier objects seem to be quite lightweight and stateless so this isn't a huge issue, and eventually the different CPU cores would converge to seeing the same multiplier objects. This might sound unsafe, but for example java.lang.String caches its hashcode internally without any sort of synchronization, and the worst thing that can happen is that it might have to calculate it multiple times.

Creating the multiplier eagerly would be the easier option, but I am not familiar enough with the library to tell whether it is a good idea or not, and I would be happy with either solution.
I am happy to raise a PR for either solution, but it would be good to discuss with a core developer which approach is better.
Thanks,
Gabor

Thanks for the report. We have now removed the synchronized keyword. We may in future look at eager initialization but the construction/configuration of ECCurve is quite complicated and there are backward-compatibility issues.

Thanks Peter, do you know when will this make its way into a release? Also, I don't seem to be able to find the pull request or commit that removed the synchronized keyword.

Ah, that makes sense. Thank you for your help

Hello,
I saw you removed the synchronized in code allowing multiple instances.
You can have both a single instance and good performance with double check locking.
The link below explain it
https://www.baeldung.com/java-singleton-double-checked-locking
However if you want best performances you should avoid volatile after the first access, see example below
public class DclSingleton {

    // can be cached in processor memory
    private static DclSingleton instance;

    // will only be set when the object is fully initialized
    // volatile will work correctly only after Java 1.4
    private static volatile DclSingleton volatileInstance;

    public static DclSingleton getInstance() {
        if (instance == null) {
        	if (volatileInstance == null) {
                    synchronized (DclSingleton.class) {
                        if (volatileInstance == null) {
                            volatileInstance = new DclSingleton();
                        }
                    }
        	}
        	instance = volatileInstance;
        }
        return instance;
    }

}

@gaellalire Thanks for the suggestion. Currently all ECMultiplier implementations are very lightweight, so in this case it seems better to simply not bother, but such a pattern could be useful elsewhere.

Please enlighten me. Why do this not break due to 'double checked locking
is broken'
https://www.cs.cornell.edu/courses/cs6120/2019fa/blog/double-checked-locking/
Wiki have a similar warning
Rgds Lars
tir. 13. okt. 2020 04.43 skrev Peter Dettman<EMAIL_ADDRESS>
@gaellalire https://github.com/gaellalire Thanks for the suggestion.
Currently all ECMultiplier implementations are very lightweight, so in this
case it seems better to simply not bother, but such a pattern could be
useful elsewhere.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bcgit/bc-java/issues/778#issuecomment-707449075, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AH6E4RYYXFWIWGHOIUBDJSTSKO5DNANCNFSM4QR35WCQ
.

@worsaae The answer is in your link, just search for "volatile".
Starting with JDK5 a volatile reference will not be set while the constructor job is not finished.
Your article also notes that volatile variable triggers cache coherence protocol, that why I'm using both a volatile and not volatile reference.


slot filling

Can I use intent to fill slots?

since this is a usage question, I suggest you ask in the forum

Since this has been moved to the forum, I'll close this issue.


Consider collaborating with Fluffy?

Hey!
It appears to me that you are shooting for a cross platform file system watcher. Great!
I have written a inotify library tinkershack/fluffy that might interest you. Please have a look?
Sorry about the unseemly 'issue'. Couldn't find other means to reach out.

Sorry, uninterested.


Exit code 1 for passed specs

After bundle update my tests start to return exit code 1 for passed specs.
brbrr$ MNZ_ENV=dev bundle exec rspec spec -t ~failing
# logs here
Finished in 12 minutes 46 seconds (files took 1.72 seconds to load)
409 examples, 0 failures

brbrr$ echo $?
1

My dependencies are:
  spec.add_dependency 'rake', '~> 11.2.0'
  spec.add_dependency 'rspec', '~> 3.5'
  spec.add_dependency 'rest-client'
  spec.add_dependency 'oauth', '~> 0.5.1'
  spec.add_dependency 'pry'
  spec.add_dependency 'oily_png'
  spec.add_dependency 'mysql2'
  spec.add_dependency 'allure-rspec', '~> 0.7.7'
  spec.add_dependency 'net-ssh'

And when I run only :smoke specs - I got correct 0 exit code.
brbrr$ MNZ_ENV=dev bundle exec rspec spec -t smoke -t ~failing

It's likely one of your dependencies is adding an at_exit hook and overriding the return status.

@JonRowe's idea is definitely a good possible root cause but there are others.  Whatever the cause is, it's something in your environment, not something intrinsic to RSpec itself (we all have projects where rspec runs successfully and exits with 0, so it's not a general RSpec problem).  Something in your environment is either directly causing this or is interacting with RSpec in a weird way to cause it.  Regardless, without more information, there's nothing we can do about this.  If you can pair this down to a reproducible example (preferably one that runs in less than 12 minutes...we don't have the time to debug such a slow test suite) we can try take a look.  That said, in these situations, usually the exercise of trying to pair it down to something reproducible surfaces the issue so I suspect if you go through the exercise of pairing it down you'll be able to find it yourself.
Closing, but feel free to continue the conversation.


How to use document nesting?

Let's we have
require 'pry'
require 'ostruct'
require 'representable/json'

hash = {
 "title" => "Roxanne",
 "details" => { "track" => 3, "length" => "4:10"}
}

class Song < OpenStruct
end

class SongRepresenter < Representable::Decorator
  include Representable::JSON

  property :title

  nested :details do
    property :track
    property :length
  end
end

puts SongRepresenter.represent(Song.new).from_hash(hash)
# #<Song title="Roxanne", track=3, length="4:10">

Why I don't see nested results( details wrapper) on results?( doc)

Because details is only a document element and not a property on your object, that's the whole point of ::nested. It helps you when the document has nesting but the actual represented object doesn't. Remember - a representer (without nested properties) is an instruction on how to render parse a "flat" object.

Ok, thank's for explanation !


Website look can be enhanced by adding the color full scroll bar which matches the website theme

The right scroll bar is default which doesn't look better and we have to and bg color and color to the scroll which matches the theme of the website

please assign this issue to me


alloc aligned memory for block bloom filter to work with jemalloc

Proposed changes
Issue Number: close #xxx
Problem Summary:
Describe the overview of changes.
Checklist(Required)

Does it affect the original behavior: (Yes/No/I Don't know)
Has unit tests been added: (Yes/No/No Need)
Has document been added or modified: (Yes/No/No Need)
Does it need to update dependencies: (Yes/No)
Are there any changes that cannot be rolled back: (Yes/No)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

LGTM @dataroaring rebase for p0 and UT


SQLITE_ERROR: no such table: scores

Describe the bug
Im trying to start this bot but sqlite does not create a table
To Reproduce
Steps to reproduce the behavior:

Open bot dir
type in console node Main.js
Bot successfully connecting to Discord
type any message in channel when bot run
see this error in console

 [FATAL] Possibly Unhandled Rejection at: Promise  Promise {
  <rejected> { [Error: SQLITE_ERROR: no such table: scores] errno: 1, code: 'SQLITE_ERROR' } }  reason:  SQLITE_ERROR: no such table: scores

Expected behavior
Bot work

the bot creates the db after you invite it to a discord server, if you invite the bot in a discord server while its offline this will happen.


Update Issue - Automate_undefined_99265

default description

Property
Value

Browser
Chrome 100.0

Operating System
Windows 10

Resolution
1280x588

Screenshot Attached
Screenshot URL

Updated Screenshot Attached


Generating truck objects from lists of points

Hi!
I'm working on a 2d constraint-solving CAD, and would like to implement the ability to export a linear/lathe extrusion of the drawing as an STL or STEP file. Truck seems like the best library for the job as far as I can tell.
Can you help me work out how to do this?
I think I need to create a Wire to represent the boundary geometry, creates two faces from this wire for the top and bottom faces, and then glue them together somehow, perhaps glue_at_boundaries ? And then presumably i need to use boolean operations to cut out interior geometry?
Lathe extrusion might be easier, if I can create a Wire maybe I can use rsweep ?

I ... think I got it working? could it be?
https://github.com/twitchyliquid64/liquid-cad/commit/a0b5fb50946e002efc4c5f3284570c7cec1861c7#diff-1d4b34bc50dcd41fc559b9563fbba91dfab991671496865a28cee406e0e976ef
PrusaSlicer says my generate STL has a number of errors - I'm not sure if there is a bug in truck or I'm doing something incorrect.

Hello,
Are you familiar with tsweep? It may be the function most relevant to your needs, if I understand them correctly. The key is to sweep a face, not a wire, so that the outcome is a solid, not a shell.
Cheers,
Matt

Thanks!
Yepp I'm sweeping a face to make a solid, the remaining issue is just that truck is producing an STL which PrusaSlicer says has errors, but I can file a separate issue for that!

For the sake of posterity, here's how I did it:

Make a Wire by collecting a Vec of lines which connect your points to form a closed loop (you could also use the other builder methods here, no need for everything to be a straight line)

let lines = vec![
  builder::line(builder::vertex(Point3::new(1.0, 0.0, 0.0)), builder::vertex(Point3::new(2.0, 0.0, 0.0))),
  builder::line(builder::vertex(Point3::new(2.0, 0.0, 0.0)), builder::vertex(Point3::new(2.0, 3.0, 0.0))),
  builder::line(builder::vertex(Point3::new(2.0, 3.0, 0.0)), builder::vertex(Point3::new(1.0, 0.0, 0.0))),
];
// NOTE: Don't actually create duplicate vertex objects, reuse ones that point to the same point

let wire: Wire = lines.into();

Convert into a face

let face = builder::try_attach_plane(&wires).unwrap();

Sweep to form a solid object:

let solid = builder::tsweep(&face, height * Vector3::unit_z());


migrate from pages route to app route for optimization and improve developer experiences

This PR migrate the project from the Pages Router to the App Router for better optimization, scalability, and alignment with modern Next.js features
Key Changes

Refactored all routes from the pages/ directory to the app/ directory.
Updated file structure to utilize nested layouts and server components.
Updated metadata handling to the new API.

Hello! @thibaudcolas , please let me know if its a good step towards enhancing the areweheadlessyet application

@mynk2611 I’m sorry but I don’t think this is a good fit for the project. This site is effectively in maintenance mode and due to be decommissioned, so there’s not much of a point in us making such big refactorings.
It’s a valuable change and I think we’ll make sure to use App Router and reference your code here for future Next.js sites built with Wagtail, but at this point in time for this project it’s not a good fit.


RSA token secret

Moved from main repo feathersjs/feathers#423
Using node-rsa, export it to keyfile, as default token secret rather than hardcode it in config file will be better option I think. Reduce unnecessary steps to create new (good) token, production ready, and rotating it in every x days/wk/mo will make it more secure.

Rotating the token secret will invalidate all existing token, but after we implement real refresh token support it won't really be an issue.

The new generator has a secret generator (see https://github.com/feathersjs/generator-feathers/blob/master/generators/secret/index.js) which I think is all we'll do for now.


Search in UI returns rows unrelated to keyword

This is on Kovan's network.
URL: https://v2.augur.net/
When I search for markets that contain "Augur" the results shown are unrelated to "Augur" keyword.
Screenshot attached for more info

This is a duplicate of #7593 -- we definitely need to see what we can do about search as there seem to be lots of little cases where things are just wrong.


[WFCORE-5306]: Clarifications needed for Message Translations II.

Fixing some incorrect messages.

Jira: https://issues.redhat.com/browse/WFCORE-5306
Related to : https://issues.redhat.com/browse/WFLY-750 and thus to https://github.com/wildfly/wildfly/pull/14031
Supersedes https://github.com/wildfly/wildfly-core/pull/4492

Core - Full Integration Build 10362 outcome was FAILURE using a merge of 7cc5b9a2102063a25bade19b634be4849adf805e
Summary: Execution timeout (new); tests passed: 6832, ignored: 113 Build time: 05:01:28


In-App Messaging not running on capacitor for iOS

Bug report

CHECKLIST

[x ] I have read the issue reporting guidelines

[x] I confirm this is a suspected bug or issue that will affect other users

[x] I have reproduced the issue using the example project or provided the necessary information to reproduce the issue.

[x] I have read the documentation thoroughly and it does not help solve my issue.

[x] I have checked that no similar issues (open or closed) already exist.

Current behavior:

Install the non CLI plugin normally then attempt to run using capacitor in Xcode. Using Cordova, IAM will fire up fine and execute normally as long as a non-cli build is used, which is expected. Has anyone successfully gotten Capacitor for iOS to work with this though? I have been successful with manually adding everything and modding for Android but still can't get iOS to actually use all of the frameworks yet. I know this isn't generally supported but where else can we even ask this question? Capacitor is heavily pushed now and unfortunately Firebase-X is still the only solution so here we are.

It's a clean project
Expected behavior:

In-App Messaging should load fine
Steps to reproduce:

Install the non CLI plugin normally for iOS then attempt to run using capacitor in Xcode
Screenshots

Environment information

Cordova CLI version

cordova -v
10.0.0

Cordova platform version

cordova platform ls
Installed platforms:
android 9.0.0
ios 6.2.0
Available platforms:
browser ^6.0.0
electron ^1.0.0
osx ^6.0.0

Plugins & versions installed in project (including this plugin)

cordova plugin ls

Dev machine OS and version, e.g.

OSX

sw_vers

Windows 10

winver

Runtime issue

Device details

e.g. iPhone X, Samsung Galaxy S8, iPhone X Simulator, Pixel XL Emulator

OS details

e.g. iOS 12.2, Android 9.0

Android build issue:

Node JS version

node -v

Gradle version

ls platforms/android/.gradle

Target Android SDK version

android:targetSdkVersion in AndroidManifest.xml

Android SDK details

sdkmanager --list | sed -e '/Available Packages/q'

iOS build issue:

Node JS version

node -v

XCode version

Related code:
insert any relevant code here such as plugin API calls / input parameters

Console output

console output

// Paste any relevant JS/native console output here

Other information:

I find Ionic's claim that Capacitor works seamlessly with Cordova plugins to be a bit of a marketing ploy to encourage developers to move over from Cordova to Capacitor. Let's not forget that Cordova is maintained by a collection of commercially uninterested developers whereas Capacitor is very much a proprietary product of Ionic from which they make money.
While it may be the case that simple Cordova plugins will work with Capacitor, more complex plugins such as this one or cordova-diagnostic-plugin (one of my others) will not work as they leverage Cordova-specific functionality such as hook scripts and plugin variables which are not supported by Capacitor.
I also find it bold of Ionic to suggest it is the responsibility of open-source Cordova plugin developers such as myself - who don't benefit commercially from their open-source work - to ensure their plugins work with an entirely different development framework than that for which they were designed, for the benefit of Ionic.
It's a bit like Apple saying it's the responsibility of Android app developers to fix their Android apps because they crash when run in an emulated mode on an iPhone.

I have to be honest here, I completely agree with you @dpa99c. As much as I like the idea of Capacitor, unless they take on the responsibility of actually creating non-enterprise solutions to issues such as Firebase, it's destined to never catch the traction they expect.
My main issue with this plugin is the inability to use Firebase In App Messaging when in CLI mode. So if I have to manually bust out Xcode to do a build for iOS anyway, then Capacitor would be nice to switch to for other benefits. However, I am currently stuck in the middle with products right now. Either use the CLI build with Fastlane / CI and lose some Firebase-X features or build it all manually and get all features. If IAM was fixed in the CLI builds, I would gladly just stick with Cordova knowing I don't see both projects solving this issue anytime soon. Capacitor 3RC fix it all? haha, not likely.

I find Ionic's claim that Capacitor works seamlessly with Cordova plugins to be a bit of a marketing ploy to encourage developers to move over from Cordova to Capacitor. Let's not forget that Cordova is maintained by a collection of commercially uninterested developers whereas Capacitor is very much a proprietary product of Ionic from which they make money. While it may be the case that simple Cordova plugins will work with Capacitor, more complex plugins such as this one or cordova-diagnostic-plugin (one of my others) will not work as they leverage Cordova-specific functionality such as hook scripts and plugin variables which are not supported by Capacitor.
I also find it bold of Ionic to suggest it is the responsibility of open-source Cordova plugin developers such as myself - who don't benefit commercially from their open-source work - to ensure their plugins work with an entirely different development framework than that for which they were designed, for the benefit of Ionic. It's a bit like Apple saying it's the responsibility of Android app developers to fix their Android apps because they crash when run in an emulated mode on an iPhone.

Well,, w(e)ho are the victims,.? those who rely on these plugin systems that get into these tussles..


TypeError: long() argument must be a string or a number, not 'JpegImageFile'

when I run :python download_and_convert_data.py
`>> Converting image 23751/82783 shard 11

Converting image 23801/82783 shard 11
Converting image 23851/82783 shard 11
None Annotations data/coco/train2014/COCO_train2014_000000167118.jpg
Traceback (most recent call last):
File "download_and_convert_data.py", line 36, in
tf.app.run()
File "/mnt/data1/daniel/tensorflow/_python_build/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "download_and_convert_data.py", line 30, in main
download_and_convert_coco.run(FLAGS.dataset_dir)
File "/mnt/data1/daniel/codes/FastMaskRCNN/libs/datasets/download_and_convert_coco.py", line 338, in run
'train2014')
File "/mnt/data1/daniel/codes/FastMaskRCNN/libs/datasets/download_and_convert_coco.py", line 299, in _add_to_tfrecord
img = img.astype(np.uint8)
TypeError: long() argument must be a string or a number, not 'JpegImageFile'`

why this happened?

Can you try changing the line 292 in /libs/datasets/download_and_convert_coco.py
img = np.array(Image.open(img_name))
to
img = np.asarray(Image.open(img_name))  ?
This might not be the solution, it is just a lucky guess

@kevinkit, unfortunately, guess does not help.

@kevinkit, I’ve tried， didn’t work. @CharlesShang would u  help us?

The error comes from the fact that if you open a file with
Image.open(...)
It creates an Image object which is from the PIL-library, which is called 'JpegImageFile' . The command
img.astype(np.uint8) refers to a numpy array. So it happens because at the point this command is called img is not a numpy array but a JpegImagFile

I solved that problem.

@Designbook1, please, tell how?

@vaklyuenkov  I just comment img.astype(np.uint8), and that works.

@kevinkit your answer is right, but how can I do to solve it? it means I need get the img's data?detail?

@Designbook1, thank you.

@Designbook1 I would love to tell you but I am not able to reproduce the problem. You can give the command np.array(Image.open(img_name)) a dataype so it should convert it automatically when opening the image with:
 img = np.array(Image.open(img_name),dtype=np.uint8)
However I am not quite sure if this will speed up the training process. In the original MASK-RCNN paper they trained for 2 days on a 8 GPU system. Therefore 7 days seems like an "okay" time to train for this architecture, when only one GPU is available.
Furthermore, the datatype may get converted when going on the GPU to a suitable representation (int16 / int8 - if your GPU does support it)

@Designbook1 Sounds like the input JPEG image is corrupted (possibly just truncated).
The imaging library can identify the file type as JPEG, but it can't read the data, so it can't convert the JpegImageFile into a numpy array.
Compare:
>>> im = Image.open("image.jpg")
>>> im
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=637x800 at 0x1086AFF50>
>>> np.array(im, dtype=np.uint8)
array([[[105, 100,  94],
        [107, 102,  96],
        [109, 104,  98],
        ...

versus
>>> im = Image.open("truncated.jpg")
>>> im
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=637x800 at 0x107073810>
>>> np.array(im, dtype=np.uint8)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: long() argument must be a string or a number, not 'JpegImageFile'

May be worth re-downloading the images. If you're feeling particularly motivated you could use (say) djpeg to find which images are broken.

A nice feature request would be a checksum to check if the downloaded data is ok.

@kevinkit Yeah, I guess, but it's pretty easy to run md5sum on the downloads. I'd prefer the project to focus on the tricky groundbreaking stuff :-)
For what it's worth, I report the following MD5 sums for the big zip files:
5750999c8c964077e3c81581170be65b  captions_train-val2014.zip
59582776b8dd745d649cd249ada5acf7  instances_train-val2014.zip
926b9df843c698817ee62e0e049e3753  person_keypoints_trainval2014.zip
0da8c0bd3d6becc4dcb32757491aca88  train2014.zip
a3d79f5ed8d289b7a7554ce06a5782b3  val2014.zip

@tweedmorris It seems your MD5 sums are the same with me. So I think there is no problem with the files.

An idea for further investigation:

Try to replace Image.open with cv2.imread, to do so you must install opencv (pip install opencv-python)

If this fixes the error there is an issue with PIL alternatively you could also try it with scipy to load the images

If this does not help make a try except block around the code and check which images give you the exception, either if it is all images or just some or even just one image.

Update all libs etc.

@kevinkit Good idea about the cv2.imread – be aware though that OpenCV will load the image in BGR byte order rather than RGB, so you'll need to switch the byte order, and perhaps make a copy of the re-strided array just in case there are any downstream functions that are expecting contiguous data:
img = cv2.imread("image.jpg").astype(np.uint8)[:,:,::-1]
img = np.ascontiguousarray(img)

Note that although OpenCV does provide a separate image I/O library, the imread / imsave functions in scipy are just fairly thin wrappers around the PIL / Pillow library so you would expect the same error as before:
>>> from scipy.misc import imread
>>> imread("truncated.jpg")
array(<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=637x800 at 0x11CB5FFD0>, dtype=object)

@Designbook1 Hope you've solved the problem, please do let us know.

Could it be that the EXIF data is corrupted? I had a similar issue with some images the other day. It turned out the EXIF data was corrupted. I was able to fix it by clearing out the EXIF data using piexif.
import piexif
piexif.remove("pathtofile")

@kevinkit @tweedmorris  your answers have helped me to solve the problem. Thank you very much!

@Designbook1 I want to know how to solve the problem,thank you.

I also have this problem, and the file is different from @Designbook1  (it's around 70,000 and his is around 23851). I don't believe the downloaded file has problems, but during the unzip process my disk is full so I have to switch to another disk which might cause some of the images to be corrupted - so I just unzip the data again and it looks good now.

I had same problem too, but i tried deted the dataflie and properly unzip(one by one) and its work. Thank you, @xiaoyongzhu

Please tell me how to solve the problem,thank you very much. @Designbook1

@ty0803 u try to delete the unzipped files.
then unzip all files one by one again . It works for me.

OK! I try it.Thank you very much!

@WKChung1028 OK!I try it.Thank you!

you can remove this image by follow code:
img = Image.open('img_path')
img1 = np.array(img)
if len(img1) == 3:
.......
elif :
continue
this code maybe work.


MX_DEVICE_VERSION is missing

Helllo. I tried to use this pack with L0 and F4 DFPs and CUBE MX output and result was this error. All packs are latest, Keil and CubeMX too. MX_Device.h doesn't have this macros

Hi @MaxFlatline ,
the new concept depends on support in the DFP and also latest cmsis-toolbox.
Can you try with https://github.com/Open-CMSIS-Pack/STM32F4xx_DFP/releases/tag/v3.0.0 and and https://github.com/Open-CMSIS-Pack/cmsis-toolbox/releases/tag/2.6.1
New DFP is not jet available for the L0 series.

Can this issue be closed @MaxFlatline ?

If the pack will be changed not to replace working solutions, then yes. Right now it acts as if it supports every single ST MCU and disrupts my workflow


Add Year column to "AKFIN_SPECIMEN"

Issue

I was pulling the length-weight data for my assessment from the "AKFIN_SPECIMEN" table and noticed there is no column that indicates the year in which the data was collected. This information would be valuable to have since it would allow assessment authors to see if there has been a change in the length-weight relationship over years. I was wondering if it would be possible to add this column to the "AKFIN_SPECIMEN" table?

Hi Lee,
I agree this information is important. To reduce redundancies, only the key bits of info for a table are included (to the extent possible) and the idea is to join other akfin_* tables to get axillary information (like year, vessel, region, lat/lon, etc.). For example, the YEAR column can be extracted from akfin_cruise (with key field cruisejoin) and it can then be joined with the akfin_specimen table via the hauljoin key. Try out this example below and you can add further WHERE clauses after the two JOIN lines
select * from gap_products.akfin_specimen
join (select hauljoin, cruisejoin from gap_products.akfin_haul) using (HAULJOIN)
join (select cruisejoin, year from gap_products.akfin_cruise) using (CRUISEJOIN)

Hey @Lee-Cronin-Fine-NOAA ,
Just checking in on this issue. Did you find this code helpful or is there something I can clarify further?
Thanks,
Zack

Hey Zack,
The code you gave me worked.
Thanks for the help and checking in.
Sorry for the late reply.
Lee
On Mon, Jul 1, 2024 at 5:37 PM zoyafuso-NOAA @.***>
wrote:

Hey @Lee-Cronin-Fine-NOAA https://github.com/Lee-Cronin-Fine-NOAA ,
Just checking in on this issue. Did you find this code helpful or is there
something I can clarify further?
Thanks,
Zack
—
Reply to this email directly, view it on GitHub
https://github.com/afsc-gap-products/gap_products/issues/46#issuecomment-2201541746,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFP2M3UXEK2SQD37L5M3IFLZKHY3HAVCNFSM6AAAAABI5D6CMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBRGU2DCNZUGY
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Lee Cronin-Fine, Ph.D.
Research Mathematical Statistician
Alaska Fisheries Science Center
NOAA Fisheries Service
Tel:<PHONE_NUMBER>
Email: @.***