[$1000] iOS - Attachments - Uploading image throws error "Unsupported PDF format"

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to Settings > Camera > Format > ensure that "high-efficiency" is enabled
Open the iOS app
Visit any DM
Take a picture with the native iOS camera, ensuring that "live" photo is disabled
Tap the grey plus icon to the right of the chat input
Select a image from your camera roll (for iOS should be HEIC files by default)
Upload the image

Expected Result:
The image should upload natively with no errors
Actual Result:
RBR error appears, with the message "Unsupported PDF format"
Workaround:
Unknown
Platform:

Where is this issue occurring?

iOS

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/93399543/198847076-65bd917b-150a-41cb-8da0-a5e5bb649a6a.mp4
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

having engineering take a look

I don't have a physical iPhone to repro this, but this does look to be fixable by a contributor. Marking external.

@dhairyasenjaliya Thanks for flagging. I commented in the linked issue immediately above this post.

@mmarceau Your proposal looks let's make it a new method isAttachmentDownloadable and add a checks with relevant comments for understanding!

@kbecciv

Try to upload unsupported file type and long type on the error message--> Download

Can you send a screen recording for this step and a file please!

@Santhosh-Sellavel Please attached video, you need to turn off live mode when taking a picture before attaching in NewDot app.
https://user-images.githubusercontent.com/93399543/200347442-09252155-cace-4fb5-b1d1-ec93b01fa217.mp4

This issue still seems like a duplicate of the crash that we're also discussing in https://github.com/Expensify/App/issues/10907. Right?

@Gonals Do you mind chiming in on my last question? Looking at the reproduction steps, I'm failing to see the difference.

@JmillsExpensify Uploading is a duplicate of the other issue, but this here https://github.com/Expensify/App/issues/12283#issuecomment-1305774920, where we show a preview of the image they selected and followed by an error on sending is weird, I think we should do something about it

Yes, I think @Santhosh-Sellavel. We seem to be dealing with two different problems here:

Error after showing the image preview
Crash downloading the attachment after failure (duplicate)

Nice, yeah agreed. I think one is also a duplicate as well though. Let me try to find this issue, this is related to changes to the attachment flow, where everything is running through Web-PDFs.

Error after showing the image preview - @Gonals
I think one is also a duplicate as well though. Let me try to find this issue, this is related to changes to the attachment flow, where everything is running through Web-PDFs. - @JmillsExpensify

So this issue can be closed can it, if it's being taken care of elsewhere?

Updated this issue to remove the duplicate discussion around the crash. That's handled in https://github.com/Expensify/App/issues/10907 moving forward. This issue is now focused on the on the "Unsupported PDF format" RBR error message moving forward.

Separately, @youssef-lr still would like confirmation if you are aware of this issue or working on any PRs elsewhere that resolve this?

Updated this issue to remove the duplicate discussion around the crash. That's handled in #10907 moving forward. This issue is now focused on the on the "Unsupported PDF format" RBR error message.

Thanks, @JmillsExpensify, now its more clear!

Also an update on this issue: It's pretty tricky. I tested two flows, and received a different result each time. Basically, if you follow the testing flow on mWeb, you receive no error. It's also possible to see the (HEIC) attachment after upload. However, if you test on native iOS, then the reproduction steps above produce the error. That makes me think that we're doing some combination of file processing in one flow that doesn't hit the error, whereas the native flow is treating the file differently and so, triggers the error.

Now that we're aligned on the issue/next steps, I'm doubling the price of this issue!

Still waiting on proposals.

@Gonals To help us clear out the large backlog of /App bugs, we're putting the spotlight every bug in the repo already than 4 weeks old. To help unblock the roadmap and get our bug pipeline back in equilibrium, can you:

Decide whether any proposals currently meet our guidelines and can be approved as-is
For any that can't, please take this issue internal and treat it as one of your highest priorities
If you have any questions, don't hesitate to start a discussion in #bug-zero

Hey @puneetlath! So we don't plan to increase the price of this? It got doubled just a few days ago

Ah actually, my mistake. This isn't 4+ weeks old yet, so it doesn't have to be taken internal. I think it might be worth you asking if another engineer wants to trade you for this issue though, since you don't have iOS.

Proposal
diff --git a/src/components/AttachmentPicker/index.native.js b/src/components/AttachmentPicker/index.native.js
index 418be07b8..aa965e5e7 100644
--- a/src/components/AttachmentPicker/index.native.js
+++ b/src/components/AttachmentPicker/index.native.js
@@ -34,6 +34,7 @@ const imagePickerOptions = {
     saveToPhotos: false,
     selectionLimit: 1,
     includeExtra: false,
+    quality: 0.95,
 };

 /**

Details
My proposal is more like a workaround, the bug is in react-native-image-picker
I think we can proceed with the workaround without waiting for the react-native-image-picker.

Strictly No workarounds, if the issue exists at the library, it's better to address it there! If you have a solution to solve please post it here, i.e at the library itself!

@Santhosh-Sellavel I have submitted a bug report already.
I have found the root cause, proposed a solution for the library and proposed a workaround here.

@Santhosh-Sellavel I have submitted a bug report already. I have found the root cause, proposed a solution for the library, and proposed a workaround here.

If you have found the root cause then why was the workaround submitted here?

@Santhosh-Sellavel
From Expensify App perspective react-native-image-picker is the root cause. It's related to the quality prop. I know that the issue exist for setting the value equal to 1.
From react-native-image-picker perspective I don't know the root cause, I just submitted a suggestion to where to start looking in the code.

While the conversation above continues, doubling the price of this issue once more! We're now at $1,000 💰

The issue seems to be a regression from https://github.com/react-native-image-picker/react-native-image-picker/pull/2006
I have asked if this can be reverted, then it should be sorted out, except that the returned file will seem to be double but that's just due to the conversion from heic to jpeg I suppose. We may also want to keep the quality at 0.95

except that the returned file will seem to be double but that's just due to the conversion from heic to jpeg I suppose

I dont think reverting that fix is great, that was introduced to fix another issue. Doubling the size is not what we want and I believe it doubles the size also for images which are jpeg in the gallery

@mountiny What I'm saying is that maybe the PR didn't fix any issue in the first place and the returned size was actually the correct size.
I have downloaded the image that seems to have the size doubled, and the size was in fact correct.
Uploaded image at quality 1 => size 9.6MB (this seems to be double but it's correct) => downloaded the image and the size is in fact 9.6MB ==> Correct
Uploaded image at quality 0.95 => size 5.1MB => downloaded the image and the size is 5.1MB ==> Correct

@mountiny What I'm saying is that maybe the PR didn't fix any issue in the first place and the returned size was actually the correct size.@s77rt
Which platform did you check?

@Santhosh-Sellavel Your question is not quite clear, check what exactly?

@Santhosh-Sellavel Ah, you quoted the wrong part, your question was out of context.

I have modified the code (basically a revert) so I can get it working.
Send the image using quality 1 (on iOS)
Downloaded the image that I sent (on Linux, but this does not matter, it just happen to be Linux)
Verified that the reported size on iOS 9.6MB is indeed the size of the photo that I downloaded

Here is the image, you can download it and you will see it's indeed 9.6MB. There was nothing wrong it just happen that the conversion at quality 1 is so inefficient that the size seems to be double.

Send the image using iOS and download it on any platforms size will change.

Send the image using iOS and download it on any platforms size will change.

Isn't that what i just said i did?

So with the revert of the PR you mention did not solve any issue, if you upload JPG file and then download it again, is the size not doubled? Are you saying this only happens for HEIC images because of the conversion?

@mountiny Why do you think the size will double? it's just impossible to get the size doubled on download. You download what you upload.
Let me make it clear:
If you upload the file at quality 1 (original size: 5.1MB), you will be uploading the converted file and since the conversion is not that good, the file size will be much larger than what you'd expect (9.6MB).
The reported size by react-native-image-picker is correct (9.6MB).
If you download it, you will of course get the same amount of data that has been uploaded, thus the same file size (9.6MB).

5.1 MB shouldn't become 9.6 MB that's the problem that is addressed in the PR https://github.com/react-native-image-picker/react-native-image-picker/pull/2006

@Santhosh-Sellavel it's not that 5.1MB is magically set to 9.6MB it's the returned data that's much larger as I said due to the conversion.
With the PR, the returned size at quality 1 is about 2.6MB which is totally wrong because at quality 0.95 it's 5.1MB and that does not make sense.

it's the returned data that's much larger as I said due to the conversion.

Yes, the scenario I am asking about is if we upload JPG file, then there should be no conversion, why would we convert JPG. I understand the HEIC will get converted because Apple but what about normal formats like PNG or JPG, those should not change type and hence quality 1 should keep the same filesize when uploaded @s77rt

@mountiny Ah okay, now I see the problem, every file is being converted but the file size correct.
Will have to look into this more

Okay so far I found the root cause. I can apply this fix:

Process other files just normally (jpg, png, ...) (without extra conversion / doubling the size)
Convert HEIC files to jpeg

However the converted HEIC file size is almost the double (5.1 MB TO 9.6MB)
I think this can be improved, because browsers can select HEIC files and convert them without that much diff, maybe they don't set the quality to 1?
Will update you shortly

Proposal (Updated)
diff --git a/node_modules/react-native-image-picker/ios/ImagePickerUtils.m b/node_modules/react-native-image-picker/ios/ImagePickerUtils.m
index 620ddc9..3afe9d6 100644
--- a/node_modules/react-native-image-picker/ios/ImagePickerUtils.m
+++ b/node_modules/react-native-image-picker/ios/ImagePickerUtils.m
@@ -58,7 +58,7 @@
         configuration = [[PHPickerConfiguration alloc] init];
     }

-    configuration.preferredAssetRepresentationMode = PHPickerConfigurationAssetRepresentationModeCurrent;
+    configuration.preferredAssetRepresentationMode = PHPickerConfigurationAssetRepresentationModeAutomatic;
     configuration.selectionLimit = [options[@"selectionLimit"] integerValue];

     if ([options[@"mediaType"] isEqualToString:@"video"]) {

Update
Fixed the issue on the react-native-image-picker level
Details
The problem is in PHPicker. The selected file per default will have it's original format, in this case it's heic.
And we are uploading this file directly to the server which is not supported.
In my proposal I changed the preferredAssetRepresentationMode from PHPickerConfigurationAssetRepresentationModeCurrent to PHPickerConfigurationAssetRepresentationModeAutomatic This setting will automatically select the appropriate format; for heic files we will get jpeg. This is also the behaviour of UIImagePickerController (prior iOS 14) so this issue only effect iOS 14+ users due to said problem
If it is not clear, the library uses two image pickers. UIImagePickerController and PHPickerViewController. Uses PHPickerViewController if available (which is available on iOS 14+). else fallback to UIImagePickerController.
In my proposal I'm matching the behaviour of PHPickerViewController to UIImagePickerController.
PS: This needs to be tested on a real device, iOS simulator does not properly support phpicker heic to jpeg format.

Additional info and correction about the file size:
2.8MB is the file size of the heic file
5.1MB should be the expected file size of jpeg (this will be supplied directly from iOS, we won't handle any conversion from our side)
9.6MB is the size of the converted heic file to jpeg using UIImageJPEGRepresentation (very inefficient)
Meaning, with my proposal you should expect the file size of the image (pink and yellow flowers) to be 5.1MB

General update: We're still working through the issue/cause and related proposals. Latest proposal is immediately above for review.

@s77rt Thanks for the proposal.
I'm not sure this is the right path

In my proposal I changed the preferredAssetRepresentationMode from PHPickerConfigurationAssetRepresentationModeCurrent to PHPickerConfigurationAssetRepresentationModeAutomatic This setting will automatically select the appropriate format; for heic files we will get jpeg. This is also the behaviour of UIImagePickerController (prior iOS 14) so this issue only effect iOS 14+ users due to said problem

Can you elaborate more on how this address the issue? What really happens when switching?
Have you submitted the PR on react-native-image-picker repo?  If not, please raise a PR to let's hear from them too!

@Santhosh-Sellavel
I think I have already stated the reason for switching. Which is to use the JPEG format and not the HEIC format.
It's either:

Use HEIC then convert it to JPEG using UIImageJPEGRepresentation
Use directly the provided JPEG format

The first approach is inefficient and results in a bigger file unless we carefully choose the quality.
The second approach is better since the conversion is done by iOS and the quality is (I suppose) chosen automatically.
I have proposed the solution on the created issue https://github.com/react-native-image-picker/react-native-image-picker/issues/2052
I have also proposed an alternative solution that follows the same principle

Summary:
We are getting the HEIC file and uploading it to the server which is unsupported thus the error.
We should be getting the JPEG file and upload it instead. Changing preferredAssetRepresentationMode will get us the JPEG file directly without the need of an explicit conversion.

@s77rt
It's either:

Use HEIC then convert it to JPEG using UIImageJPEGRepresentation
Use directly the provided JPEG format

Can you share any supporting doc for this?

@Santhosh-Sellavel
For what exactly? the statement is clear, does not need an external docs.
PHPickerConfigurationAssetRepresentationModeCurrent will select the original format for every file.
PHPickerConfigurationAssetRepresentationModeAutomatic will select the format automatically
PHPickerConfigurationAssetRepresentationModeCompatible will select the most compatible format
You don't want to use PHPickerConfigurationAssetRepresentationModeCurrent because it will give you the HEIC file format.
You have to convert it unless you want to support HEIC files internally or make the conversion server-side.
or
Just get the JPEG format in the first place and upload it to the server
I'm not sure what part requires docs

@Santhosh-Sellavel
For what exactly? the statement is clear, does not need an external docs.
PHPickerConfigurationAssetRepresentationModeCurrent will select the original format for every file.
PHPickerConfigurationAssetRepresentationModeAutomatic will select the format automatically
PHPickerConfigurationAssetRepresentationModeCompatible will select the most compatible format
You don't want to use PHPickerConfigurationAssetRepresentationModeCurrent because it will give you the HEIC file format.
You have to convert it unless you want to support HEIC files internally or make the conversion server-side.
or
Just get the JPEG format in the first place and upload it to the server
I'm not sure what part requires docs

From where we can learn this?

I have provided links already for the preferred selection modes.
https://developer.apple.com/documentation/photokit/phpickerconfigurationassetrepresentationmode/phpickerconfigurationassetrepresentationmodeautomatic?language=objc
https://developer.apple.com/documentation/photokit/phpickerconfigurationassetrepresentationmode/phpickerconfigurationassetrepresentationmodecurrent?language=objc

@Gonals @Santhosh-Sellavel what do you think here, bumping this issue.

@mountiny Looks good to me so far, yet to test this one on a real device.
Meanwhile please @s77rt, please bump the maintainer of the library. I would suggest opening up a PR there instead of an issue report, thanks!

@Santhosh-Sellavel The library does not seem very active it may be better to apply a patch or fork the repo if it took to long
Let's test the proposal first

Let's try it out first if unresponsive for over a week, let's do it on a fork!

That plan sounds fine to me, though @Gonals please weigh in tomorrow when you're back on.

Here is the PR for reference
https://github.com/react-native-image-picker/react-native-image-picker/pull/2056

That sounds good to me too, but let's not wait a week. If we don't hear back in a couple of days, let's go ahead and create a fork

Forking is quite annoying to maintain, this maintainer has helped us before, I will ping them

Agreed, very non-ideal.

The PR has been merged
Now we need a tester

We will sponsor the library as we had couple of PRs in there.
@Santhosh-Sellavel do you have Android and iphone to test this on real device once we will be in a review?
@s77rt We need a PR, can you create it with the new version and test also for the regressions/ previous issues we had in the library

Applied in Upwork (+50% bonus) in case this fixed the issue
PR is ready

PR is failing for android, so I went ahead and tested on other platforms, seems we don't allow this format on the web/desktop either.
https://user-images.githubusercontent.com/85645967/205384598-bfa7454f-2e38-4f1e-bd70-ba1f29c8b0d7.mov

@Santhosh-Sellavel You are testing it the wrong way.
You are not supposed to select a raw .heic file that way.
What I meant is to select an image that is internally stored as heic.
You are only supposed to test on iOS

@s77rt @Santhosh-Sellavel The issue is mentioned as iOS but we should aim to solve this for Android too, the usecase is edge case so I would say we should merge the PR which fixed iOS where this is a blocker to sending images and then we should create a follow up PR to fix Android.
This issue mentions iOS, but I would argue in this case it is obvious and makes sense to fix any native mobile too.
@Santhosh-Sellavel the issue shown above is not related to this change. cc @youssef-lr afaik you have worked on accepting the heic on web, do you know what is the state of that?

@mountiny I don't see an issue with Android. Can you upload a video?

Just coming from the video @Santhosh-Sellavel has shared here, it is using jpeg image, this should be working on main, should it not?

The issue is mentioned as iOS but we should aim to solve this for Android too, the usecase is edge case so I would say we should merge the PR which fixed iOS where this is a blocker to sending images and then we should create a follow up PR to fix Android.

I agree with this.

@Gonals @Santhosh-Sellavel What are our next steps for this issue? Any movement on the Android consideration?

Thanks for the bump! @Gonals Please could you add help wanted label again!

@JmillsExpensify @Santhosh-Sellavel @Gonals Shouldn't we handle the android case in it's own separated issue? as a feature request?

Since the issue existed both in Android and iOS to begin with, and the whole conversation is here, I think it makes sense to keep using this issue

@Gonals What about the compensation is it 2x $1000 in this case?
Are we expecting two contributors and two PRs for one issue?

Discussing next steps in Slack.

Readding the External label will create a new job in Upwork @Gonals just noting. I think you might have wanted to use Bug label instead :)
@s77rt It will be a new job and I think it might have to be internal job after all as it might need access to API.

I'm returning on the 20th, so I'll circle back then for the regression test and payment.

It sounds like @JmillsExpensify has this one handled for BZ, so I'm going to un-assign, as it looks like I was just assigned to help cover OOO. LMK if there's anything I can do to help though!

Thanks @joekaufmanexpensify i really appreciate the help while OOO. I'll circle back to this one in the coming hours to close the loop. 🙌

Ok I'm circling back on this one. We have a decision to make here before going through the Upwork hire process and issuing payment, specifically:
@mountiny and @Santhosh-Sellavel I think we need to address @s77rt's question in the PR as to whether the 50% bonus applies. I think that his perspective is that it could have been merged sooner but wasn't due to the discussion that occurred during the testing process. However, I think another perspective is that it wasn't, and due to the through testing by C+ to make sure we got the scope of the issue right, as well as didn't introduce regressions, it took longer. I think I am more amenable to the latter argument. Please weigh in regardless and let's treat this objectively and fairly thanks!

Additionally, @Santhosh-Sellavel (@mountiny - I'm pulling you in given that Alberto is OOO until the New Year and you have context from the PR), do you mind handling the first three items on the Bug Zero checklist? Given the upstream fix and considerations, I think we should be good on those, though let's confirm either way. Then I can circle back on the regression test pulling from the linked PR.

Wait, I'm OOO un til New Year? When did that happen? I'm only OOO some days! 😆
@JmillsExpensify, I'm with you on this one. I don't think I've seen other cases where the bonus was requested when the PR took longer to merge, but, in my opinion, discussions are part of the PR process, and this was not something that got "stuck" without movement for a while. There was back and forth and that's part of the process and, sometimes, PRs take longer for that 🤷 .

I agree with both of you, in this particular case I think the delay was not due to the lack of response from us but more like a normal discussion about the outcome of the PR.
@Gonals will you be able to handle the checklist since you are assigned here and apparently not ooo 😂

Sure thing!

Thanks for weighing in! I'll move forward with the following payments:

Contributor: @s77rt $1,000
Contributor+: @Santhosh-Sellavel $1,000

Also @s77rt while I appreciate you want to be optimistic and submit for the bonus when the PR is up, I did want to reiterate that this is not our process per our Contributing.md. The bonus is applied and eligible when the PR is merged, not when the PR is up for approval. Let's please standardize on not asking for the bonus ahead of time moving forward. Thank you.

Quick update: Contract extended to @s77rt and @Santhosh-Sellavel invited since he wasn't on the job yet.

All contributors paid out. Closing this issue and I'll link the regression test very shortly.

Closed the loop on the regression test. It is linked immediately above!


Accessibility: Pagination links cannot receive focus

The pager currently doesn't add href or tabIndex to its links, which makes it substantially more difficult to navigate pages with the keyboard.
Proposed Solution
Add hrefs to the links in paginator.jsx and update the event handlers to call preventDefault.

ooh, yeah, absolutely. What would you like to see in the hrefs? just #?

That would be the simplest solution imho, otherwise something like the way react-bootstrap does it in SafeAnchor, but that might be overkill.

Definitely like the idea of adding role="button" to all the pagination links, too.


[RLM-322]Nova hostname fix

Since nova-compute would delete nova.compute.node due to hostname
updated, we fix nova to certain hostname to prevent this happening.

Closed for better and more efficient solution found.


docs: Graceful shutdown for Envoy proxy

Create design documentation covering graceful shutdown behavior as implemented in #2633 with example configuration to adjust any exposed knobs.

This woujld be nice. I'm looking at how to do this. So far its not graceful by default. my tests show dropped connections when rolling restart with envoy gateway, but not with nginx ingress controller

should be graceful by default, @davem-git can you create a new issue to describe how you are running your test and in which environment ?

i created this https://github.com/envoyproxy/gateway/issues/4002


Alignment with fractal-server 2.9.0

Checklist before merging

[X] I added an appropriate entry to CHANGELOG.md

Closes #638

Deactivate:

Reactivate:


use reserved spelling for inline assembly

Use the reserved spelling for asm as it is a non-standard compiler
extension.  NFC.

@swift-ci please test

CC: @MadCoder @das @parkera

Seems fine to me but @MadCoder and @das should be the ones to approve in this repository.

@MadCoder updated to use __asm__ __volatile__ for the non-label cases.  I've merged the prefix change as well.

@swift-ci please test

Actually, let me merge in the reserved spelling for __typeof__ as well.  Sorry, I have a number of patches that I've had laying around that I'm trying to get through.  These are all intended to help setup everything to revive the Windows port.

this is fine :)

@swift-ci please test

@swift-ci please test

@MadCoder anything else for this change or good to merge?


stampit v3 rfc: thru

(examples used reflect Stampit v2 API)
Stampit currently has support for init, which allows you to chain initialization functions together when composing stamps:
const Stamp = stampit({
  // "option" form
  init() {
    this.foo = 'bar';
  }
})
  // function-chain form
  .init(function() {
    this.foo = `${this.foo}-baz`;
  })
  // function-chain form
  .init(function() {
    this.foo = `${this.foo}-quux`;
  });

const obj = Stamp();
obj.foo // 'bar-baz-quux'

Each init function is called in the order it was defined.  Note that there are three stamps being created above; each function-chain call to init() returns a new stamp.
init is called with a "context" object, containing properties instance, args, and stamp, which are an Object, Array, and Function, respectively.
init only applies to object instantiation.  What Stampit currently lacks is a (convenient) way to assert control over the composition of stamps.
An example use case is non-enumerable static properties.  e.g.:
const fooKey = Symbol();
const Stamp = stampit({
  static: {
    [fooKey]: 'bar'
  }
});

When the above stamp is used to compose another stamp--in any way--the property keyed on the Symbol fooKey is lost, because functions such as Object.assign() and the like ignore non-enumerable properties by definition.
const fooKey = Symbol();
const Stamp = stampit({
  static: {
    [fooKey]: 'bar'
  }
});
Stamp[fooKey] // defined

const AnotherStamp = Stamp.refs({
  baz: 'quux'
});
AnotherStamp[fooKey] // undefined

Indeed, the only way to ensure the property keyed on Symbol fooKey makes it into composed stamps is to monkeypatch each of the following properties in the stamp's fixed property: static, init, props, refs, methods, create and compose to explicitly add the property to the new stamp:
// untested, but something like this
_(Stamp.fixed)
  .filter(_.isFunction)
  .forEach((func, key) => {
    Stamp.fixed[key] = function(...args) {
      const retval = func.call(this, ...args);
      retval.fixed.static[fooKey] = 'bar';
      return retval;
    });
  })
  .value();

I assert the above is not user-friendly, and probably doesn't scale well.  This leads me to the following proposition:
thru
thru is an option and a chainable function.  In "option" form, the property name is thru and the value is a user-defined function.
This user-defined function is called whenever a new stamp is composed from the stamp in which thru is defined.  The user-defined function is called with a single parameter, stamp, which is the factory function.  It should be called with the context of the stamp itself; much like in functions defined by static.

The user-defined function should be called after the call to the method which creates a new stamp; if methods() is called, the user-defined function would be called with the stamp after a new stamp is created with the new methods.  However, I could envision a scenario in which a user wanted to run the function before methods() is called; to that end, perhaps thruBefore() and thruAfter() are appropriate.

thru simplifies the above two examples into something like this:
const fooKey = Symbol();
const Stamp = stampit({
 thru() {
   this[fooKey] = 'bar'; // or maybe this.fixed.static[fooKey] = 'bar'; not sure
 }
});

I have the following questions:

Is thru an appropriate method name?
Should we have thruBefore and thruAfter instead?
Should thru expect a returned value?

vs ES2015 Classes
ES2015 classes are not particularly friendly to static properties, but the following still works:
const symbol = Symbol();
class Foo {}
Foo[symbol] = 'bar';

class Bar extends Foo {}
Bar[symbol] // 'bar'

I believe thru--or a similar proposal--is necessary for stamps to offer an equivalent feature set to ES2015 classes.
The above may illustrate handling of non-enumerable properties is a weakness of Stampit and/or composable object factories in general.  I believe at the very least, more consideration around the topic is necessary.

@ericelliott will probably spit up his coffee at this idea, but the most direct route from A to B might actually be to leverage ES2015 classes in some capacity under-the-hood to ease working with non-enumerable properties.  :imp:

I had a proposal to accept ES6 classes as stamps to the "compose" function:
class C {};
const Foo = SomeStamp.compose(C);

But that's a hard task.

First of all, I really-really like the thru idea. I will add up my thoughts later.
But first, I'd like to say that stamp specs. should support Symbols out of the box. I.e. Symbols should be treated as usual properties. @troutowicz your thoughts?

thru can be useful in some scenarios. I met so far:

debugging, understanding which stamps my object consists of (I got very confused once, accidentally composed same stamp twice).
debugging, finding out which stamp overrode my property and why (solves some other people "complains").
statistics - finding out how many times a stamp gets composed.
deprecation warnings.
validation (incompatibility warnings/errors, etc).

This is what composables specs. lack. I truly believe that thru has to be part of the specs.
Proposed syntax.
Another more readable name for the thru method can be postCompose.
Similar to the init methods, the postCompose should be able to override the resulting stamp.
const FooStamp = compose({ postCompose({left, right, result}) { // all three are stamps
  // logic goes here
  console.log(left, right, result);
  // ...
  return thisWillOverwiteTheResultingStamp;
} });

const AStamp = compose({ methods: { a(){} } });
const ResultStamp = FooStamp.compose(AStamp); //prints all: FooStamp, AStamp, ResultStamp

@troutowicz @unstoppablecarl @JosephClay @sethlivingston @yasinuslu @zebulonj thoughts?

I think a post-compose method could be useful. A use that comes to mind is logging/debugging. Right now, composition is a black box. We throw some stamps in and a new stamp might pop out if used correctly. Being able to step through each step of the composition process could make the process more verbose and easier to troubleshoot. I don't particularly care for thru, postCompose is more readable IMO.
I don't see a problem with supporting Symbols in the stamp spec.

Symbols
I have not used symbols much so I am not very familiar with the way they are used by others. Not knowing of a universal intended use, my first thought is that symbol assignment should be left up to the user. The fact that classes in js inherit symbols (including statically) should probably not be relevant in our decision to include them automatically when composing. If it is decided that they are included automatically when composing, there should be a way to disable this behavior. How much performance overhead would including symbols cause?
How should symbols be handled when creating a stamp from a constructor (are we still supporting that?). That object may have symbols that you may or may not want to carry over.
The most strait forward solution that covers all use cases:

Add an (inheritable?) way to set a stamp to copy all symbols as if they were enumerable
Add .symStatic() .symProps() .symRefs() for a way to directly specify and control which symbols are set

With these two things users can use symbols in an inclusive or exclusive way.
I don't feel strongly about the symbol stuff. I have not used them extensively. Just sharing my thoughts. I do think symbols should have their own tools and not have a composition function be the only way to handle them.
Stamp Composition Functions
I could see many use cases for this and it is worth including in the spec. The name thru is not very communicative of what it does postCompose is much better but I would prefer .initStamp() or .afterComposed()
In the examples given so far the .postCompose() method is not composable itself right? It is only called on the first stamp in the list to be composed. So you would not be able to compose or inherit that behavior. Is this intended? Maybe I am not understanding.
What would the before / after functions do exactly? I can only think of use for a function that has a chance to make changes to the stamp before returning it when creating / composing stamps.
Composable Stamp Composition Functions (Proposal)
Add .initStamp() method to stamps. It behaves exactly the same as .init() as far as setting, storing, and composing goes (internally stored as an array of functions, these arrays are concatenated when composing stamps). When a stamp is composed / created, after all other composition behavior (including merging the initStamp array), each function in the initStamp array is called on the resulting composed stamp. Each function is expected to return the same or new stamp to be passed to the next function in the array etc. No intermediate stamp objects are created maintaining non-enumerable properties on the stamp.

I just did some experimenting. In node v5.0.0 Object.assign() does merge symbol properties.

If the purpose of stamps is to allow you to use them wherever you would otherwise use ES2015 classes, then what ES2015 classes do with objects (including how they handle Symbols) is relevant to the conversation.
However, it's not really about Symbols per se--it's about any non-enumerable property.

I am curious about just what the hell Object.assign() is up to in v5, however.

I haven't looked at the spec there, but I imagine that's what Object.assign() is supposed to do.
const obj = {
  [Symbol.iterator] = function *() {
    // ..
  }
}
Object.assign({}, obj) // I would expect the iterator to work

I believe in node v5 is working according to the standard
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Object/assign
On Tue, 29 Dec 2015 09:03 Christopher Hiller<EMAIL_ADDRESS>wrote:

I am curious about just what the hell Object.assign() is up to in v5,
however.
—
Reply to this email directly or view it on GitHub
https://github.com/stampit-org/stampit/issues/188#issuecomment-167665469
.

@boneskull

If the purpose of stamps is to allow you to use them wherever you would otherwise use ES2015 classes, then what ES2015 classes do with objects (including how they handle Symbols) is relevant to the conversation.

I do not know if that is the intent or not. I do not have a strong opinion one way or the other to be honest. The intent should be decided and written in the docs somewhere to keep everyone on the same page and be consistent.
As I said above I am not sure what the universal / general intended use case for symbol properties is (static or instance). I suppose if it is intended to be inherited when extending classes, and copied when using Object.assign(); it would seem the intent is for them to behave the same as normal properties as far as object creation is concerned. Then again if that were the intent, why are they specifically excluded from Object.getOwnPropertyNames() and Object.keys(obj) and for(var key in foo). The more I dig into symbols the less I understand how you would use them. I get that they are a unique key which is nice but why do they behave differently from string property keys?

why are they specifically excluded from Object.getOwnPropertyNames() and Object.keys(obj) and for(var key in foo).

Because TC39 do not want to break the web. In other words: because compatibility

@unstoppablecarl What I understand of symbols is that they're a way to make properties "private" in javascript's class-ish-based programming, since privacy via encapsulation is lost. It's privacy via obscurity.

yep. Quoting the link above about the Object.assign:

Both String and Symbol properties are copied.

@JosephClay That is one thing that you can (sort of; see below) use a Symbol for, but they also allow you to define the iteration behavior of an object (the for ... of syntax), among other things.  I think that may be the "main" purpose of a Symbol.
If an object has a Symbol property, it can be accessed via Object.getOwnPropertySymbols(), so it doesn't really make anything private.
AFAIK, the only way to achieve privacy is still via scope, whether that be a closure or use of a WeakMap.

This is being further discussed (and then implemented in stampit v3) here: https://github.com/stampit-org/stamp-specification/issues/63
Feel free to discuss if any.

The same goal can be achieved using this new feature: https://github.com/stampit-org/stamp-specification/pull/71
  let counter = 0;
  function newCompose(...args) {
    counter++;
    return compose({staticProperties: {compose: newCompose}}, this, ...args);
  }
  newCompose().compose().compose();
  console.log(counter); // 3


Enhancement: Add an option to have index as prefix to the filename

Currently, files will be named as <prompt>_1.jpg, <prompt>_2.jpg, etc. While it might be helpful if I'm looking up the first part of prompts, it's less so when I wish to view them chronologically (by default) for something like looking through multiple saved results of spamming a specific prompt. Sorting by date doesn't help because of the asynchronous nature of retrieving the images.
As such, with this option enabled, I could have the files named as something like 00001_<prompt>.jpg, 00255_<prompt>.jpg, etc. The padding is there to ensure that it would appear as how the items are ordered in the collection when it gets sorted by name.
As for the name of the option, --prefix_index <pad length> would be my suggestion.

I'm not really into command args though, would a config file like an .ini or .cfg work for you too?

Agreed. While I'm okay with either way, I can see the elegance of putting the setting in a config file, or TOML.

I think having the creation date as the first part in ISO format could also be useful.

+1. It has its robustness and it saves me from having to increment the index for however many each run. One caveat I can think of, however, is that it would not work for collections whose items have been manually organized, either by manually moving and combining items between collections, or by the Manage feature in a specific collection in the Edge browser.

@rc-gr I meant the actual creation date of the image itself. It requires another API call but it should be possible to fetch that information.

@rc-gr I meant the actual creation date of the image itself. It requires another API call but it should be possible to fetch that information.

Gotcha. And again, I'm not averse to going with this as it has its benefits.
But to clarify...Suppose I have a collection of 3 images, A, B and C, where chronologically in terms of creation date descending, A > B > C, and would be arranged as A, B, C by default upon saving them in the order they were generated. As mentioned in my previous comment, I could reorder them so they would appear as eg. B, A, C in the collection. But if we adopted the ISO format, we'd lose out on respecting this arrangement filename-wise.
In this case, after processing, it would work out to be something like 00001_B.jpg, 00002_A.jpg and 00003_C.jpg. Whereas date-wise, it might end up like<PHONE_NUMBER>00_A.jpg,<PHONE_NUMBER>00_B.jpg and<PHONE_NUMBER>00_C.jpg.

@rc-gr I think I'm starting to get what you mean.
You mean I should include the index in the item array of the collection response, right?
I meant that the naming could be something like this:
2023-11-08T1736Z_0001_Prompt.jpg
2023-10-07T1253Z_0002_Prompt.jpg
2023-11-08T1842Z_0003_Prompt.jpg
In the end it's dynamic anyways. I'll just include different values that users can include for the file name and the user can build their own format string.
Basically something like this:
    file_name_substitute_dict = {
        'date': datetime.datetime.isoformat(datetime.datetime.now()),
        'index': '1',
        'prompt': 'Nice image',
        'sep': '_'
    }
    t = string.Template('$date$sep$index$sep$prompt.jpg')
    s = t.safe_substitute(file_name_substitute_dict)

The date is just a placeholder and I would add padding etc.
But the template would be supplied by the config file.

@rc-gr I'm pretty much done with this feature. I just want to de-duplicate the code in the download_and_save_image function and test the overall feature, but it looks pretty good thus far.

Closed by 3c59afa


Request: package for limetext

Lime Text is a powerful and elegant text editor primarily developed in Go that aims to be a Free and open-source software successor to Sublime Text.
Lime has a few frontends (QML, command-line interface) that can be selectively used with the pluggable backend.

http://limetext.org/

[ ] Update limetext wiki page

I have made package for limetext-termbox in my branch (Bevaz/nixpkgs@d848f3dbf9b2ef1fb8adae9069b0331870d6d155 & Bevaz/nixpkgs@b474a24f00714b9075b1a8c7e3942191ee0eb6ea)
It is possible to compile. But it crashes on start.
Here is the dump:
[09:31:42 UTC 2015/08/04] [EROR] (log.(*logger).Error:5) Unable to load colorscheme definition: open ../packages/themes/TextMate-Themes/Monokai.tmTheme: no such file or directory
[09:31:42 UTC 2015/08/04] [EROR] (log.(*logger).Error:5) Unable to use 256 color mode: termbox: TERM does not contain "256"
[09:31:42 UTC 2015/08/04] [DEBG] (log.(*logger).Debug:4) {{0 0} map[log:{3 12077025} clip:{1 19477} show:{1 47035}]}
[09:31:42 UTC 2015/08/04] [CRIT] (log.(*logger).Critical:3) runtime error: invalid memory address or nil pointer dereference
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x418d72]

goroutine 1 [running, locked to thread]:
main.func·014()
	/tmp/nix-build-go1.4-limetext-termbox-30af493.drv-0/go/src/limetext-termbox/main/main.go:812 +0xd9
main.setSchemeSettings()
	/tmp/nix-build-go1.4-limetext-termbox-30af493.drv-0/go/src/limetext-termbox/main/main.go:764 +0x272
main.createFrontend(0x0)
	/tmp/nix-build-go1.4-limetext-termbox-30af493.drv-0/go/src/limetext-termbox/main/main.go:172 +0x466
main.main()
	/tmp/nix-build-go1.4-limetext-termbox-30af493.drv-0/go/src/limetext-termbox/main/main.go:816 +0xec

goroutine 5 [chan receive]:
github.com/ccpaging/log4go.(*ConsoleLogWriter).run(0xc2080422a0, 0x7f6b063debd8, 0xc208036008)
	/tmp/nix-build-go1.4-log4go-cb4cc51.drv-0/go/src/github.com/ccpaging/log4go/termlog.go:77 +0x7f
created by github.com/ccpaging/log4go.NewConsoleLogWriter
	/tmp/nix-build-go1.4-log4go-cb4cc51.drv-0/go/src/github.com/ccpaging/log4go/termlog.go:56 +0x13f

goroutine 6 [syscall]:
os/signal.loop()
	/nix/store/spzkn897hriimk3imbnlksw3s3m3fvhy-go-1.4.2/share/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/nix/store/spzkn897hriimk3imbnlksw3s3m3fvhy-go-1.4.2/share/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 7 [chan receive]:
github.com/ccpaging/log4go.(*ConsoleLogWriter).run(0xc208042690, 0x7f6b063debd8, 0xc208036008)
	/tmp/nix-build-go1.4-log4go-cb4cc51.drv-0/go/src/github.com/ccpaging/log4go/termlog.go:77 +0x7f
created by github.com/ccpaging/log4go.NewConsoleLogWriter
	/tmp/nix-build-go1.4-log4go-cb4cc51.drv-0/go/src/github.com/ccpaging/log4go/termlog.go:56 +0x13f

goroutine 8 [chan receive]:
github.com/limetext/text.(*SerializedBuffer).worker(0xc208094008)
	/tmp/nix-build-go1.4-text-d6b8f8c.drv-0/go/src/github.com/limetext/text/serialized_buffer.go:40 +0x61
created by github.com/limetext/text.(*SerializedBuffer).init
	/tmp/nix-build-go1.4-text-d6b8f8c.drv-0/go/src/github.com/limetext/text/serialized_buffer.go:26 +0xd9

goroutine 9 [syscall]:
syscall.Syscall6(0xe8, 0x4, 0xc2080e7cb4, 0x7, 0xffffffffffffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
	/nix/store/spzkn897hriimk3imbnlksw3s3m3fvhy-go-1.4.2/share/go/src/syscall/asm_linux_amd64.s:46 +0x5
syscall.EpollWait(0x4, 0xc2080e7cb4, 0x7, 0x7, 0xffffffffffffffff, 0x0, 0x0, 0x0)
	/nix/store/spzkn897hriimk3imbnlksw3s3m3fvhy-go-1.4.2/share/go/src/syscall/zsyscall_linux_amd64.go:376 +0x93
gopkg.in/fsnotify%2ev1.(*fdPoller).wait(0xc20801e260, 0x0, 0x0, 0x0)
	/tmp/nix-build-go1.4-fsnotify.v1-v1.2.0.drv-0/go/src/gopkg.in/fsnotify.v1/inotify_poller.go:85 +0xc7
gopkg.in/fsnotify%2ev1.(*Watcher).readEvents(0xc208038230)
	/tmp/nix-build-go1.4-fsnotify.v1-v1.2.0.drv-0/go/src/gopkg.in/fsnotify.v1/inotify.go:179 +0x16f
created by gopkg.in/fsnotify%2ev1.NewWatcher
	/tmp/nix-build-go1.4-fsnotify.v1-v1.2.0.drv-0/go/src/gopkg.in/fsnotify.v1/inotify.go:58 +0x32b

goroutine 10 [chan receive]:
github.com/limetext/lime-backend/lib/log.func·001()
	/tmp/nix-build-go1.4-lime-backend-4d984c5.drv-0/go/src/github.com/limetext/lime-backend/lib/log/logwriter.go:37 +0x69
created by github.com/limetext/lime-backend/lib/log.NewLogWriter
	/tmp/nix-build-go1.4-lime-backend-4d984c5.drv-0/go/src/github.com/limetext/lime-backend/lib/log/logwriter.go:40 +0x16b

goroutine 11 [chan receive]:
github.com/limetext/lime-backend/lib.(*Editor).inputthread(0xc208001560)
	/tmp/nix-build-go1.4-lime-backend-4d984c5.drv-0/go/src/github.com/limetext/lime-backend/lib/editor.go:366 +0x10f
created by github.com/limetext/lime-backend/lib.GetEditor
	/tmp/nix-build-go1.4-lime-backend-4d984c5.drv-0/go/src/github.com/limetext/lime-backend/lib/editor.go:142 +0x75a

goroutine 12 [select]:
github.com/limetext/lime-backend/lib/watch.(*Watcher).Observe(0xc208038280)
	/tmp/nix-build-go1.4-lime-backend-4d984c5.drv-0/go/src/github.com/limetext/lime-backend/lib/watch/watch.go:201 +0x333
created by github.com/limetext/lime-backend/lib.GetEditor
	/tmp/nix-build-go1.4-lime-backend-4d984c5.drv-0/go/src/github.com/limetext/lime-backend/lib/editor.go:143 +0x77a

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/nix/store/spzkn897hriimk3imbnlksw3s3m3fvhy-go-1.4.2/share/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 13 [select]:
github.com/ccpaging/log4go.(*FileLogWriter).run(0xc208086000)
	/tmp/nix-build-go1.4-log4go-cb4cc51.drv-0/go/src/github.com/ccpaging/log4go/filelog.go:106 +0xa42
created by github.com/ccpaging/log4go.NewFileLogWriter
	/tmp/nix-build-go1.4-log4go-cb4cc51.drv-0/go/src/github.com/ccpaging/log4go/filelog.go:92 +0x3de

goroutine 15 [chan receive]:
github.com/limetext/lime-backend/lib.(*View).parsethread(0xc208104000)
	/tmp/nix-build-go1.4-lime-backend-4d984c5.drv-0/go/src/github.com/limetext/lime-backend/lib/view.go:216 +0x158
created by github.com/limetext/lime-backend/lib.newView
	/tmp/nix-build-go1.4-lime-backend-4d984c5.drv-0/go/src/github.com/limetext/lime-backend/lib/view.go:75 +0x23d

goroutine 16 [chan receive]:
github.com/limetext/text.(*SerializedBuffer).worker(0xc208112088)
	/tmp/nix-build-go1.4-text-d6b8f8c.drv-0/go/src/github.com/limetext/text/serialized_buffer.go:40 +0x61
created by github.com/limetext/text.(*SerializedBuffer).init
	/tmp/nix-build-go1.4-text-d6b8f8c.drv-0/go/src/github.com/limetext/text/serialized_buffer.go:26 +0xd9

wip here: https://github.com/MasseGuillaume/nixpkgs/commit/3123399c651991f40184ed6760e5e4563af9fd9b

(triage) @MasseGuillaume, @Bevaz, have you succeeded in building/running limetext?

@rasendubi limetext dev said the editor was not quite ready the last time I asked.

we should probably close this until then.

Yep this projects look dead. So sad.
https://github.com/limetext/lime-qml/graphs/contributors
https://github.com/limetext/lime-termbox/graphs/contributors
https://github.com/limetext/lime-html/graphs/contributors


dotnet restore fails

dotnet restore fails with weird error message:
C:\doc\dev\electron.net-musicplayer-sample\MyElectronMusicPlayer\MyElectronMusicPlayer.csproj : error NU1202: Package ElectronNET.CLI 0.0.11 is not compatible
 with netcoreapp2.1 (.NETCoreApp,Version=v2.1). Package ElectronNET.CLI 0.0.11 supports: netcoreapp2.1 (.NETCoreApp,Version=v2.1) / any [C:\doc\dev\electron.n
et-musicplayer-sample\MyElectronMusicPlayer.sln]
C:\doc\dev\electron.net-musicplayer-sample\MyElectronMusicPlayer\MyElectronMusicPlayer.csproj : error NU1212: Invalid project-package combination for Electron
NET.CLI 0.0.11. DotnetToolReference project style can only contain references of the DotnetTool type  [C:\doc\dev\electron.net-musicplayer-sample\MyElectronMu
sicPlayer.sln]

dotnet sdk installed on machine: 2.1.500, 2.1.504, 2.2.104.
dotnet version 2.1.500 is selected for this specific build by using global.json file.
Any ideas how to get this working?
Thanks

Fixed by removing line with DotNetCliToolReference from csproj file.


[runtime] Add a dumper/verifier for existential containers (i.e. Any)…

… in c++.
The dumper method dumps:

The container's metadata pointer.
The container's metadata representation.
A pointer to the container's value.
Whether or not said value is stored inline in the container.
The names of the conformance, protocol of all witness tables that trail the container.

This provides a general overview that can be used even when working with SIL
code in the debugger by grabbing a pointer to swift Anys and then calling the
c++ any method upon them.
The verifier is intended to be used in conjunction with ASAN for maximum
effect to catch use-after-frees of existential boxes. Specifically:

The memory where the box itself is stored is tested to see if the memory is
live by performing a load from the box to get the Type field.
If the value is stored out of line, the first byte of the value is read
from. Volatile is used to make sure that the optimizer does not remove the
read.

Additionally it goes through all of the protocol conformance descriptor and
runs their respective verification routine as well.

@swift-ci smoke test

@aschwaighofer Can you take a look at this? Just crossing my Ts.

@swift-ci smoke test

@swift-ci smoke test

@swift-ci smoke test

@swift-ci smoke test

@aschwaighofer How does this look?

@swift-ci smoke test


Native Model Parallellism in equinox

Oftentimes, one wants to do a more general n-way data parallelism, m-way model parallelism as helpfully explained in the official JAX docs.
Here, the common convention is to alternatively shard the layers, as laid out in the Megatron paper. The linked JAX example also uses this:
W2 = jax.device_put(W2, NamedSharding(mesh, P(None, 'model')))
...
W3 = jax.device_put(W3, NamedSharding(mesh, P('model', None)))

Which reduces the communication required.
We don't really have an API for that in equinox. While an external library might be a better fit, ultimately I this is such a common usecase that is should be a core feature IMO.
Scalax has a rule based system where it tries to guess the "correct" axis, like the FSDPShardingRule and for a while I was thinking of simply porting it to work with eqx, but it was proving rather hairy - its doing everything explicitly, because the primary usecase is for operating with flax, and thus requires quite a bit of fiddling, plumbing and replacing things with filtered versions that I ultimately gave up.
I think eqx should adopt a simpler, flexible API system wherein one can configure sharding through a configurable system and be able to apply it to arbitary PyTrees, without needing to explicitly provide sharding for every leaf.
What do you think?

So this is something which Haliax, a downstream library of Equinox, explicitly adds.
I'd be open to open suggestions on how to accomplish something similar within Equinox, although I'm also concious of not wanting to step on Haliax's toes.

The problem with depending on downstream libs is that often the maintainers move on with life and are unable to contribute as much for reasons - so having crucial features mostly tied to the core framework would make equinox better equipped for researchers, who can then extend its capabilities through wrappers without having to implement everything from scratch.
(Not to mention that porting an equinox codebase to haliax requires a non-trivial amount of effort)
I'm not sure what a good API here is, tbh. Personally, I'm strongly considering spinning off my own lib for parallelization w/ equinox, but I'd rather prefer to see some sort of interface for autoparallelization within equinox itself.
I wonder - for a quick and dirty hack that is compatible with current codebases, could we just somehow integrate a jmp-like utility API, so atleast its to port computations to being sharded. and allow us to do alternating-sharding-between-weights by maintaining some state and explicitly specifying a "sharding policy"?

(Disclaimer: Haliax is mine.)
I really think you have to do something like what Haliax (or really even Flax) does in the general case, which is associate semantic names with at least a subset of axes and then map those to the physical mesh axes. In Haliax I decided to go "all in" on names (which is not proving to be super popular) but there is a middle ground like flax where you could use a jmp-style object (or global state like in flax) to hold the semantic-to-physical mapping and then (like flax) make a shard_with_names(tensor, ("fsdp", "...")) fn.
(Totally get where you're coming from w.r.t. people moving on. I will point out that I still merge PRs into my last "big" library Breeze and it's been going for almost 15 years, so I have a decent track record of not totally abandoning things!)

Thinking more about it, I had been in the process of refactoring how Haliax worked to be a bit more jmp like anyway (making a "mesh env" that had a mesh and an axis mapping.) With a bit more work I could break it out into a library, if you wanted to collaborate on that.

If the question is directed at me -- happy to help out with anything needed from the Equinox side / if you think it's best to upstream anything into Equinox itself. If you're planning a totally separate library then I think I have enough commitments right now, and will politely decline 😅

No, I meant @neel04 but would be happy to discuss upstreaming into Equinox.  😄  IMHO the right thing to do is to make it as a separate lib, and iterate until we get it right. Then, if you decide you like it, you can bring it into equinox.

Haha!
In that case, SGTM!

(Disclaimer: Haliax is mine.)
I really think you have to do something like what Haliax (or really even Flax) does in the general case, which is associate semantic names with at least a subset of axes and then map those to the physical mesh axes. In Haliax I decided to go "all in" on names (which is not proving to be super popular) but there is a middle ground like flax where you could use a jmp-style object (or global state like in flax) to hold the semantic-to-physical mapping and then (like flax) make a shard_with_names(tensor, ("fsdp", "...")) fn.
(Totally get where you're coming from w.r.t. people moving on. I will point out that I still merge PRs into my last "big" library Breeze and it's been going for almost 15 years, so I have a decent track record of not totally abandoning things!)

Haha I know - this wasn't in reference to you 🙂 Rather, I just feel that approach of leveraging 3rd party support almost always leads to fragmentation.
I have strong opinions about torch's ecosystem precisely because of this - 3rd party libs often don't operate on shared abstractions which makes things tricky to then operate with other libs, leading to unforeseen edge cases and an awful dev experience - where it feels like you're perpetually in 'integration hell'.
I feel like equinox might be slipping into a similar mistake here. Parallelization is the lifeblood of JAX, so it should have first-class support in any JAX based framework. Ideally, as strong as flax's.
Looking at flax's approach, I feel like it's overcomplicating it and moving away from the spirit of equinox.
hmmm.. I wonder - is there any iterative way we can inject sharding_constraint and filter_shard at each step of computation in a Module? Even if it requires some hacks to analyze the __call__, the reward would be a drop-in replacement in any equinox codebase and operate with arbitrary PyTrees, potentially.
I'll have to think more about, but I'm imagining it like a wrapper hook on the forward effectively, wherein we insert sharding dataset per some policy, but without explicit changes like jmp

I've been playing around with a couple approaches in this Colab. I've done a minimal implementation of approach #1 there.

For consecutive layers, we need to alternate the sharding to minimize communication, as taken from the Megatron paper.
So, we need a way to maintain state and track the index of the leaf which we are at. A couple naive solutions come up:

Using a tree_map to inject an index each "shardable" leaf. i.e, tree_map with f: Array -> (Array, int) and then during our sharding stage, we can extract that index and based on that information, adjust our sharding.

Or somehow use a jax.lax.scan. The problem here is that the leading axes aren't guranteed to be same which causes problems.

Maybe one can develop some custom map solution that incorporates scan or some other utility somehow and embed state enough to hold an index + other information.

Specify an explicit spec beforehand, wherein we provide our sharding utility a PyTree of key: sharding which it can directly address. This is similar to how scalax but is a very brittle and time-consuming approach.

Here's how a sample spec would look:
model_sharding_rule = TreePathShardingRule(
    ('embedding', P('fsdp', 'tp')),
    ('lm_head/kernel', P('tp', 'fsdp')),
    # Megatron style feedfoward sharding
    ('mlp/(up|gate)_proj/kernel', P('fsdp', 'tp')),
    ('mlp/down_proj/kernel', P('tp', 'fsdp')),
    # Attention should be sharded by heads
    ('self_attn/(k|q|v)_proj/kernel', P('fsdp', 'tp')),
    ('self_attn/o_proj/kernel', P('tp', 'fsdp')),
    ('norm', P()),
)

None of these approaches feel right to me. Especially for the approach #1 which is intuitive, but might make a copy of the PyTree.
I can use some combination for my personal codebase, but I think we're just hitting the limitations of the toolset equinox has to offer in terms of manipulating PyTrees.
What do you guys think?


Using lint apply unsafe fixes on only rule error

Environment information
CLI asked me to report error while performing this command.
I want to apply fix to only a single rule. For some reason it applies a lot more.
pnpm biome lint --write --only=lint/suspicious/noDoubleEquals --unsafe
Details

Biome encountered an unexpected error
This is a bug in Biome, not an error in your code, and we would appreciate it if you could report it to https://github.com/biomejs/biome/issues/ along with the following information to help us fixing the issue:

Source Location: /Users/runner/work/biome/biome/crates/biome_rowan/src/ast/mod.rs:238:13
Thread Name: biome::worker_14
Message: Tried to cast node with kind JS_EXPRESSION_STATEMENT as `"biome_js_syntax::generated::nodes::AnyJsExpression"` but was unable to cast

Biome encountered an unexpected error

This is a bug in Biome, not an error in your code, and we would appreciate it if you could report it to https://github.com/biomejs/biome/issues/ along with the following information to help us fixing the issue:

Source Location: /Users/runner/work/biome/biome/crates/biome_rowan/src/ast/mod.rs:238:13
Thread Name: biome::worker_3
Message: Tried to cast node with kind JS_EXPRESSION_STATEMENT as `"biome_js_syntax::generated::nodes::AnyJsExpression"` but was unable to cast

/Users/emil/Projects/repos/feide-kp/frontend/src/Components/UserRoles/components/UserRoleTableItem/UserRoleTableItem.tsx internalError/panic  INTERNAL  ━━━━━━━━━━

  ✖ processing panicked: Tried to cast node with kind JS_EXPRESSION_STATEMENT as `"biome_js_syntax::generated::nodes::AnyJsExpression"` but was unable to cast

  ⚠ This diagnostic was derived from an internal Biome error. Potential bug, please report it if necessary.

/Users/emil/Projects/repos/feide-kp/frontend/src/Pages/Organization/Panels/OrganizationRoles.tsx internalError/panic  INTERNAL  ━━━━━━━━━━

  ✖ processing panicked: Tried to cast node with kind JS_EXPRESSION_STATEMENT as `"biome_js_syntax::generated::nodes::AnyJsExpression"` but was unable to cast

  ⚠ This diagnostic was derived from an internal Biome error.  For some reason it applies a lot more.
Code of Conduct

[X] I agree to follow Biome's Code of Conduct

Details

test

@ematipico  Updated with repo

Thank you. Here's a plyaground link of the snippet that creates an invalid CST: https://biomejs.dev/playground/?code=ZQB4AHAAbwByAHQAIABmAHUAbgBjAHQAaQBvAG4AIABQAGEAbgBpAGMAKAApACAAewAKACAAIAByAGUAdAB1AHIAbgAgADwAZABpAHYAPgB7AGYAbwBvACAAJgAmACAAPAA%2BAHsAYAAoACQAewBiAGEAcgB9ACkAYAB9ADwALwA%2BAH0APAAvAGQAaQB2AD4AOwAKAH0A
Help wanted

Would just like to mention that this is perhaps two issues on one "card". There were some unexpected linting as well, as described in initial report. Maybe separate into two issues?

i'd like to work on this.


Origin header is not present in Curl request?

Hi,
In my web api backend application, we are performing authorization token origin validation. This is done by reading Origin header parameter in API request and matching that with the Origin url embedded in Authorization token. This works fine with our Angular client application.
To allow swashbuckle/swagger to pass Authorization token, I added a Authorization parameter using IOperationFilter as explained in https://github.com/domaindrivendev/Swashbuckle/issues/501, it works great.
Since we perform Request Origin header validation, requests from swagger doesn't work now. I initially thought I can add another Header parameter using IOPeratinFilter called Origin and pass origin url along with API request and implemented the same but when request reaches Curl request does not contain a Origin header.
How do I configure swashbuckle/swagger configure in such a way that it pass Origin header always with a curl request?

It's been a while since I've worked with CORS, but isn't it the browsers job to add "Origin" headers. In fact, I don't believe it's possible to add them manually, which is what you're asking the swagger-ui code to do.
I'm assuming the CURL requests you're referring to are not cross-origin and hence the browser isn't adding an Origin header.

Closing due to inactivity


Local state query re-updates component with no necessity

Hi
I have this code where I simply fetch the loggedIn info from the local state :
const sessionQuery = gql`
  query {
    session @client {
      loggedIn
    }
  }
`

export default graphql(sessionQuery, { name: 'localState' })(
  ({ localState: { session }}) => {
    console.log('rendering App with session:', session)
    return !session.loggedIn ?
      <Logger/>
    : [
      <Grid key={0}>
        <Actions/>
        <Authors/>
      </Grid>,
      <Grid key={1}>
        { Array.from(Array(16).keys()).map(
            shelf =>
              <Shelf key={ shelf + 1 } shelf={ shelf + 1 }/>
          )
        }
      </Grid>
    ]
  }
)

Even if the query data is fully available on first render, somehow an update and second render is triggered. This logs with a few ms delay :
rendering App with session: {loggedIn: true, __typename: "Session", Symbol(id): "$ROOT_QUERY.session"}
rendering App with session: {loggedIn: true, __typename: "Session", Symbol(id): "$ROOT_QUERY.session"}

Why is that ?
Also, I noticed sometimes the data from the local state is available immediatly, sometimes it's not (right after mutations ?). I suggest you clarify that behaviour in the doc.
Thanks a lot
Olivier

@ostrebler, maybe the same question? https://github.com/apollographql/apollo-link-state/issues/217

This is a react-apollo issue, not apollo-link-state. Discussion has been opened up on the react-apollo repo:
https://github.com/apollographql/react-apollo/issues/1777
Thanks! 😀


Spike: Frequency Provider UX Notes

Problem:

The Frequency Provider UX could be improved.

Solution:

Writeup of initial thoughts on UX improvements.

Writeup: https://docs.google.com/document/d/19QyZt1j2gzeEMF4OZeB_AgskiGoyml1cxB2XI2uLOTQ/edit?usp=sharing
Next Steps: Update wireframes.


Config resetting to default

Running 2023.12.3 as an addon, any changes I make disappear when I restart the addon, dashboard comes back empty with this error in the log:
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
Starting Fusion...
Listening on <IP_ADDRESS>:5050
Error reading or parsing ./data/configuration.yaml: Error: ENOENT: no such file or directory, open './data/configuration.yaml'
    at async open (node:internal/fs/promises:633:25)
    at async readFile (node:internal/fs/promises:1242:14)
    at async loadFile (file:///rootfs/build/server/chunks/3-85oe4GFn.js:8:18)
    at async Promise.all (index 0)
    at async load (file:///rootfs/build/server/chunks/3-85oe4GFn.js:16:38)
    at async load_server_data (file:///rootfs/build/server/index.js:1891:18)
    at async file:///rootfs/build/server/index.js:3301:18 {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: './data/configuration.yaml'
}

Having the same Problem
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
Starting Fusion...
Listening on <IP_ADDRESS>:5050
Error reading or parsing ./data/dashboard.yaml: Error: ENOENT: no such file or directory, open './data/dashboard.yaml'
at async open (node:internal/fs/promises:633:25)
at async readFile (node:internal/fs/promises:1242:14)
at async loadFile (file:///rootfs/build/server/chunks/3-sykjDTvk.js:8:18)
at async Promise.all (index 1)
at async load (file:///rootfs/build/server/chunks/3-sykjDTvk.js:16:38)
at async load_server_data (file:///rootfs/build/server/index.js:1891:18)
at async file:///rootfs/build/server/index.js:3301:18 {
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: './data/dashboard.yaml'
}
Error reading or parsing ./data/configuration.yaml: Error: ENOENT: no such file or directory, open './data/configuration.yaml'
at async open (node:internal/fs/promises:633:25)
at async readFile (node:internal/fs/promises:1242:14)
at async loadFile (file:///rootfs/build/server/chunks/3-sykjDTvk.js:8:18)
at async Promise.all (index 0)
at async load (file:///rootfs/build/server/chunks/3-sykjDTvk.js:16:38)
at async load_server_data (file:///rootfs/build/server/index.js:1891:18)
at async file:///rootfs/build/server/index.js:3301:18 {
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: './data/configuration.yaml'
}

Same issue:
Error reading or parsing ./data/dashboard.yaml: Error: ENOENT: no such file or directory, open './data/dashboard.yaml'
at async open (node:internal/fs/promises:633:25)
at async readFile (node:internal/fs/promises:1242:14)
at async loadFile (file:///rootfs/build/server/chunks/3-aw3qjczk.js:8:18)
at async Promise.all (index 1)
at async load (file:///rootfs/build/server/chunks/3-aw3qjczk.js:16:38)
at async load_server_data (file:///rootfs/build/server/index.js:1891:18)
at async file:///rootfs/build/server/index.js:3301:18 {
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: './data/dashboard.yaml'

Having the same Problem
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
Starting Fusion...
Listening on <IP_ADDRESS>:5050
Error reading or parsing ./data/configuration.yaml: Error: ENOENT: no such file or directory, open './data/configuration.yaml'
at async open (node:internal/fs/promises:633:25)
at async readFile (node:internal/fs/promises:1242:14)
at async loadFile (file:///rootfs/build/server/chunks/3-j8qz_UKp.js:8:18)
at async Promise.all (index 0)
at async load (file:///rootfs/build/server/chunks/3-j8qz_UKp.js:16:38)
at async load_server_data (file:///rootfs/build/server/index.js:1891:18)
at async file:///rootfs/build/server/index.js:3301:18 {
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: './data/configuration.yaml'
}
Error reading or parsing ./data/dashboard.yaml: Error: ENOENT: no such file or directory, open './data/dashboard.yaml'
at async open (node:internal/fs/promises:633:25)
at async readFile (node:internal/fs/promises:1242:14)
at async loadFile (file:///rootfs/build/server/chunks/3-j8qz_UKp.js:8:18)
at async Promise.all (index 1)
at async load (file:///rootfs/build/server/chunks/3-j8qz_UKp.js:16:38)
at async load_server_data (file:///rootfs/build/server/index.js:1891:18)
at async file:///rootfs/build/server/index.js:3301:18 {
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: './data/dashboard.yaml'
}

https://github.com/matt8707/ha-fusion/releases/tag/2023.12.4


Update for Node.js 16?

This action is the only one we've been using that still has not been updated to Node.js 16. Each time we run our workflow we see this warning:
Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: pocket-apps/action-setup-firebase@v2. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.

Came here for the very same reason.
@vimtor

Sorry but I am no longer maintaining this extension. In fact, I don't even own this repository fork.


CloudFormation stack to deploy lambda functions, S3 bucket, and Notebook instance with appropriate IAM roles is not accessible.

link: S3 URL: https://orthanc-on-aws.s3.amazonaws.com/sagemaker_resources.yaml
error: S3 error: Access Denied For more information check http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html

Hi ,
I am also getting the same error as mentioned above : "S3 error: Access denied " when accessing the S3 URL link https://orthanc-on-aws.s3.amazonaws.com/sagemaker_resources.yaml . Please help us on this

Hi,
You can use a file below in this repo instead of link you mentioned.
sagemaker-groundtruth/sagemaker_resources.yaml


Feature Request - Add QRCODE

Hi Benny, can you add a QRCODE like on https://github.com/BennyE/stellar-dynpsk with option for WPA/WPA2-PSK or NOPASS for Open SSID's.
I found this code from https://github.com/lakhanmankani/wifi_qrcode_generator showing how to generate a QRCODE for an Open SSID.
if authentication_type in ('WPA', 'WEP'):
if password is None:
raise TypeError('For WPA and WEP, password should not be None.')
return 'WIFI:T:{type};S:{ssid};P:{password};H:{hidden};;'.format(
type=authentication_type, ssid=ssid, password=password, hidden=hidden
)
elif authentication_type == 'nopass':
if password is not None:
raise TypeError('For nopass, password should be None.')
return 'WIFI:T:nopass;S:{ssid};H:{hidden};;'.format(
ssid=ssid, hidden=hidden
)
raise ValueError('Unknown authentication_type: {!r}'.format(authentication_type))
Thanks for your Github Projects that we use for PoC and ALE Presales Presentations and for your awesome wiki at http://dokuwiki.alu4u.com

Hi,
Do you mean to have a captive portal on a either an open or WPA2-protected SSID? This script creates a guest user in the Captive Portal (UPAM). I need to find out if the necessary library (qr / png) can run on the OmniSwitch, otherwise it would have to be a separate branch.
Thanks,
Regards,
Benny

Hi Benny,
Usually the iOS devices don't trust on Open SSID's used for Guest users, when the iPhone sleeps and wakes up on my experience they don't reconnect every time. As workaround I usually manage Guest SSID's with WPA2-PSK (AES) and Captive Portal because usually old phones don't support WPA3 (old hardware or operational system old releases).
So the idea is to use your implementation for create a guest account and attach the QRCODE to simplify the on boarding of guest users :)
Regards,
Eduardo

Hi Benny,
Take your time and enjoy your vacation time. In mean time I'll will use both scripts to show the concepts of creating a guest user using OmniSwitch and the possibility to generated a QR code with the https://github.com/BennyE/stellar-dynpsk, thanks!
Regards,
Eduardo Coelho

Hi @ebcoelho,
Please review/try https://github.com/BennyE/stellar-add-guest/tree/qrcode and let me know.
Benny


testing for hca-to-scea

Prior to adding features to hca-to-scea it needs to have proper unit tests.
A few months back @ami-day and @amnonkhen started creating a characteristic test for this tool.
This ticket is to finish this work.

@ami-day please review the PR.
I have fixed the Characteristic Tests.
Currently, one of them fails.
Could you please run and analyse the diff?


support postgres vector store

Description
I added support for postgres as a vector store using pgvector, sqlalchemy and psycopg2
Type of Change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Added new notebook (that tests end-to-end)

Suggested Checklist:

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

@helmanofer  thanks for the contribution! Taking a look soon!

@Disiok Thanks for your comments. I committed the changes and added some unit tests

@Disiok Thanks for your comments. I committed the changes and added some unit tests

Amazing! Will take another look!


Change kuttl job to use base-simple-crc as parent

As a pull request owner and reviewers, I checked that:

[x] Appropriate testing is done and actually running
[x] Appropriate documentation exists and/or is up-to-date:

[x] README in the role
[x] Content of the docs/source is reflecting the changes

recheck
issue with the inventory file?

recheck

/approve


How to specify different remote besides origin?

When working with forks of primary repos, sometimes it is important to create a changelog for the fork even though you are working inside a close which has origin configured for the primary repo. Additionally some repos are set up to use a name besides 'origin' for the sole remote (for example, this may be required for a private repository due to policy).
How can a user specify which remote should be used instead of origin?

Hi @espears4sq
Can you clarify your question with an example? Thanks!


📢 KerasNLP is becoming KerasHub 📢

tl;dr

We are consolidating KerasNLP and KerasCV into a new KerasHub package.
We will be renaming this keras-nlp GitHub repository to keras-hub.
All existing usages will continue to work!

You can keep running pip install keras-nlp & import keras_nlp—nothing will break. Same with keras_cv.

What's happening
Popular pretrained models are frequently becoming multi-modal. In the text domain, chat LLMs are adding support for image and audio inputs and outputs. In the vision domain, using text encoders is common for a wide range of tasks from image segmentation to image generation.
We do not believe any particular division of pretrained models will help the Keras ecosystem going forward. Distinctions between model architectures (transformers vs convnets vs diffision models) become fuzzy, as do divisions between modalities.
By consolidating to a single library that focuses on easy-to-use, pretrained architectures and weights, we can better deliver a set of features that apply to all pretrained models—easy model publishing and sharing, PEFT and quantization support, scaled up muli-host training.
To that end, we are consolidating KerasNLP and KerasCV into a KerasHub package.
The plan
We will start start by simply renaming the KerasNLP library to KerasHub. KerasNLP already supports multiple modalities with models like PaliGemma, as well as many features (easy model publishing), we would like to bring to CV models.

This repo will be renamed to keras-hub. Library symbols will be renamed from keras_nlp to keras_hub.
We will keep a keras_nlp package with all the old imports. This change will be fully backward compatible.
To move from keras_nlp to keras_hub, you should be able to simply find and replace all instances of keras_nlp with keras_hub in code.
keras_cv models will move to keras_hub on an ongoing bases. You can expect some usage changes to CV models, as we make sure we deliver a consistent UX across modalities.

When is this happening
The repository is now a preview for the upcoming KerasHub release, but we have not yet made an official release of the keras-hub package. If you would like to try things out as we build, you can do so by trying our nightly package: pip install keras-hub-nightly.
We are tentatively aiming for a mid October release of KerasHub.
Feedback and help
We would appreciate any feedback from the community on this! Please feel free to use this issue to send us thoughts.
We have a number of issues related to the port with the contributions welcome tag.

Update: we have now renamed the repository and code to keras-hub, but we have not yet released the KerasHub package.
If you would like to try things out as we build, use the nightly package:  pip install keras-hub-nightly. If you want the backwards compatible keras_nlp imports, you can try that on nightly too: pip install keras-nlp-nightly.
Lastly, if you were building the package from source, and want to keep things working with the old imports, you will need to install as follows:
!pip install "git+https://github.com/keras-team/keras-hub" "git+https://github.com/keras-team/keras-hub#subdirectory=keras_nlp"

That will install both the new keras_hub source, as well as a keras_nlp shim package.
As always, feedback welcome!


RAM requirement for building from source

Fixes #421

This change is

Got LGTM on this related PR, so merging this as well.


Curl cant resolve host

Hello,
I am trying to use your config file to deploy vannila minecraft server, everything seems ok, but server cant download minecraft_server..jar so it cant boot.
Checking version information.
curl: (6) Could not resolve host: launchermeta.mojang.com
Checking type information.
Downloading minecraft_server..jar ...
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
server.properties already created, skipping
Skipping JSON check. No files present.
Setting initial memory to 1G and max to 1G
Error: Unable to access jarfile minecraft_server..jar
2018/07/09 15:43:33 Done

In cases like this it’s actually caused by name resolution outside of the container. Are you able to ping launchermeta.mojang.com from the host running the docker daemon?

Yes, i am able to ping.

I tried to wget from containter, but server cant resolve. If i do it outside i am able to resolve.

Problem fixed, i had bad --pod-network-cidr=<IP_ADDRESS>/16 while kubeadm init


Bloyal tensorflow iris byom

Issue #, if available:
Description of changes:
Testing done:
Merge Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

[ ] I have read the CONTRIBUTING doc and adhered to the example notebook best practices
[ ] I have updated any necessary documentation, including READMEs
[ ] I have tested my notebook(s) and ensured it runs end-to-end
[ ] I have linted my notebook(s) and code using tox -e black-format,black-nb-format

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Handled in another PR


Missing Code after self update

Issue Type: Bug
VS Code updated itself with no warning and when it reopened the tabs I had open with unsaved changes went completely blank (no code within them).
VS Code version: Code 1.32.0 (507312a3e3b34b084b467dfd983263bc7c9d87e0, 2019-03-06T18:57:55.589Z)
OS version: Windows_NT x64 10.0.17763

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz (8 x 1896)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Memory (System)
7.84GB (0.91GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (1)

Extension
Author (truncated)
Version

compareit
in4
0.0.2

Duplicate of https://github.com/Microsoft/vscode/issues/69972
Should be fixed in the recovery release 1.32.2 (not yet released, see linked issue for more context)
Read this comment : https://github.com/Microsoft/vscode/issues/69972#issuecomment-470706961 before closing your VSCode window, you might still have a chance to retrieve your documents...


Type generator from for CLI commands

Add type generator for CLI commands. Use it in the InitCommand.
This PR was done partially as a side project to learn compiler related concepts.

Why not generate CLI from these?

It would be harder to integrate with the existing code.

But from project's point of view, I don't think we should merge this. To me this seems like additional maintenance burden which to my intuition will unfortunately only be slowing our pace of development instead of bringing expected improvements. It's nice that this gives us type safety and autocompletion, but I can imagine we would have to constantly modify these scripts to accommodate new features we introduce in our CLI.

The problem is, we don't have tests for every single argument. This is a place in the application we could rely on types more heavily instead. Translating args to typed args manually is hard to maintain, because we have two sources of information that need to be kept in sync.

BTW I would happily merge these small tweaks you did to InitCommand code.

Tweaks depend on the generated *Input type.

It would be harder to integrate with the existing code.

So perhaps, something is wrong with existing code?

The problem is, we don't have tests for every single argument. This is a place in the application we could rely on types more heavily instead. Translating args to typed args manually is hard to maintain, because we have two sources of information that need to be kept in sync.

We do have E2E tests though, which I believe cover a lot of stuff? At least an amount that we are pretty confident into making changes there.

Ok, I'll close it. The alternative approach would require more work, hence it's not going to happen soon.

So perhaps, something is wrong with existing code?

The current approach has certain flaws, but I wouldn't say it's wrong. The alternative approach would change the design that would affect all commands. This PR introduces a change that is opt-in.

We do have E2E tests though, which I believe cover a lot of stuff? At least an amount that we are pretty confident into making changes there.

Tests don't cover everything.


H.265 support

Is your feature request related to a problem? Please describe.
The videos encoded with H.265 play black, despite the thumbnails are generated correctly.
Describe the solution you'd like
HEVC videos should be played as well.
Describe alternatives you've considered (optional)
A clear and concise description of any alternative solutions or features you've considered.
My new camera apparently encodes videos using H.265. So older videos play well in pigallery2, but the newer ones show just the black screen. While the audio plays fine. I tried using different browsers, mobile vs desktop to no avail. So it's likely to be the backend deficiency. Is it a known limitation? Or is there anything that could be done?

Hi
the issue is with the browsers, they do not support it: https://caniuse.com/hevc
The app can convert the videos back to h264. You can find that in the settings, advanced mode and under video or jobs
You can manually start the process or periodically run it.

Oh, right. Perhaps, I should just avoid using HEVC at all! Thanks!

Chrome now supports h.265. I've tried playing a h.265 encoded video in PiGallery2 version 1.9.3 but get an "Error during loading of the video". Without using the transcoding option is there another way to get h.265 encoded videos to playback?

+1 curious about this with native support being available in chrome now?


Exception is nowhere to be found

I have constant issues with ModelBuilder failing to build, but I can't find the actual error anywhere, even though it says to check the Umbraco log. The Umbraco widget is said to show the latest error, it doesn't.
I have also tried to use the Console-tool, it generates just fine, but it doesn't compile it to a DLL for me, which is why the Umbraco generator is handy.

Update: After pulling my hair for over 2 hours, I noticed a small models.err file in the App_Data/Models folder, this contained exactly the issue I needed. It did however never show up on the dashboard widget as expected.

Weird. So there is a models.err fine in App_Data/Models but no error is reported in the dashboard? Can you tell me what Models Builder config you have in web.config?

Correct, and I didn't change a thing compared to the default Umbraco installation.

Closing this issue, as it is quite (too) old. Please come back to me if you think it's still valid!


Objax Implementation

Hi,
First of all the papers is great and these models are really cool!
I've implemented the models with weights in Objax here: https://github.com/benjaminjellis/Objax-RepVGG

Thanks! Added a link in README to your repo. Nice work!


Implement configuration of webserver listening address

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

/kind release

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area engine

/area tests

/area proposals

/area CI

What this PR does / why we need it:
Currently the webserver is listening on the hard coded <IP_ADDRESS> address.
This patch keeps this default but allows the administrator to change it through the falco.yaml file.
Which issue(s) this PR fixes:

Fixes #2774
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

feat(userspace/falco): falco administrators can now configure the address on which the webserver listen using the new listen_address field in the webserver block of the `falco.yaml` file.

Welcome @sgaist! It looks like this is your first PR to falcosecurity/falco 🎉

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: sgaist
Once this PR has been reviewed and has the lgtm label, please assign lucaguerra for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/milestone 0.37.0

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.
:memo: Please follow instructions in the contributing guide to update your commits with the DCO
Full details of the Developer Certificate of Origin can be found at developercertificate.org.
The list of commits missing DCO signoff:

724ae63 test(configuration): implement basic webserver listen address test

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.
:memo: Please follow instructions in the contributing guide to update your commits with the DCO
Full details of the Developer Certificate of Origin can be found at developercertificate.org.
The list of commits missing DCO signoff:

9e0e7b1 feat(userspace/falco): add configuration support for IPV6 webserver listen address

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

LGTM, will approve after you re-push the last commit with signed off, ty!

sign-off added and re-based as well :-)

LGTM label has been added.  Git tree hash: 5afa56f2ba5006f250cd66d1233175859e261fba

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: incertum, sgaist
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, incertum, sgaist
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


[Android] Expose entry points in XWalkExtensionClient class

BUG=XWALK-2840

@halton @wang16 please help to review. I tested it on a offline case.

LGTM, thanks.

Good work, merging.


Upgrade localization priority

This article gets high localization page views, suggest upgrading the priority so it gets human translated.

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.


cardinality of 'checksum'

Pascal Heus 7:24 AM Jun 27, from Google doc
We can have more than one checksum. Would make this repeatable.

This would only make sense to me if the multiple values were for checksums calculated with different algorithms.  A checksum fingerprints a digital object. In the current design, the metadata record can be about a single digital object, or about a resource that has multiple representations (distributions), each of which is a distinct digital object. In the second case there could be multiple checksums in the metadata record, but each is scoped to a different distribution.  Different checksum == different digital object.

In ResourceSync, one hash is allowed. It is provided in the hash attribute of the not repeatable rs:md child element of the sitemap url element. The hash value is preceded by a token that indicates the algorithm that was used to compute the hash. An example would be:
<rs:md hash="md5:1e0d5cb8ef6ba40c99b14c0237be735e" />
See http://www.openarchives.org/rs/1.1/resourcesync#DocumentFormats


Blank window on standalone_app.py

I'm on fedora 37, in a venv, python 3.11.2.
I installed pywebview[qt] and remi with pip.
python standalone_app.py results in:
python standalone_app.py
remi.server      INFO     Started httpserver http://<IP_ADDRESS>:35585/
remi.request     INFO     built UI (path=/)
remi.request     DEBUG    get: /
<IP_ADDRESS> - - [03/Mar/2023 17:08:58] "GET / HTTP/1.1" 200 -

Note: I added debug=True to the start function but it didn't give much more info.
Thnaks for Remi!

It was a problem with pywebview and Qt, using Gtk solved the issue.

Thank you very much for reporting ;-)


Redundant output from test_logistic_regression_model

🐛 Bug
There are sometimes so many redundant lines in the tests output which makes it hard to review PRs:
...
tests/models/test_classic_ml.py::test_logistic_regression_model
tests/models/test_classic_ml.py::test_logistic_regression_model
tests/models/test_classic_ml.py::test_logistic_regression_model
...

This behaviour happens in https://github.com/PyTorchLightning/pytorch-lightning-bolts/pull/403/checks?check_run_id=1669189373
while it doesn't in https://github.com/PyTorchLightning/pytorch-lightning-bolts/pull/475/checks?check_run_id=1669579907
To Reproduce
I'm still unsure when this happens.
Expected behaviour
Only one line output:
tests/models/test_classic_ml.py::test_logistic_regression_model

Environment

PyTorch Version (e.g., 1.0):
OS (e.g., Linux):
How you installed PyTorch (conda, pip, source):
Build command you used (if compiling from source):
Python version:
CUDA/cuDNN version:
GPU models and configuration:
Any other relevant information:

Additional context

It seems the latest commit https://github.com/PyTorchLightning/pytorch-lightning-bolts/commit/c997f8576ca6f520df940f8e93fb874a8b130ef7 on the master branch doesn't have this issue for unknown reasons...

It seems the latest commit https://github.com/PyTorchLightning/pytorch-lightning-bolts/commit/c997f8576ca6f520df940f8e93fb874a8b130ef7 on the master branch doesn't have this issue for unknown reasons...

I'll continue investigating this further in #506.

I'll continue investigating this further in #506.