Requests to /notify return 500 if error in processing even though we save the notification

Discovered as part of our issues with concurrency but not directly related, when there is an error processing a notification we save the notification but return 500 to Adyen. This makes Adyen repeat the notification, but because that's a duplicate we return 200, and don't attempt to process.
Not processing the duplicate is fine for us because we have a rake task which goes over unprocessed notifications, but might be for someone that doesn't have that.
So either we should recommend to have a job to clean up unprocessed notifications, and don't 500 on failure to process (so Adyen is happy and doesn't send again). Or we should 500 but actually process again on the duplicate notification sent afterwards (I assume we can rely on Adyen to always send a duplicate?)

Thanks for filing this, this is a known issue that I've never got around to writing up.
Cleaning up unprocessed notifications is probably a bad idea. Notifications should only 500 if an error occurs during processing.
For example: a refund notification is received and for whatever reason the refund can't be created.
An unprocessable notification like report_ready will be marked as unprocessed and will return a 200.
If we rescue the error so that we can return a 200 then all exceptional behavior in processing is going to be suppressed. The only way you'll ever know that something went wrong is by looking at the the notification history for a specific payment and see that something is unprocessed - but again you wouldn't know why.
In the other case, if we were to return a 500, and attempt to process when we receive the duplicate then we're never going to return anything other than a 500. We're going to report the same error over and over again because the system can't process the notification.
Due to the way that Rails does rendering it's impossible to raise an error and return anything other than a 500.
For this reason I think it's fine for us to raise the error as this does indicate that something has gone wrong and someone needs to look into it. It might not be ideal, but this is probably that closest we can get.
What I would really like to do in the event is report the error to some 3rd party like Honeybadger or Rollbar, but  because it's a gem I can't assume either are present. As a compromise how about supporting an extension point where the application can define how errors should be handled in processing?

Going to close this for now unless there is any other further concerns.


Removed needless editorconfig file

I found .editorconfig file under the lib/csv directory. It's not affected in this repository. So, It should be removed.

Good catch!


Any plan on releasing Training code ? , or just a model with speech ?

Sorry I personally do not have bandwidth for now so might not be released in the near future. The training code will be somehow similar to https://github.com/haoheliu/AudioLDM-training-finetuning so I would suggest building upon this code base if interested.

thanks :)


Snssqs subscription policy

Description
after an SQS queue is created by daprd, the change introduced modifies the policy it creates to one that limits the SendMessage requests to those only originating from the daprd-created SNS topic, unlike current situation which gives a wildcard ("any") permission to send messages to the aforementioned SQS queue
Issue reference
Please reference the issue this PR will close: #1230
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[ ] Created/updated tests
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

hi committers. what's happening in terms of (secretstores.azure.keyvault) certification test that are running every 30minutes and report issues. any documentation I can follow along to solve this (if any issue)?


Fixing #433 - instance should inherit defaults

https://github.com/mzabriskie/axios/issues/443

Coverage remained the same at 93.924% when pulling 194f41a53755a20e1d2c1e48a9f6d9931b7f92b4 on yanivefraim:feature/443 into 949d08df71c87cac1754087128eb845be9a6ba2c on mzabriskie:master.

Question - do we want defaults to affect instances after instance was created?
For example - I create an instance using default 'a'. Then I change this default to be 'b'. Do I want my instance to change accordingly?

@yanivefraim Good question! I think changing defaults should affect at least the default instance. For example:
defaults.baseURL = 'http://foo.com/';
axios.get('/user');

defaults.baseURL = 'http://bar.com/';
axios.get('/user');  // should make a request to http://bar.com/user

If we do so for the default instance, we should probably do the same for custom instances for consistency. Does it make sense?
Thank you for your help!

@nickuraltsev - IMO when you create an instance with specific defaults it will be confusing / not predictable if those defaults will be changed later when someone is changing the main defaults... And what if I already changed my instance's defaults? Then I would definitely not want those to be changed!

I'm not sure about this.  For example:
const instance = axios.create();

instance.defaults.timeout =123;
instance.get('/foo'); // the new timeout will be used

defaults.validateStatus = status => true; // all statuses are ok
instance.get('/foo'); // should new validateStatus be used?

I agree. Anyway, it should be well documented, and I will be happy to implement whatever you guys decide...

Is there a plan to merge this?
This makes testing apps that use axios.create() SO much easier using axios-mock-adapter.

Regarding syncing the default and custom instance configs: I agree with @yanivefraim. Once you create a new instance, I wouldn't expect its configs to change if I change the defaults.

I think that those issues are critical, it is a show stopper for us. Mainly https://github.com/mzabriskie/axios/issues/385, which can create a mess in your requests. I'll be happy to help and fix those, but we must have a decision from maintainers.
cc @nickuraltsev @mzabriskie

@yanivefraim Thanks a lot for your help! I've been busy, but I'll review your PRs and get back you soon.

Cool. I would really love to be involved and contribute here. I think you / we should consider refactoring defaults mechanism, due to all of those issues

@yanivefraim What if we make axios.defaults a reference to the global defaults object (exported by lib/defaults.js) instead of creating a new object? Custom instances already use the global defaults when sending a request, so everything will work as designed.
// change the global defaults before creating an instance
axios.defaults.timeout = 100;

const instance = axios.create();
instance.get('/foo'); // timeout is 100

// change the global defaults after creating the instance
axios.defaults.timeout = 200;
instance.get('/foo'); // timeout is 200 because the global defaults are merged in request()

// also, we can override anything at the instance level
instance.defaults = 300;
instance.get('/foo'); // timeout is 300 because the instance defaults take precedence over the global defaults

Thoughts?

Closed by mistake

Coverage increased (+0.03%) to 93.955% when pulling dfcb18ac0dc3a7eacc230dfbb9bc8647d39319c8 on yanivefraim:feature/443 into 949d08df71c87cac1754087128eb845be9a6ba2c on mzabriskie:master.

@nickuraltsev - I fixed the PR according to your idea.
Another thing that might consider is exposing global defaults, something like:
axios.GLOBAL_DEFAULTS or  axios.global_defaults, so we'll have axios as a default instance on one hand, and as an api on the other hand (including axios.Cancel, CancelToken and so on, and including the new axios.global_defaults). This might prevent confusion between axios default instance defaults and global defaults.
WDYT?

ping... ?
@nickuraltsev @mzabriskie

What's the reason for replacing the default instance's config with defaults after the default instance is created? Can we pass the correct config to the constructor instead (for the default and custom instances)?

Closing in favor of 6d0e19343a6180fb8f5c50d843dd15feeb1f49f3


How do you load a saved pretrained model

Can you tell us how do you load saved pretrained model to finetune with supervised dataset ?

You can refer to this notebook.


Pods stuck in ContainerCreating

Describe the bug
I have two pods stuck in ContainerCreating.
One pod shows “No online replicas are available for Volume”.
  Warning  FailedAttachVolume  83s (x9 over 3m35s)  attachdetach-controller  AttachVolume.Attach failed for volume "pvc-6738a102-e39f-4413-baf9-8b5e697ebbaa" : rpc error: code = Internal desc = Operation failed: GenericOperation("error in response: status code '412 Precondition Failed', content: 'RestJsonError { details: \"No online replicas are available for Volume '6738a102-e39f-4413-baf9-8b5e697ebbaa'\", kind: FailedPrecondition }'")

The other shows “Nexus uuid already exists for nexus”.
  Warning  FailedAttachVolume  6s (x2 over 2m8s)      attachdetach-controller  (combined from similar events): AttachVolume.Attach failed for volume "pvc-134925ab-e047-4e8f-b0c9-d8c863347995" : rpc error: code = Internal desc = Operation failed: GenericOperation("error in response: status code '500 Internal Server Error', content: 'RestJsonError { details: \"create_nexus::status: Internal, message: \\\"Nexus uuid \\\\\\\"bc2d9042-4c1a-4450-baee-c91ce6c4d5d0\\\\\\\" already exists for nexus \\\\\\\"134925ab-e047-4e8f-b0c9-d8c863347995\\\\\\\"\\\", details: [], metadata: MetadataMap { headers: {\\\"content-type\\\": \\\"application/grpc\\\", \\\"date\\\": \\\"Mon, 21 Feb 2022 18:53:46 GMT\\\"} }\", kind: Internal }'")

To Reproduce
Steps to reproduce the behavior:
N/A
Expected behavior
Replicas should be available.
Screenshots
If applicable, add screenshots to help explain your problem.
** OS info (please complete the following information):**

Distro: Talos v0.14.2
Kernel version 5.15.23
MayaStor revision or container image 1.0.0

Additional context
I think this happened after the node the pods were running on was rebooted.
The storage class has 2 replicas. The cluster has 3 nodes and 3 Mayastor pools, all are ready/online.

If you've restarted all mayastor pods you may have hit an issue where we were incorrectly setting all replicas an unusable. We've just release v1.0.1 which fixes the issue but at this point if you want to recover your data you'll have to take manual steps to do so.

I have restored the volumes from a backup in the meantime.
I'm not sure if this is related to this issue, but kubectl mayastor get volumes shows one faulted volume:
 94d6f5c7-9a07-4c27-9cdb-8b499783571b  2         <none>       <none>         Faulted<PHONE_NUMBER>

This isn't referenced by any PV or PVC anymore. How can I delete this volume?

Something like this:
curl -X DELETE http://NODE_IP:30011/v0/volumes/94d6f5c7-9a07-4c27-9cdb-8b499783571b

With that I get this:
{"details":"Storage Error: Volume Config for Resource id 94d6f5c7-9a07-4c27-9cdb-8b499783571b not committed to the store","kind":"FailedPersist"}

Seems like we're not able to persist the config to etcd. Would you be able to share the core-agent logs? Otherwise you can try restarting the core-agent.

core-agent logs show:
  Feb 22 23:03:41.621 ERROR core::volume::service: error: Storage Error: Volume Config for Resource id 94d6f5c7-9a07-4c27-9cdb-8b499783571b not committed to the store
    at control-plane/agents/core/src/volume/service.rs:59
    in core::volume::service::destroy_volume with request: DestroyVolume { uuid: VolumeId(94d6f5c7-9a07-4c27-9cdb-8b499783571b, "94d6f5c7-9a07-4c27-9cdb-8b499783571b") }, volume.uuid: 94d6f5c7-9a07-4c27-9cdb-8b499783571b
    in core::volume::destroy_volume with request.service: true

  Feb 22 23:03:41.621 ERROR common: Error processing message: Failed to handle message id 'v0/destroyVolume' on Channel 'v0/volume', details: Storage Error: Volume Config for Resource id 94d6f5c7-9a07-4c27-9cdb-8b499783571b not committed to the store
    at control-plane/agents/common/src/lib.rs:386

So the spec is "dirty", as in not flushed to etcd. We should attempt to sync up, so either we have a bug in that logic or we can't connect to etcd. todo: we probably need a health probe so we can find out about the state of the etcd connection..
Could you please run:
curl http://NODE_IP:30011/v0/volumes/94d6f5c7-9a07-4c27-9cdb-8b499783571b or `kubectl mayastor get volume 94d6f5c7-9a07-4c27-9cdb-8b499783571b -o yaml'

---
spec:
  labels:
    local: "true"
  num_replicas: 2
  operation:
    operation: Destroy
  size:<PHONE_NUMBER>
  status: Deleting
  uuid: 94d6f5c7-9a07-4c27-9cdb-8b499783571b
  topology:
    node_topology:
      explicit:
        allowed_nodes:
          - talos-cp-2
          - talos-cp-3
          - talos-cp-1
        preferred_nodes:
          - talos-cp-1
          - talos-cp-2
          - talos-cp-3
    pool_topology:
      labelled:
        exclusion: {}
        inclusion:
          openebs.io/created-by: msp-operator
  policy:
    self_heal: true
state:
  size:<PHONE_NUMBER>
  status: Faulted
  uuid: 94d6f5c7-9a07-4c27-9cdb-8b499783571b
  replica_topology: {}

Also tried restarting core-agent now, nothing changed.

@tiagolobocastro, one of my nodes went into NotReady. After rebooting it, I have again one pod stuck in ContainerCreating with
  Warning  FailedAttachVolume  23s (x8 over 2m9s)  attachdetach-controller  AttachVolume.Attach failed for volume "pvc-9eef8a66-14b7-4a38-b49d-37db8ee7c787" : rpc error: code = Internal desc = Operation failed: GenericOperation("error in response: status code '412 Precondition Failed', content: 'RestJsonError { details: \"No online replicas are available for Volume '9eef8a66-14b7-4a38-b49d-37db8ee7c787'\", kind: FailedPrecondition }'")

kubectl mayastor get volume 9eef8a66-14b7-4a38-b49d-37db8ee7c787 -o yaml shows that both replicas for that volume are online though:
---
spec:
  labels:
    local: "true"
  num_replicas: 2
  size:<PHONE_NUMBER>
  status: Created
  uuid: 9eef8a66-14b7-4a38-b49d-37db8ee7c787
  topology:
    node_topology:
      explicit:
        allowed_nodes:
          - talos-cp-2
          - talos-cp-3
          - talos-cp-1
        preferred_nodes:
          - talos-cp-2
          - talos-cp-3
          - talos-cp-1
    pool_topology:
      labelled:
        exclusion: {}
        inclusion:
          openebs.io/created-by: msp-operator
  policy:
    self_heal: true
state:
  size:<PHONE_NUMBER>
  status: Online
  uuid: 9eef8a66-14b7-4a38-b49d-37db8ee7c787
  replica_topology:
    af5441f5-3141-410b-8a27-de871fcf7208:
      node: talos-cp-2
      pool: pool-on-talos-cp-2
      state: Online
    56c23152-c87c-4396-9d00-e09e58259336:
      node: talos-cp-1
      pool: pool-on-talos-cp-1
      state: Online

kubectl mayastor get volume 9eef8a66-14b7-4a38-b49d-37db8ee7c787 shows
 ID                                    REPLICAS  TARGET-NODE  ACCESSIBILITY  STATUS  SIZE
 9eef8a66-14b7-4a38-b49d-37db8ee7c787  2         <none>       <none>         Online<PHONE_NUMBER>

I tried restarting all pods in the mayastor namespace, didn't seem to help.
Not sure if this is also related to the same bug.

@tiagolobocastro, I have again encountered a problem with one of my volumes after a (clean, this time) node reboot.
1 replica is online, 1 is degraded, and the corresponding pod is showing:
  Warning  FailedMount  38m (x123 over 23h)    kubelet  Unable to attach or mount volumes: unmounted volumes=[datadir], unattached volumes=[kube-api-access-4bhmw backupdir datadir]: timed out waiting for the condition
  Warning  FailedMount  14m (x376 over 23h)    kubelet  Unable to attach or mount volumes: unmounted volumes=[datadir], unattached volumes=[datadir kube-api-access-4bhmw backupdir]: timed out waiting for the condition
  Warning  FailedMount  3m31s (x695 over 23h)  kubelet  MountVolume.WaitForAttach failed for volume "pvc-5fafd0e8-a753-48cc-b628-bdc58448e5d8" : volume attachment is being deleted

I tried scaling down the volume to 1 replica with kubectl mayastor scale volume 5fafd0e8-a753-48cc-b628-bdc58448e5d8 1 to remove the degraded one, but it refuses with:
Failed to scale volume 5fafd0e8-a753-48cc-b628-bdc58448e5d8. Error error in response: status code '507 Insufficient Storage', content: 'RestJsonError { details: "Storage Error: Nexus Config for Resource id f2a058e9-987b-4140-bc26-06b6a13da4d9 not committed to the store", kind: FailedPersist }'

Why is there insufficient storage for removing a replica?
After shutting down the node hosting the degraded replica, running kubectl mayastor scale volume 5fafd0e8-a753-48cc-b628-bdc58448e5d8 1 produces:
Failed to scale volume 5fafd0e8-a753-48cc-b628-bdc58448e5d8. Error error in response: status code '500 Internal Server Error', content: 'RestJsonError { details: "Internal error: Failed to find the node where a replica lives", kind: Internal }'

I forgot to mention: Even after the node with the degraded replica is rebooted, the volume's rebuildProgress is still stuck at 40.

Yeah, think I found a bug... we're not deleting volumes in the deleting state when we fail the write to etcd, should be a simple fix but sadly it missed the hotfix window (was released yesterday).


NEW EPISODES instead of NEW SEASONS

Hi,
this is not really an issue, but i am wondering if it could be possible to change on the home screen to show newly added episodes instead of newly added seasons?
or if you could point me in the right way to edit this? i am new to php and so on :)
Thanks in advance

Sorry that I never commented on your question.
But this has zero priority for me and to implement such a feature goes certainly beyond the knowledge of a PHP novice. So if I would take the time to guide you through this, I could have done it probably myself in the same time.

alright thank you


[Android] Add W3C Presentation API impl.

BUG=XWALK-4646

Please be more verbose in both your pull request message and commit message. We need to figure out from those what is being done, how it changes the existing implementation and, most importantly, why.

@jondong @pozdnyakov I've just created the PR for Android W3C Presentation API impl. Can you please review the change? (I'm supposing the review happens in a PR, isn't it?)
The current code share the similar design/code with Windows impl. , which are being unified soon... @pozdnyakov I'll start drafting the refactoring proposal, when you have time to do it? Thanks.

Hi @rakuco, roger that. Will do (please allow a few hours buffer, I'm leaving office)

Now github.com is back :)
Not sure why trybot is fail. looking into it right now.

The failure is due to coding style... is there a local tool that can help check that?

@yuankaining please find the lint tool under src/xwalk/tools/lint.py

@halton Yep, thanks. Coding issues were fixed and now I'm running a build to test it before pushing my commit.

@jondong @pozdnyakov could you prioritize your time to review this PR, it is a P1 blocker.

@yuankaining Thanks for the PR, would you please squash these 2 commits into one and force update your branch? We'd better make sure that every commit in trunk passes the trybot. Looking into your implementation...

Thanks to @jondong for the input, is there any other issues need to be taken care of? If not, I'll start to work on those ones :)

Thanks @jondong for the review, all issues you've found are fixed now.

@pozdnyakov Hi, Mikhail, what do you think of this change? Do you have time to review it? Thanks in advance :)

The code looks OK but I was hoping you would share the existing code of the Windows implementation rather than copy it..
if the issue is urgent we can fix it later, otherwise it should be re-written to a class hierarchy like (
generic  XWalkPresentationServiceDelegate and  inheriting                                        XWalkPresentationServiceDelegateAndroid  XWalkPresentationServiceDelegateWin)

+1, there is a lot that can be shared and that something I'm trying to do with new stuff I'm adding. Let's go that way please. This is valid for future contribution, and I'm sure today there is stuff that could be refactored in the Android port to be shared with desktop.

@pozdnyakov Yes, they should share the same copy of code, we've talked about this on email and I already have a plan almost done for your review. The reason I didn't do it now is that I wanted to do it in another PR so that I can close this P1 bug ASAP. This bug was not important at that time and I thought I had plenty of time before it was suddenly marked as P1 :)   I'll send out the plan to you by EOD today.

@pozdnyakov Hi, Mikhail, the refactoring proposal was just sent to you via email...
And from my understanding of your above comment, this PR is ready to merge, is this information correct? I'm still not familiar with the process yet: who will be the one to merge this PR? Are you the one? Thanks :)

And from my understanding of your above comment, this PR is ready to merge, is this information correct?
LGTM considering the promise to share the repeated code

LGTM for the android part, plus @pozdnyakov 's LGTM for the runtime part, @darktears @kenchris could you help to merge this PR?
After it landed, @yuankaining will submit PR to crosswalk-18 branch to fix the P1 blocker .

Please make sure there is a jira issue to track the removal of duplicated code, assigned and targeting Crosswalk 19.
@halton we haven't yet branched 18, so if this lands today or tomorrow it will be in 18 automatically.

@yuankaining: thanks for making the PR message much more detailed than the original version, it's appreciated.

    BUG=XWALK-4646

I'm following other colleagues's commit message style -- one line of brief message, one line of JIRA bug ID. Is this style welcomed? Let me know if there is any existing guidelines on this. Thanks :)

@yuankaining the one line of brief message is not a guideline. You could see lots of commits with very detailed information.

Thanks @halton . I dug a little bit and found the following. There are a number of existing long commit messages :)
$ git log --log-size | egrep "log size [0-9]{4}" | sort | tail
log size 2969
log size 3185
log size 3351
log size 3422
log size 3735
log size 3836
log size 4244
log size 4347
log size 6326
log size 8155

I'm following other colleagues' commit message style -- one line of brief message, one line of JIRA bug ID. Is this style welcomed? Let me know if there is any existing guidelines on this. Thanks :)

Well, I've been bugging your colleagues for years to be less concise, with mixed results ;)
We do have a few commit message guidelines in CONTRIBUTING.md, but they are not very extensive. Section 2 of the Linux kernel's SubmittingPatches file covers the subject in a lot more detail, and the majority of what's written there applies to Crosswalk commit/pull request messages too.
The rule of thumb is: if in doubt, be verbose. There is no reason why a shorter explanation would be better than a longer explanation. Always assume the people looking at your changes are not deeply familiar with what you are doing, so you need to convince them that your patch is worth integrating by describing what was wrong/broken and why your change is fixing things correctly. Not only that, but in a few years you and I might have moved on to a different project, and a new person might have just started working on Crosswalk. They must be able to look at the commit logs and understand why a certain change was made, without having to browse through mailing lists or go ask someone who was around before about it.


F-Droid inclusion

I've opened F-Droid inclusion issue:  https://gitlab.com/fdroid/rfp/issues/842
Please help to make it available.
Could you clarify the fact that "you are not against and support the inclusion".

@Steppschuh in the issue @c-e-p-x-u-o linked, the F-Droid bot reported it found Google Play Services declared in the gradle. That is a show-stopper at F-Droid. Could you provide a FOSS build-flavor?


[SPARK-8743] [Streaming]  Added call to removeSource to help de-register the streaming metrics

Can one of the admins verify this patch?

I think the design of this PR should be the following. The StreamingSource object is currently registered at the time of creating the StreamingContext. Instead it should be registered at the time of starting the context, and deregistered when StreamingContext.stop() is called. Since there can only be one StreamingContext that is active (that is, started and stopped), the problem in the JIRA does not arise.
Furthemore, please add some description in the PR to explain the problem and high design of the solution.

Also, there is way too many unrelated commits in your PR branch. Rather fork a clean branch from the current spark master branch and apply only the relevant commits to it. This keeps the commit logs clean.

TD,
Thanks for the comments.
Makes sense.
Will create a new PR for this one.
I pulled from upstream during my changes so a bunch went in along with my singular file.
Closing this one.


add raw data option  / raw body for POST

It would be nice to have a "add raw data to body-option" to be more flexble with a issue like multipart/form-data.
Enabling you to add an individual raw string to the body.
PS
My attempt to work around that issue in splitting a string at the equals sign and put both halfs in  .data() was ruined by the urlencoding.

Hey guys, yeah it is very annoying having such a great library but so limited at the same time!
If you could provide some tips on where to start, I would also like to chim in

Implemented some time back in Connection.Request.requestBody(String)


[Discussion] Change Language to Large

Hi! This is a follow-up discussion to one that was started in the impact channel, where we feel that Large would be a better candidate than Language.

BLOOM = BigScience Large Open-source Open-access Multilingual


show_progress_bars=False was not turning off the display, now modified

posterior.map() prints the progress bar regardless of setting, so I put in a check

oh I see, I didn't realize where it was passed into sampling, I just thought somebody forgot to add the if statement at some point hahah
could make a different flag, or changing it to verbose, but yeah this is probably the easiest

yes, let's keep it simple, or would you prefer a different tag, @michaeldeistler ?

I would prefer keeping it simple and not introducing another kwarg.


angular 10 help

Hi, i have some problems with angular 10+
I initialized the viewer and the "generate" and "load" events are returned correctly, but it remains "display: none" on the ".tridi-viewer". If I remove the style anyway it doesn't work
import * as Tridi from 'tridi';
[ ............. ]
@ViewChild('tridiRotator') tridiRotatorRef: ElementRef<HTMLDivElement>;
ngAfterViewInit() {
this.tridiView = Tridi.create({
      element: this.tridiRotatorRef.nativeElement,
      location: '../assets/product/spin',
      format: 'jpg',
      count: 35,
      onViewerGenerate: function() {
        console.log('Viewer generated');
      },
      onLoad: function() {
        console.log('Tridi loaded');
      }
    });
    this.tridiView.load()
  }

Solved by passing an ID to "element" instead of this.tridiRotatorRef.nativeElement.
this.render.setAttribute(this.tridiRotatorRef.nativeElement, 'id', `id-${uuidv4()}`);
...
Tridi.create({
element: `#${this.rotator.nativeElement.id}`,
})

@marcof-git thanks for submitting the issue and the resolution! Sorry I was too slow to respond. :(
No promises, but I'll consider adding support for refs as elements.


Union Operator: Property not found in object type.

Using React with Flow, I have a property on state that is initialized as an empty object {} and will have data after a promise resolves. I defined a type for the resolved data object. When I use the union operator to define the 2 possible values of the state I get a Property not found in object type error.
Here's a link to an example in flow try
Code example:
/* @flow */

type Data = {
  	name: string,
}

type State = {
	data: Data | {}
}

const state: State = {
	data: {}
}

const { data } = state
const { name = '' } = data // Property not found in object type

Is there a way for Flow to recognize that state.data.name is a possible value here even if the promise is not yet resolved? I'm not sure how to handle this case of a default state with an empty object.

state.data.name could be made optional.

@idiostruct I had thought about using the optional property and that may be the way to go. In my case the type Data is modeled on an api response that would always have a name property. I had thought that the union operator could could work here. For modeling types on api responses I was hoping to avoid making every property optional, but maybe that is the right idea here.

Another option is to use$Shape<Data>: Try Flow.

@idiostruct thank you! $Shape<Data> looks like it's exactly what I need. I searched the Flow docs but the only reference I found was it's use at the bottom of this page on casting.
Thanks!


Is there a license for the components?

Hello,
I wondered if you could add a license for the components?
For example:
https://github.com/shadcn-ui/ui/blob/main/LICENSE.md
Thanks.

@gregmsanderson  I added MIT license. Thanks.

@zaichaopan Great! Thanks.


Coins shouldn't be dynamic

Coins can actually collide with each other (and other game objects) which is not intended

Fixed; coin colliders now triggers


[$1000] Payment for https://github.com/Expensify/App/issues/21833

Payment for https://github.com/Expensify/App/issues/21833
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01c9acb7291d35f49c
Upwork Job ID:<PHONE_NUMBER>417661440
Last Price Increase: 2023-07-31

This is purely a holding payment issue for https://github.com/Expensify/App/issues/21833

Unsubscribe. 😛


I got error " 'Settings' object has no attribute 'APPS_REORDER' "

I try this package, and i just make default django project and django app. its work when display admin login page, but when login i got error " 'Settings' object has no attribute 'APPS_REORDER' " and can not enter dashboard page.
is there any clue how to solve this? thanks.

just add APPS_REORDER = {} and its works

hi there,
thanks for report, i will fix it
also im working on tox testing to test the app before every release
:+1:


Enhancement: Allow conversion to objects and solid error handling in PowerShell / 'PowerShell integration mode'

Ok, so the issue is twofold:

Posh objects: everybody who uses m365 in PowerShell often uses | Convertfrom-Json to convert the output of the cli to objects for use in scripting.
Error handling: the CLI does not work like PowerShell commandlets. PowerShell has some interesting features here, like being able to configure what a script should do when it runs into an error. (-ErrorAction and $ErrorActionPreference There are some differences between how PowerShell interacts with lowerlying commandline tools. All in all this means that when building a solid script, you should write your own code to do some correct error handling.

We'd like to improve the general e2e flow of PowerShell users in these two regards.
Idea
The idea would be to allow the user to optionally configure the CLI to run in 'PowerShell integration mode' (temporary working title). as an extension of the scripting configuration that's typically configured by the command m365 setup --scripting. For example: a user could run m365 setup --scripting after installation and the CLI would then update the m365.ps1 file that's installed by node.js. See my proposal further down for specifics.
Checkout the screenshot to see how I did that manually and what it causes:

Expanding on that, we could add a function to the file:

A possible Invoke-M365CLICommand function
function Invoke-M365CLICommand {
  [cmdletbinding()]
  param(
    [parameter(Mandatory = $true, ValueFromPipeline = $true)] $input
  )

  $output = $input

  if ($null -eq $output) {
      return $null
  }

  try {
    $parsedOutput = $output | ConvertFrom-Json -ErrorAction Stop
  }
  catch {
    return $output
  }

  if ($parsedOutput -isnot [Array] -and $null -ne $parsedOutput.error) {
    switch($ErrorActionPreference) {
      "Stop" {
        throw $parsedOutput.error
      }
      "Continue" {
        Write-Error $parsedOutput.error
      }
    }
  }

  return $parsedOutput
}

This function tries to convert the output to json, if it doesn't work it will just show the output. (in the case of help)
It also has some basic error handling.

As implied above: this only works with a specific setup of config keys. The following config keys need to be set correctly:

Key
Value
Reason

output
json
We need JSON output to be able to use ConvertFrom-Json.

errorOutput
stdout
Combined with the printErrorsAsPlainText, this is not necessary by principle, but it's the easiest way to get the error without string manipulation of the output.

printErrorsAsPlainText
false
See above...

prompt
false
The CLI Prompt causes the Invoke-M365CLICommand to 'freeze'. See explanation further down.

This is actually the setup we use then running m365 setup --scripting
Challenges and possible solutions
Challenge 1: The CLI Prompt causes the Invoke-M365CLICommand to 'freeze'.

Expand
The prompt in the CLI is run bij inquirer during the execution of the command. This poses a problem. If we run the command using `$output = $input`, the `$output` variable will only be assigned a value after the `$input` has run. In practice this means the prompts will not show to the user, and the CLI seems to freeze.
There seem to be two solutions:

Disable the prompt in this 'PowerShell integration mode'. We could decide this is just not working well together, and that you would typically use this in scripts, not in interactive mode.

We could also build a PowerShell inquirer function to still be able to request information from the user. The CLI could return a JSON-object instructing PowerShell what to request from the user. PowerShell could then request it and re-run the command.

Change the prompt to use stderr. This should be possible according to the inquirer docs.

[ ] #5489

Challenge 2: Native commands (like m365 cli) do not honour PowerShell ErrorAction preferences and show ugly exception stacktraces if they do.

Expand
In PowerShell there is a variable that controls what you would like failing commandlets to cause. It's called `$ErrorActionPreference`. By default it's set to 'Continue', which means that commandlets that fail will not stop the script execution. The script will continue execution of following lines. This is annoying behavior in scripts, where you sometimes would like Exceptions to be stopping further execution of your script. This is why in PowerShell I often set `$ErrorActionPreference = 'Stop'`.
However: Exceptions thrown in in Native commands that are called by PowerShell (cmd files, exe files, bat files, like the CLI for Microsoft 365) do not work well with this currently.
Solution:

In PowerShell 7.3 there is an experimental feature: $PSNativeCommandUseErrorActionPreference
Setting this to true will force the CLI to follow $ErrorActionPreference. Errors will be thrown, but the error text is currently not visible in the thrown error.

In PowerShell 7.2 the feature is not present. Scripts will just continue on.

There are downsides to throwing or writing errors in PowerShell. An ugly error with a useless stacktrace will show, because the command is called from a script file (m365.ps1), which is not what we would want. Clean errors without stacktrace seem hard to get.

There are some ideas on how to get clean errors though:
how to get clean errors without stacktrace
We'd need some research into what would be an optimal solution.

Challenge 3: The login command freezes without showing the device code.
Same as the prompt issue... The login command waits for the user to use the device code. But the device code has not been printed to the screen because the $output-variable is not yet assigned. This poses a problem...
A solution would be to move the device code message to the ora-spinner, which circumvents the stdout stream.

[x] #5484

Challenge 4: The spinner icon has lost it's color.

Proposal command setup
We add a new command to enable integration mode:
m365 cli integrationmode pwsh enable

This will check if the correct configurations are present. If not: It will ask the user to configure the correct properties. If yes: it will overwrite the m365.ps1 file with the correct changes. Any update of the CLI will cause this file to be reset again. So just like with m365 cli completion pwsh <verb>, this command will need to be run again after updating the CLI to a later version.
An extra command will be necessary to be able to disable it again:
m365 cli integrationmode pwsh disable

Also: because the command requires a specific set of configurations, we could append the functionality to the m365 setup command:
m365 setup --scripting --integrationMode PowerShell

The first part (m365 setup --scripting) is our shortcut into setting up the correct configurations for working with PowerShell. The --integrationMode option should not really be necessary, but otherwise this would be a breaking change.
Also: we should implement checks: If a user tries to update one of the required config keys, while in PowerShell integration mode, this should throw an error and ask the user to disable integration mode first.

Awesome idea that we should definitely follow up on to improve the user experience for folks on PowerShell. Let's dig deeper to see what's possible

This is something we definitely need to do. However, if we do this, it will be a breaking change, right?

Who knows... it should be optional and configurable after installation.
However, it could maybe lead to possible stuff that should be changed...and those could be breaking...

Updated the description with some research done on error handling. Also thought a bit about the prompt issues.

@pnp/cli-for-microsoft-365-maintainers : Curious to hear your opinions on this..

Challenge 1
Folks use CLI in PowerShell both interactively and in scripts. We shouldn't assume that they use it only in scripts and should aim for it to be working in both types of usage. Changing the prompt to be sent to stderr seems like the right solution, even outside of PowerShell, so that it's not mixed with the actual command output.
Challenge 2
agreed, let's investigate possible solutions
Challenge 3
device code message should go to stderr indeed, either via ora or directly in CLI
Proposal
I suggest that we integrate the necessary changes in the existing setup command to avoid introducing yet another way of configuring the CLI.
Why would extending the setup command without introducing --integrationMode be a breaking change?

Thanks for responding @waldekmastykarz!

Why would extending the setup command without introducing --integrationMode be a breaking change?

Scripts might break, because people already use ConvertFrom-Json. That would mean the output is converted to json (by our central function) and then piped into ConvertFrom-Json, which will cause exceptions.

I suggest that we integrate the necessary changes in the existing setup command to avoid introducing yet another way of configuring the CLI.

I agree that seems better. There is a downside in my optinion: updating the CLI will cause the functionality to be reset. You would need to rerun the setup command.
Also: what would you suggest: setting it by default when users are in PowerShell? Would it be an extra question? Should you be able to circumvent it? (I think you should)

Scripts might break, because people already use ConvertFrom-Json. That would mean the output is converted to json (by our central function) and then piped into ConvertFrom-Json, which will cause exceptions.

Wouldn't the change only apply to our Invoke-CLIMicrosoft365Command wrapper?

I agree that seems better. There is a downside in my optinion: updating the CLI will cause the functionality to be reset. You would need to rerun the setup command.

I think that this is reasonable: as we're adding new functionality, you might need to reconfigure it to get the latest features in your config setup.

Also: what would you suggest: setting it by default when users are in PowerShell? Would it be an extra question? Should you be able to circumvent it? (I think you should)

I think so. If we can verify then it's something that would reasonably be used by everyone in PowerShell, then I suggest we make it a part of the preset. If there's an exception, folks can always revert a particular setting to a different value.

Wouldn't the change only apply to our Invoke-CLIMicrosoft365Command wrapper?

The whole idea of that commandlet is that its automatically invoked from the m365.ps1 file...
So if I run:
m365 spo site list

What is returned are posh objects.
Writing the following code would then break:
m365 spo site list | ConvertFrom-Json

Hence it would be a breaking change to update the m365 setup command without some additional question or choice.

Any feedback on this one @pnp/cli-for-microsoft-365-maintainers, I know it's quite a read, but it's worth the effort 😊

Hence it would be a breaking change to update the m365 setup command without some additional question or choice.

Understood. Let's make it opt-in to avoid breaking people who have figured out using CLI without these proposed improvements.

I'd say, let's make it opt-in for now and make it the default behavior in v8.

@waldekmastykarz, @milanholemans
m365 setup --usePowerShellIntegration

m365 setup --optInPowerShellIntegration

Additionally: do you have feedback on challenge 2?

Nice job with all this research already @martinlingstuyl. We're definitely missing some integration with our most commonly used shell. Using | Convertfrom-Json is something that is very frustrating to use in scripts. It works most of the time but on occasions, it tends to fail but then you've to convert it to a hash table object. And let's not get started on error handling. That's just a pain.
Regarding your suggestion of enabling integration mode. That would be quite a nice feature out-of-the-block but it would indeed be a breaking change. So I'm also all for adding it as an additional option with m365 setup and with the next mayor making it default. My vote for the option name would be --usePowerShellIntegration. For interactive mode, what about > Do you want to use PowerShell Integration mode? and then clarify this question more in the docs.

Additionally: do you have feedback on challenge 2?

☝ and this one @Jwaegebaert, @waldekmastykarz, @milanholemans?

it tends to fail but then you've to convert it to a hash table object.

By the way: I believe we fixed all occurrences of this. It was due to duplicate properties Id and ID on listitem endpoints.

The variable contents will be converted from json.
If the conversion fails, the raw output is returned. (which will mean text, csv and md modes can still be used)

Couldn't we just check what is the expected output mode and not convert to the variable if it's other than JSON?

I'd say, let's make it opt-in for now and make it the default behavior in v8.

I agree with @milanholemans on this one👍
Challenge 1
Seems ok but

Solution: Pipe the prompt to stderr by configuring inquirer. Also move the device code message to the ora-spinner, which circumvents the stdout stream.

I think we already discussed the device code needs to end up together with prompts in stderr 🤔. Otherwise it is not possible to catch it when using CLI as API in a node.js app
Challenge 2
Seems legit 👍🙂. Maybe we could name the setting promptMessageOutput? Since (if I understand it properly) it's only meant to change the behavior of prompts
Challenge 3
The missing error message color does not seem a problem for me 👍 as long as the error is part of stderr
Challenge 4
This is a bummer 🤔. Won't setting $ErrorActionPreference to continue overwrite what is set by the user in the script?
So if someone has it set to stop if we now set it to continue in our m365-invoke.ps1 (if I understand your proposal properly 😉) won't it like overwrite the default setting?
BTW.
Sorry for the late reply 🙏
Awesome job on setting this all up. You rock 🤩

Seems legit 👍🙂. Maybe we could name the setting promptMessageOutput? Since (if I understand it properly) it's only meant to change the behavior of prompts

It's not prompts here. The point is we want to redirect error messages without redirecting other verbose content like debug logs, verbose logs, prompts and spinners. which is why I proposed errorMessageOutput and verboseMessageOutput. debug & verbose logs, prompts and spinners would then fall under the heading of verboseMessageOutput.
That logical?

This is a bummer 🤔. Won't setting $ErrorActionPreference to continue overwrite what is set by the user in the script?
So if someone has it set to stop if we now set it to continue in our m365-invoke.ps1 (if I understand your proposal properly 😉) won't it like overwrite the default setting?

My idea would be to throw the error if the user has configured $ErrorActionPreference as Stop and to Write-Error the error if the user has configured $ErrorActionPreference as Continue. So we don't overwrite $ErrorActionPreference. We decide what to do with the error based on the users preference. Also, we don't have the -ErrorAction option, so errors can only be influenced by using $ErrorActionPreference.

Sorry for the late reply 🙏

Never mind, thanks for taking the time to read and comment!
Any other opinions? @pnp/cli-for-microsoft-365-maintainers

It's not prompts here. The point is we want to redirect error messages without redirecting other verbose content like debug logs, verbose logs, prompts and spinners. which is why I proposed errorMessageOutput and verboseMessageOutput. debug & verbose logs, prompts and spinners would then fall under the heading of verboseMessageOutput.
That logical?

aaaa ok. clear. Yes, that is logical 🙂

My idea would be to throw the error if the user has configured $ErrorActionPreference as Stop and to Write-Error the error if the user has configured $ErrorActionPreference as Continue. So we don't overwrite $ErrorActionPreference. We decide what to do with the error based on the users preference. Also, we don't have the -ErrorAction option, so errors can only be influenced by using $ErrorActionPreference.

ok clear now as well 👍

It's a bit of a bummer that we'll lose the colors (Challenge 3). If I'm not mistaken, this will mean by default that errors won't be shown in red. I think we better look for a solution regarding this one as we don't have a lot of visual pointers in a CLI so losing our color somewhere would be a big plus that we'll lose.

The color of the error should still be red, because we'll write it ourselves after the CLI throws it and the invoke function intercepts it, @Jwaegebaert.
But I agree with you that loosing colors is indeed a bummer.

Of course, we also get new colors, because PowerShell colorizes the response objects.

Yeah, good point. Lost sight of that one. Then I've nothing further to add 😄


Add nokogiri dependency

The gem currently doesn't declare its dependency on nokogiri. This PR adds it to the gemspec.

Also, I released a new gem version which contains your MR.


Use “OR” for whitespace separation instead of "AND"

Can do this at the level of the query builder in SAGE - possibly here https://github.com/TAMULib/SAGE/blob/main/src/main/java/edu/tamu/sage/service/SolrDiscoveryService.java#L142

This does not require code changes.
It can be performed via the UI on the Discvery View by setting the Default Operand.


[New Language]: Traditional Chinese

Language name
Traditional Chinese (Chinese Taiwan)
Locale code
zh_tw
Please check if the language is already enabled

[X] I have checked this language isn't enabled on Crowdin

Done, thanks for the suggestion.


Can not read TIFF directory count. `TIFFFetchDirectory' @ error/tiff.c/TIFFErrors/598

magick-wasm version
0.0.19
Description
import { ImageMagick, MagickFormat } from "@imagemagick/magick-wasm";

function processImage(data) {
  ImageMagick.read(data, (image) => {
    image.write((rotData) => {
      // Process the image data
    }, MagickFormat.Tiff);
  });
}

// Call processImage multiple times
processImage(imageData);
processImage(imageData);
Steps to Reproduce
data is in jpeg format
Call the processImage function with valid image data.
Call the processImage function again with the same image data.
The error occurs during the second call.
Images
No response

That is indeed what you should do. The rotData is only allocated during the execution of that delegate inside the write method.


Unable to decode messages

After migration to Vertica 12.0.3 (from 12.0.1) I have an issue with decoding messages. I am using cluster with 3 nodes, running query on 3rd node returns an error:
conn_info = {
    'user': '***',
    'password': '***',
    'host': '<node3>',
    'port': 5433,
    'database': 'vertica_db',
    'unicode_error': 'ignore',
}

connection = vertica_python.connect(**conn_info)
cur = connection.cursor()
cur.execute('select 3;')
print(cur.fetchone())

Traceback (most recent call last):
File "test_vertica_python.py", line 14, in <module>
print(cur.fetchone())
File "/home/user/.pyenv/versions/tsg-scripts/lib/python3.7/site-packages/vertica_python/vertica/cursor.py", line 330, in fetchone
self._message = self.connection.read_message()
File "/home/user/.pyenv/versions/tsg-scripts/lib/python3.7/site-packages/vertica_python/vertica/connection.py", line 694, in read_message
message = BackendMessage.from_type(type_, self.read_bytes(size - 4))
File "/home/user/.pyenv/versions/tsg-scripts/lib/python3.7/site-packages/vertica_python/vertica/messages/message.py", line 102, in from_type
return klass(data, **kwargs)
File "/home/user/.pyenv/versions/tsg-scripts/lib/python3.7/site-packages/vertica_python/vertica/messages/backend_messages/command_complete.py", line 58, in __init__
self.command_tag = data.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: unexpected end of data

At the same time the query executes correctly on 1st and 2nd node. The issue also occurred on 2nd node, and for some time worked on 3rd node.
The issue was spotted only with vertica_python, it works fine with pyodbc and vsql, so I suppose it is possible to improve response handling. I'm using vertica_python==1.3.1. To exclude network issue I reproduced the issue on the server (executed the script locally), the error was the same.

I tried to debug the issue, I check the value of fetched bytes on working and bugged host.

Correct one returns: type: b'C', size: 5, bytes: b'\x00'
Incorrect host: type: b'C', size: 6, bytes: b'\xc2\x00'

If I blindly overwrite the value C2 00 to 00 and proceed then it works, returns correct result. Would appreciate help from someone who actually understands the meaning of these codes.

At first, it looked so. But we just upgraded dev environment and I was unable to reproduce the error there. No relation to client upgrade - same error with vertica_python==1.0.5.
Yes
TLS off
Yes. When I replaced select 3; with select <column> from <table>; then it proceeds to RowDescription branch and returns correct response

command tag length: 1
command tag content: b'\xaf'
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaf in position 0: invalid start byte
Server version: 12.0.4
Query executed (if possible): xxx
The OS of each server node (if possible): CentOS 7.9.2009
The locale of each server node (if possible): en_US@collation=binary

I also got the same error on the version 1.3.2
This is the error vertica error: 'utf-8' codec can't decode byte 0xe9 in position 1: invalid continuation byte

@sitingren Did you mean Vertica to version 12.0.4-3 right?


Version code extensions

For projects with multiple APKs it might be nice to slot in other values (API-level, Platform, Screen size, etc.). Right now this is kind of tricky to do.

As mentioned in #31 there could be a user-customizable list of digits to fill in with different content.
Considerations:

Don't exceed<PHONE_NUMBER>. (https://developer.android.com/studio/publish/versioning.html#appversioning)
Consider API Level (e.g. 25=N) and Screen Size (e.g. 34=xlarge) as components of the version code (https://developer.android.com/google/play/publishing/multiple-apks.html#VersionCodes):

So here's a provisional list of inclusions:

SS (screen size)
AA (API Level)
M (major version number 1.x.x)
N (minor version number x.2.x)
P (patch version number x.x.3)
BBBB (build number, e.g. 2838 for number of commits since beginning)
UUU (update number, e.g. 102 for number of updates since last tag)

Others?

As discussed in #31, we would call this codeFormat.
parts and multiplier would be deprecated with this change. They should continue to function as-is where codeFormat is not supplied for backwards compatibility.

(mostly) fixed in 0.3.0 / 52356447

(Didn't do BBBB, SS, or AA. Maybe later.)

Let me know what you think @artky -- hopefully this will address your use case well!


about sigmoid activation use in model:bbox_head

I use the sigmoid instead of the softmax in bbox_head.loss_cls, But the ap is 0 when i test , the  loss keeps decreasing when i training. Did I tuning parameter is wrong?

If you modify it to sigmoid, then the corresponding code also needs to be modified.


Backups Fail on Docker Desktop for Windows

Describe the bug or issue being faced
I had an error occured every backup time.
ex)
[error   ] Unable to access world path at '/backups/world.20220425-141732.mcworld'

Why throw exception always here?
<Sources/Bedrockifier/Model/World.swift>
public struct World {
    public enum WorldType {
        case folder
        case mcworld
        case javaBackup

        init(url: URL) throws {
            let isDirectory = try WorldType.isDirectory(url: url)

The url is '/backups/world.20220425-141732.mcworld'.
These backup files will seem to successful...
But, throw exception "not a directory" to failure process.
Configuration  and Verbose Logs
config.yml:
containers:
  bedrock:
    - name: aaa
      worlds:
        - /data/worlds/world

schedule:
  daily: 04:00
  startupDelay: 2m
  onPlayerLogin: true
  minInterval: 2m
trim:
  trimDays: 2
  keepDays: 14
  minKeep: 14
loggingLevel: trace

Additional context
Log:
aaa-backup-1  | [14:17:32.982][info    ] Starting Backup of worlds for: aaa (ContainerConnection.swift:124)
aaa-backup-1  | [14:17:32.982][trace   ] Data Received on file descriptor (3) (Terminal.swift:296)
aaa-backup-1  | [14:17:32.982][trace   ] Processing Listener (Terminal.swift:318)
aaa-backup-1  | [14:17:32.983][trace   ] Data Received on file descriptor (3) (Terminal.swift:296)
aaa-backup-1  | [14:17:32.983][trace   ] Processing Listener (Terminal.swift:318)
aaa-backup-1  | [14:17:32.986][info    ] Backing Up: world (ContainerConnection.swift:130)
aaa-backup-1  | [14:17:46.551][error   ] Unable to access world path at '/backups/world.20220425-141732.mcworld' (ContainerConnection.swift:134)
aaa-backup-1  | [14:17:46.551][error   ] Backup of world at /data/worlds/world failed. (ContainerConnection.swift:135)
aaa-backup-1  | [14:17:46.551][error   ] Backups for aaa had failures... (ContainerConnection.swift:155)

Thank you for your help.

This looks like there might be some permissions related error at play here, where the file is created and written to, but cannot be read from. It's not throwing because the file isn't a directory, but because it can't determine if it is a directory. i.e. something's blocking reading the file's attributes.
Can you share your docker-compose.yml, and ls -l on the backup files (the mcworld files) that are being created on the host so we can see what user is winding up owning them, and how the service is configured? Also, parts of the verbose log from the start would be helpful here as well, as it would give an idea of what permissions the service is running with.

I also first suspected access rights.
I entered the CLI and checked it, but I was able to access it as a file without any problems.
Since the world data is large, I will try to make the minimum configuration that can be confirmed to be reproduced.
root@e8bd8f3c3d5b:/# ls -al /backups
total 585836
drwxrwxrwx 1 root root      4096 Apr 26 13:53 .
drwxr-xr-x 1 root root      4096 Apr 26 13:45 ..
-rwxrwxrwx 1 root root       272 Apr 25 14:16 config.yml
-rw-r--r-- 1 root root 599890209 Apr 26 13:50 world.20220426-134955.mcworld

aaa-backup-1.log
<docker-compose.yml>
version: "3"

services:
  aaa:
    container_name: aaa
    image: itzg/minecraft-bedrock-server
    ports:
      - "19136:19136/udp"
    tty: false
    stdin_open: true
    restart: on-failure:1
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "./data:/data"
    environment:
      TZ: "Asia/Tokyo"
      LANG: "ja_JP.UTF-8"
      LANGUAGE: "ja_JP"
      SERVER_NAME: "AAA World"
      SERVER_PORT: "19136"
      EULA: "TRUE"
      LEVEL_TYPE: "default"
      LEVEL_NAME: "world"
      GAMEMODE: "survival"
      DIFFICULTY: "hard"
      ALLOW_CHEATS: "true"
      MAX_PLAYERS: "20"
      ONLINE_MODE: "true"
      ALLOW_LIST: "false"
      DEFAULT_PLAYER_PERMISSION_LEVEL: "visitor"
    labels:
      - manymine.enable=true
    networks:
      - proxy_default

  backup:
    image: kaiede/minecraft-bedrock-backup
    restart: on-failure:1
    depends_on:
      - "aaa"
    environment:
      TZ: "Asia/Tokyo"
      LANG: "ja_JP.UTF-8"
      LANGUAGE: "ja_JP"
    tty: true
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "./backups:/backups"
      - "./data:/data"
    networks:
      - proxy_default

networks:
  proxy_default:
    external: true

I can also make changes so that the underlying error is more completely reported, and push that out to the test tag which will give us more details and help diagnose similar issues in the future. I’ll do that tomorrow.
And yes, it does look like the backup is being made okay. But somehow the file then can’t be checked to confirm we got a valid backup afterwards. Strange. It’s possible that something about the size of the backup is somehow triggering a bug in Swift on Linux that will need to be worked around.

I found successful case...
<docker-compose.yml>
  volumes:
#   - "./backups:/backups" ... throw exception in "Docker Desktop for Windows".
      - "aaa-backups:/backups" # using docker volumes

volumes:
  aaa-backups:

It may be a problem due to the permissions of Windows local folder!

I am not understand swift.
If possible, how about outputting detailed information as follows?
<World.swift>
        private static func isDirectory(url: URL) throws -> Bool {
            do {
                let values = try url.resourceValues(forKeys: [.isDirectoryKey])
                return values.isDirectory == true
            } catch {
+		Library.log.trace(error.localizedDescription)
                throw WorldError.invalidUrl(url: url)
            }
        }

It may be a problem due to the permissions of Windows local folder!

Very possibly. I don't have any experience with Docker Desktop on Windows. I wonder if it uses WSL, and if it doesn't, if Docker in WSL behaves any better.

If possible, how about outputting detailed information as follows?

That was close to my intent yes, but it was close to midnight local time, so I wasn't going to jump into it until I had gotten some sleep. :)
I've made the change, and merged it into master, so it will be available on the test tag once this build run completes: https://github.com/Kaiede/Bedrockifier/actions/runs/2227958575

aaa-backup-1-20220427.log
[error detail]
[16:44:25.995][error   ] Unable to access world path at '/backups/world.20220427-164425.mcworld': Error Domain=NSCocoaErrorDomain Code=260 "The file doesn’t exist." (ContainerConnection.swift:134)

[console in aaa-backup-1]
root@829f2da6288d:/opt/bedrock# ls -al /backups
total 184
drwxrwxrwx 1 root root   4096 Apr 27 16:44 .
drwxr-xr-x 1 root root   4096 Apr 27 16:43 ..
-rwxrwxrwx 1 root root    269 Apr 27 16:43 config.yml
-rw-r--r-- 1 root root 183610 Apr 27 16:44 world.20220427-164425.mcworld

One last thing I want to confirm, if you would.
Is it entirely in the Linux filesystem in WSL? Or is it hosted on a Windows drive?

It hosted on a Windows drive.
ex)
C:\MCS\docker\aaa\docker-compose.yml

Docker Desktop for Windows version 4.7.1 (77678)
aaa.zip

start up command:
C:\MCS\docker\aaa>docker compose up -d
[+] Running 3/3
 - Network aaa_default     Created                                                                                0.0s
 - Container aaa           Started                                                                                2.0s
 - Container aaa-backup-1  Started                                                                                2.3s

stop down command:
C:\MCS\docker\aaa>docker compose down
[+] Running 3/3
 - Container aaa-backup-1  Removed                                                                                10.4s
 - Container aaa           Removed                                                                                0.7s
 - Network aaa_default     Removed                                                                                0.2s

Yeah, I wonder if you’ve uncovered an issue when bind-mounting to the Windows filesystem which has a couple known limitations. Docker themselves recommend bind-mounting to the Linux filesystem in WSL instead: https://docs.docker.com/desktop/windows/wsl/. There might also be some differences depending on if your setup using WSL 1 or 2 as well. Do you happen to know which version of WSL is being used?
Since named volumes are part of the Linux filesystem, I suspect the recommendation of bind-mounting to the Linux filesystem will also work. Could you try it?

Do you happen to know which version of WSL is being used?
I'm using WSL 2.

Could you try it?
Yes. I installed Ubuntu with Docker on WSL2.
It was successful!  (It was using Docker Engine of Docker Desktop.)

[Ubuntu shell on Linux filesystem]
/opt/docker/aaa$ docker compose up -d
[+] Running 3/3
 ⠿ Network aaa_default     Created                                                                                0.0s
 ⠿ Container aaa           Started                                                                                0.6s
 ⠿ Container aaa-backup-1  Started                                                                                1.0s

But, In the following case has other trouble occurred.
[PowerShell on Linux volume via windows filesystem]
PS Microsoft.PowerShell.Core\FileSystem::\\wsl$\Ubuntu\opt\docker\aaa> docker compose up -d
[+] Running 3/3
 - Network aaa_default     Created                                                                                0.0s
 - Container aaa           Started                                                                                0.8s
 - Container aaa-backup-1  Started                                                                                1.1s

[log]
DEBU[0000] Using /backups to match uid and gid
DEBU[0000] Resolved UID=1000 from match path
DEBU[0000] Resolved GID=999 from match path
[17:41:19.907][info    ] Initializing Bedrockifier Daemon
[17:41:19.908][info    ] Loading Configuration From: /backups/config.yml
[17:41:19.910][info    ] Configuration Loaded, Entering Event Loop... (main.swift:84)
[17:41:19.919][info    ] Host PTY Handle Opened: /dev/pts/1 (6) (Terminal.swift:141)
[17:41:19.920][info    ] Child PTY Handle Opened: /dev/pts/1 (7) (Terminal.swift:162)
[17:41:19.920][debug   ] Process attached (Terminal.swift:81)
[17:41:19.920][debug   ] Starting Container Process (ContainerConnection.swift:69)
[17:41:19.921][info    ] Checking for servers that might not be cleaned up (BackupActor.swift:73)
[17:41:19.921][info    ] Backup Time: 04:00 (Service.swift:162)
[17:41:19.921][info    ] Next Backup: Apr 29, 2022, 4:00:00 AM (Service.swift:169)
[17:41:19.921][debug   ] [interval] Scheduled For Apr 29, 2022, 4:00:00 AM (ServiceTimer.swift:73)
[17:41:19.921][info    ] Starting Listeners for Containers (Service.swift:188)
[17:41:19.921][debug   ] [interval] Registered (ServiceTimer.swift:102)
[17:41:19.921][info    ] Backup Minimum Interval is 120.0 seconds (Service.swift:92)
[17:41:19.939][trace   ] Data Received on file descriptor (6) (Terminal.swift:296)
[17:41:19.939][trace   ] Processing Listener (Terminal.swift:318)
[17:41:19.940][trace   ] Data Received on file descriptor (6) (Terminal.swift:296)
[17:41:19.940][trace   ] Processing Listener (Terminal.swift:318)
[17:41:19.942][trace   ] Process terminated (Process.swift:41)
[17:41:19.942][debug   ] Calling process detach handlers (Terminal.swift:94)
[17:41:19.942][debug   ] Process detached (Terminal.swift:99)

Process terminated...
I will switch to WSL + docker soon... (T-T)

Hello @Kaiede, I'm not much of a programmer but I think I've made some headway resolving this.  I too ran into the same problem that @nakanotti ran into.  First, with respect to this code in world.swift (Line 36):
   init(url: URL) throws {
        let isDirectory = try WorldType.isDirectory(url: url)
        if isDirectory {
            self = .folder
        } else if url.pathExtension == "mcworld" {
            self = .mcworld
        } else if url.pathExtension == "zip" {
            self = .javaBackup
        } else {
            throw WorldError.invalidWorldType
        }
    }

I changed it to:
    init(url: URL) throws {
        if url.pathExtension == "mcworld" {
            self = .mcworld
        } else if url.pathExtension == "zip" {
            self = .javaBackup
        } else {
            let isDirectory = try WorldType.isDirectory(url: url)
            if isDirectory {
                self = .folder
            } else {
                throw WorldError.invalidWorldType
            }
        }
    }

This passed the initial hangup that @nakanotti mentioned.  Specifically, the url.resourceValues(forKeys: [.isDirectoryKey] function in line 50 always threw an error (because the URL passed was a file, not a folder).  No idea why but this always problematic.
    private static func isDirectory(url: URL) throws -> Bool {
        do {
            let values = try url.resourceValues(forKeys: [.isDirectoryKey])
            return values.isDirectory == true
        } catch {
            throw WorldError.invalidUrl(url: url, innerError: error)
        }
    }

However I encountered another problem after that.  I narrowed it down to an issue with the .part file extension.  I just disabled that temporary by commenting out lines 147 and 160 respectively:
// let tempUrl = url.appendingPathExtension(World.partialPackExt)
// try FileManager.default.moveItem(at: tempUrl, to: url)
After that, everything seemed to begin working correctly... or at least it no longer gave me errors.  I haven't determined if the .part file extension was somehow related to the first problem (maybe there needs to be a case for .part in the init?).  However, this suggests that the overall problem is not likely related to a Docker Desktop WSL/WSL2, Windows, or a permissions problem.  It feels more like a race condition and/or a problem with the .part file extension not being handled properly.  Once again, I'm not much of a programmer (never touched Swift until now) but I'm pretty good at mashing keys and doing Google searches.  This is my first time posting on GitHub.  I guess I really, really wanted your backup system to work for me.  LOL.  I hope this can provide some guidance on where to go next.
BTW, now that I have this working, wow this is awesome.  I absolutely love having my Bedrock worlds backup after exiting my server.  Way better than crappy batch file scripts I put together for daily backups.  Thanks a ton!!!

Specifically, the url.resourceValues(forKeys: [.isDirectoryKey] function in line 50 always threw an error (because the URL passed was a file, not a folder). No idea why but this always problematic.

That's a bug in the Swift Foundation library, or WSL. It shouldn't be throwing, but instead just set the key to false like Linux and MacOS does.
By moving it, it does work around the issue, but it also means that if I have a folder I've called "foo.zip" then the logic breaks in new ways as it tries to unzip a folder. Fun. But this could make it easier to look up the issue and see if there might be fixes in newer versions of Swift, or if it is fixed in the upcoming 5.7 release.
Do you have the error from the part file issue? It sounds a lot like WSL doesn't like the move of the temporary file to the final location, but the error would be valuable in understanding why.

Hi @Kaiede - I have messed with the code a bit more to resolve your file/folder concern.  So far, this seems to work:
    init(url: URL) throws {
        let isFile = try WorldType.isFile(url: url)
        if isFile {
            if url.pathExtension == "mcworld" {
                self = .mcworld
            } else if url.pathExtension == "zip" {
                self = .javaBackup
            } else {
                // Not a mcworld or zip file...
                throw WorldError.invalidWorldType
            }

        } else {
            let isDirectory = try WorldType.isDirectory(url: url)
            if isDirectory {
                self = .folder
            } else {
                // What in the heck am I??
                throw WorldError.invalidWorldType
            }
        }
    }
    private static func isFile(url:URL) throws -> Bool {
        do {
            let values = try url.resourceValues(forKeys: [.isRegularFileKey])
            return values.isDirectory == false
        } catch {
            Library.log.info("isFile resourceValues problem")
            throw WorldError.invalidUrl(url: url, innerError: error)
        }
    }

    private static func isDirectory(url: URL) throws -> Bool {
        do {
            let values = try url.resourceValues(forKeys: [.isDirectoryKey])
            return values.isDirectory == true
        } catch {
            throw WorldError.invalidUrl(url: url, innerError: error)
        }
    }

With respect to errors after this change, I get the following (after I log out of my server):
MinecraftTEST     | [2022-08-11 15:39:06:935 INFO] Player connected: FarmerTip, xuid: XXXXXXXXXXXXXXXX
BackupSystemTest  | [11:39:06.520][info    ] Player Logged In: MinecraftTEST, Players Active: 1
BackupSystemTest  | [11:39:19.108][info    ] Player Logged Out: MinecraftTEST, Players Active: 0
BackupSystemTest  | [11:39:19.108][info    ] Running Single Backup for MinecraftTEST
MinecraftTEST     | [2022-08-11 15:39:19:526 INFO] Player disconnected: FarmerTip, xuid: XXXXXXXXXXXXXXXX
MinecraftTEST     | save hold
MinecraftTEST     | Saving...
MinecraftTEST     | save query
MinecraftTEST     | Data saved. Files are now ready to be copied.
MinecraftTEST     | MinecraftTEST/db/000025.log:26818, MinecraftTEST/db/000026.ldb:118504, MinecraftTEST/db/CURRENT:16, MinecraftTEST/db/MANIFEST-000023:355, MinecraftTEST/level.dat:2544, MinecraftTEST/level.dat_old:2544, MinecraftTEST/levelname.txt:13
BackupSystemTest  | [11:39:21.285][info    ] Starting Backup of worlds for: MinecraftTEST
BackupSystemTest  | [11:39:21.297][info    ] Backing Up: MinecraftTEST
BackupSystemTest  | [11:39:21.423][error   ] The operation could not be completed. A file with the same name already exists.
BackupSystemTest  | [11:39:21.423][error   ] Backup of world at /MinecraftTEST/worlds/MinecraftTEST failed.
MinecraftTEST     | save resume
BackupSystemTest  | [11:39:21.423][error   ] Backups for MinecraftTEST had failures...
MinecraftTEST     | Changes to the world are resumed.
BackupSystemTest  | [11:39:21.431][error   ] Single Backup for MinecraftTEST failed
BackupSystemTest  | [11:39:21.432][error   ] Server container had worlds that failed to backup: /MinecraftTEST/worlds/MinecraftTEST

I was digging deeper into this last weekend.  I seem to recall removing the .part file extension naming/renaming stuff and it resolved the problem.  The good thing is, the file seems to back up fine regardless of the error.  I'll try diving deeper in the next few days to pinpoint what specifically is triggering the erroneous error.  Thanks!

Oh, my attempt to determine if the file/folder is a file might not have worked.  I reverted to the code in the first post (without using resourceValue), I get the following:
MinecraftTEST     | [2022-08-11 16:06:46:211 INFO] Player connected: FarmerTip, xuid: XXXXXXXXXXXXXXXX
BackupSystemTest  | [12:06:46.797][info    ] Player Logged In: MinecraftTEST, Players Active: 1
MinecraftTEST     | [2022-08-11 16:07:28:314 INFO] Player disconnected: FarmerTip, xuid: XXXXXXXXXXXXXXXX
BackupSystemTest  | [12:07:28.913][info    ] Player Logged Out: MinecraftTEST, Players Active: 0
BackupSystemTest  | [12:07:28.915][info    ] Running Single Backup for MinecraftTEST
MinecraftTEST     | save hold
MinecraftTEST     | Saving...
MinecraftTEST     | save query
MinecraftTEST     | A previous save has not been completed.
MinecraftTEST     | save query
MinecraftTEST     | Data saved. Files are now ready to be copied.
MinecraftTEST     | MinecraftTEST/db/000035.log:221870, MinecraftTEST/db/000036.ldb:118456, MinecraftTEST/db/CURRENT:16, MinecraftTEST/db/MANIFEST-000033:416, MinecraftTEST/level.dat:2544, MinecraftTEST/level.dat_old:2544, MinecraftTEST/levelname.txt:13
BackupSystemTest  | [12:07:41.010][info    ] Starting Backup of worlds for: MinecraftTEST
BackupSystemTest  | [12:07:41.017][info    ] Backing Up: MinecraftTEST
BackupSystemTest  | [12:07:41.137][error   ] World archive is not a valid zip or mcworld file
BackupSystemTest  | [12:07:41.137][error   ] Backup of world at /MinecraftTEST/worlds/MinecraftTEST failed.
MinecraftTEST     | save resume
BackupSystemTest  | [12:07:41.137][error   ] Backups for MinecraftTEST had failures...
MinecraftTEST     | Changes to the world are resumed.
BackupSystemTest  | [12:07:41.163][error   ] Single Backup for MinecraftTEST failed
BackupSystemTest  | [12:07:41.164][error   ] Server container had worlds that failed to backup: /MinecraftTEST/worlds/MinecraftTEST

Slightly different results.  Makes me think that .isRegularFileKey doesn't work either.  At least this method pointed to the .part file extension problem I mentioned earlier.  Seems clear that the ability to distinguish between a file and folder (or how it's handled in Docker) is the culprit.  Maybe there's another way around this...

Hmm, ideally, I'd love to be able to attack this with some form of unit testing, but it requires a pretty specific environment: Linux container being run on Windows, with the files being accessed from the host. So it's more a big integration test that requires setup to perform. Maybe I could setup a Dockerfile specifically to run the test suite from within a docker container, but it'd take some work.
Out of curiosity, how did you setup this environment on Windows? While I don't currently have something setup, I could probably setup a Windows VM to replicate this, and then start playing around with possible solutions using a swift development docker container. So long as the POSIX APIs work correctly, it should be possible to drop down to the C APIs and call them directly to check for file vs directory. At least as long as the Zip framework doesn't have issues of its own.

Well, my setup isn't probably the most efficient by any means.  I've been running your build-docker.sh script on my mid-2015 Macbook Pro to build the image.  From there I tag it and push to docker hub.  Then I hop over to my Windows box where I've got a quick-and-dirty docker-compose configuration which spins up the minecraft-bedrock-backup (obviously pulling the latest version of my own hackery) and a single instance of the itzg/minecraft-bedrock-server.  Lastly, I jump into the Minecraft server long enough to trigger the backup event and begin troubleshooting.  I know, not great but it's working.  The Windows box is where my servers will ultimately reside once this is resolved.

Hi @Kaiede - I think I've got a decent solve for this.  In World.swift (starting at line 31), the following change resolved the problem with distinguishing between a folder and file on my system:
    public enum WorldType {
        case folder
        case mcworld
        case javaBackup

        init(url: URL) throws {
            // let isDirectory = try WorldType.isDirectory(url: url)

            // if isDirectory {
            if url.hasDirectoryPath {
                self = .folder
            } else if url.pathExtension == "mcworld" {
                self = .mcworld
            } else if url.pathExtension == "zip" {
                self = .javaBackup
            } else {
                throw WorldError.invalidWorldType
            }
        }

        // private static func isDirectory(url: URL) throws -> Bool {
        //     do {
        //         let values = try url.resourceValues(forKeys: [.isDirectoryKey])
        //         return values.isDirectory == true
        //     } catch {
        //         throw WorldError.invalidUrl(url: url, innerError: error)
        //     }
        // }
    }

With that resolved, I also found a work-around with the ".part" file problem.  For some reason FileManager.default.moveItem doesn't play nice.  Maybe the newly renamed file isn't ready to be opened right after it was renamed.  Regardless, in World.swift around line 160 I did the following which solved that problem:
        // With it packed successfully, rename it.
        // try FileManager.default.moveItem(at: tempUrl, to: url)
        try FileManager.default.copyItem(at: tempUrl, to: url)
        try FileManager.default.removeItem(at: tempUrl)

So with those two changes, everything appears to be working correctly on my setup.  I haven't had a chance to see if there's errors during the trim process yet.  I'll have to figure out a way to simulate that.  For now, I'm off to look at my eyelids.
MinecraftTEST     | [2022-08-14 05:59:00:862 INFO] Player connected: FarmerTip, xuid: XXXXXXXXXXXXXXXX
BackupSystemTest  | [01:59:00.943][info    ] Player Logged In: MinecraftTEST, Players Active: 1
MinecraftTEST     | [2022-08-14 05:59:02:728 INFO] Player disconnected: FarmerTip, xuid: XXXXXXXXXXXXXXXX
BackupSystemTest  | [01:59:02.851][info    ] Player Logged Out: MinecraftTEST, Players Active: 0
BackupSystemTest  | [01:59:02.851][info    ] Running Single Backup for MinecraftTEST
MinecraftTEST     | save hold
MinecraftTEST     | Saving...
MinecraftTEST     | save query
MinecraftTEST     | Data saved. Files are now ready to be copied.
MinecraftTEST     | MinecraftTEST/db/000103.log:0, MinecraftTEST/db/000104.ldb:146090, MinecraftTEST/db/CURRENT:16, MinecraftTEST/db/MANIFEST-000101:250, MinecraftTEST/level.dat:2544, MinecraftTEST/level.dat_old:2544, MinecraftTEST/levelname.txt:13
BackupSystemTest  | [01:59:04.487][info    ] Starting Backup of worlds for: MinecraftTEST
BackupSystemTest  | [01:59:04.492][info    ] Backing Up: MinecraftTEST
BackupSystemTest  | [01:59:04.839][info    ] Backed up as: MinecraftTEST.20220814-015904.mcworld
BackupSystemTest  | [01:59:04.839][info    ] Backups for MinecraftTEST finished successfully...
MinecraftTEST     | save resume
MinecraftTEST     | Changes to the world are resumed.
BackupSystemTest  | [01:59:04.914][info    ] Performing Trim Jobs
BackupSystemTest  | [01:59:04.947][info    ] Single Backup Completed

I guess my question was more: Are you just using Docker Desktop on Windows or something else?

With that resolved, I also found a work-around with the ".part" file problem. For some reason FileManager.default.moveItem doesn't play nice. Maybe the newly renamed file isn't ready to be opened right after it was renamed.

Aha, this might explain the issue. Windows tends to be a bit more picky about when you can manipulate a file on disk, and will usually lock it from being renamed so long as it remains open. Something I just re-learned the last time I was playing with Windows a few days ago.
Because the Archive is not closed until the object deallocates, then the file cannot be moved until that happens. Linux/BSD/Mac will let you rename the file while it is still open. So the full fix here is to ensure the Archive is deallocated before we rename. So let's see if that works before falling back to copies, as a copy will be more expensive in time and disk space, especially on Java.
As for the URL fix, that does generally work so long as the URLs properly act the same way all the time. hasDirectoryPath is really only checking for the presence of a trailing slash, which in this case may be working, but could cause problems down the road. I still suspect I'll need to implement something at the POSIX layer.
But thanks for digging into this so far, it has been a huge help narrowing things down while I've been distracted by real life and a separate (large) side project.

Yes, just running docker desktop for windows.
Same here, life has been nuts (dealing with a family member's estate, my truck got totaled by a distracted driver, multiple work projects and deadlines, daughter starting kindergarten soon, etc.).  I'm looking forward to August being done but September is shaping up to be just as crazy.
For the short-term, this has dropped in my priority but I'm hoping I'll be able to circle back to it in a month or so.  Sounds like we both need a well deserved break.

Initially, I used the same approach as Mr. @farmertip.
However, the main container "itzg/minecraft-bedrock-server" itself has better performance on linux volume,
so I moved everything to "\wsl$\Ubuntu\opt\docker\bedrock"...
ex) start up script file from Windows: \wsl$\Ubuntu\opt\docker\bedrock\up.ps1
wsl -d Ubuntu -- docker compose up -d

We can also add the following message to the error message, as another approach...
If you are using Docker Desktop for Windows, we recommend using a Linux volume.
By the way, I hope that the image size will be smaller.. (^_^;

I've make changes to ensure the Archive is closed before moveItem() is called on the file, and will get it merged as part of this PR: #58
@farmertip This hopefully should fix half of the problem, leaving just the isDirectory concern behind. I still don't yet have a Windows VM with Docker setup, so if you do happen to find some time to take a look, it'd be great. Otherwise I'll take a look once I have that environment available to me.

By the way, I hope that the image size will be smaller.. (^_^;

Unfortunately, most of the fat comes from the base images I use. I can trim about 9MB if I leave out bedrockifier-tool from the container.


piVPN with pihole

Hi,
I have setup pihole with piVPN, i have assigned static virtual IP for VPN clients, I want to access pihole from <IP_ADDRESS> only and block access from other clients. How do I do it? Is it possible with iptables?

Solved @ https://discourse.pi-hole.net/t/restrict-access-to-pihole/3397/10


[tests] Refactor SqlServer tests to use testcontainers

Microsoft Reviewers: Open in CodeFlow

@eerhardt Are there any tests that I missed here?

I made the suggested changes. And also had to update ActivitySourceName:
-    protected override string ActivitySourceName => "OpenTelemetry.Instrumentation.SqlClient";
+    protected override string ActivitySourceName => "Aspire.Microsoft.Data.SqlClient";

-    protected override string ActivitySourceName => "OpenTelemetry.Instrumentation.SqlClient";
+    protected override string ActivitySourceName => "Aspire.Microsoft.EntityFrameworkCore.SqlServer";

does that change look correct?


Replace the Logo of Layer5 with Meshery in config.yml workflow

Description

Currently, we are using the Logo of Layer5 instead of Meshery. This repository is under the organization Meshery, so, there should be the image of Meshery.
How to resolve this?

Go to .github and replace this welcome image with the new one:

Make sure to display the images conditionally, based on the theme selected by user.
Here's the sample code:

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/GiorgosXou/Random-stuff/main/Programming/StackOverflow/Answers/70200610_11465149/w.png">
  <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/GiorgosXou/Random-stuff/main/Programming/StackOverflow/Answers/70200610_11465149/b.png">

</picture>

New images are:

Download these images, and upload with an appropriate name. Make sure to use conditional rendering as described above.

After uploading the image, and removing the previous image, go to .github/config.yml folder and update the reference to previous image to the new image in the following part:

firstPRMergeComment: >
  Thanks for your contribution to the Layer5 and Meshery community! :tada:

  ![Congrats!](https://raw.githubusercontent.com/meshery/meshery.io/master/.github/welcome/Layer5-celebration.png)
    	   	    	    	 :star: Please leave a [star on the project](../stargazers). :smile:

Here, replace ![Congrats!](https://raw.githubusercontent.com/meshery/meshery.io/master/.github/welcome/Layer5-celebration.png) with the code
<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/GiorgosXou/Random-stuff/main/Programming/StackOverflow/Answers/70200610_11465149/w.png">
  <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/GiorgosXou/Random-stuff/main/Programming/StackOverflow/Answers/70200610_11465149/b.png">

</picture>

make sure to change the image sources with the path to image in the folder where you upload it.

That's it.

Contributor Guides and Handbook

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

@thisiskaransgit kindly assign me this issue

Hi, Can I work on this issue?

@Amukayoung since you are already assigned a similar issue, assigning @Shyam-12 for this one 🚀

Hi @thisiskaransgit , I have figured it out but there was a small doubt, There is the main "meshery" repo and "meshery-istio", both have the config.yml file and the welcome image, but the meshery repo does not have the "welcome" folder and image is uploaded directly inside of assets folder and meshery-istio matches the above folder details but not the code snippet you have provided. So, can you just quickly hint me where to make changes, I will do it right away, little confused. Thank You!

Hi @thisiskaransgit , I have made the changes,  I should be pulling request on the meshery folder right where I have made the changes, but this issue is not there on the main meshery folder. Please guide me a little bit, a little confused, I have done all the changes and committed but not sure where to pull request? Thank You!

@Shyam-12
Here's how you can go ahead:

Create a new config.yml inside the .github folder. You may copy the contents for this file from other repos inside Meshery Organization.
Add the welcome images to either .github/welcome folder, or ./github/assets/images/welcome folder. You would need to research on how to create directories in a GitHub repo. (Hint: Drag and drop a similar folder from the local machine).
Update the config.yml with correct srcset to raw images which you just uploaded.

Hi @thisiskaransgit , I have made the changes, I should be pulling request on the meshery folder right where I have made the changes, but this issue is not there on the main meshery folder. Please guide me a little bit, a little confused, I have done all the changes and committed but not sure where to pull request? Thank You!

The Pull Request will go inside this repository only. Changes also need to be done inside this repository.

Alright got it, I thought I needed to update the meshery repo. Will be making the changes in few minutes. Thank you!

@Shyam-12  mention the fixed issue in the PR description, to link it.

I have added the PR description, please check and let me know if it's fine
or not.
On Thu 18 May, 2023, 12:33 AM Karan Thakur, @.***>
wrote:

@Shyam-12 https://github.com/Shyam-12 mention the fixed issue in the PR
description, to link it.
—
Reply to this email directly, view it on GitHub
https://github.com/meshery/meshery-cpx/issues/104#issuecomment-1551911631,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARKRO4E47OC32V2LRFIB4ITXGUOJRANCNFSM6AAAAAAW3XYRYA
.
You are receiving this because you were mentioned.Message ID:
@.***>


To be able to run js code by the press of a button.

The button should somewhere in the right top corner.
Press the button, output should be sent to the console.
I know there are extensions like code runner, but that overwrites the python runner

Please try to start debugging with the builtin JS debug support https://code.visualstudio.com/docs/editor/debugging#_run-and-debug-view


workerThreadPool is initialized lazily

Hi,
We want to use Apache Kafka in our project, and the parallel-consumer library meets our expectations.
However, we would like to use it in a way that fits our architecture.
What do I want to achieve?
I would like to have control over the worker thread pool definition. We have our abstraction over thread pools that allows us to monitor them easily, and we have one place where we have all thread pool definitions.
What steps have I taken?
I created a class that inherits from ParallelEoSStreamProcessor, and I see that the setupWorkerPool(int poolSize) method in AbstractParallelEoSStreamProcessor is protected, so I overridden it and used my ThreadPoolExecutor implementation.
Nonetheless, I get a NPE because the setupWorkerPool method is called in the constructor of the AbstractParallelEoSStreamProcessor class, so it happens before the fields are assigned in my custom superclass (via the constructor). The solution is quite simple; that is, workerThreadPool is initialized lazily. The overridden abstract method will be called only the first time it is used, and then we can be sure that class, which inherits from ParallelEoSStreamProcessor, will be fully initialized.
What do you think about this change?
By the way, I'm just reading the javadoc of ParallelConsumerOptions regarding maxConcurrency and I see this sentence:

When using getBatchSize(), this is over and above the batch size setting. So for example, a getMaxConcurrency() of 2 and a batch size of 3 would result in at most 15 records being processed at once.

I don't understand where 15 came from, could you clarify that?

When using getBatchSize(), this is over and above the batch size setting. So for example, a getMaxConcurrency() of 2 and a batch size of 3 would result in at most 15 records being processed at once.

I don't understand where 15 came from, could you clarify that?

Yeah that sounds complete wrong. From a quick look, it should read 6, but I'll take a look.

Your proposal seems good - just remove the Guava dep and we should be good to go.

I didn't notice that there is zero~ dependencies in the features list. I removed Guava dependency and I wrote my own cached supplier implementation based on the information here: https://stackoverflow.com/a/35332514


hello, how to disable the popover in the month view?

Hi I just want to use the month view. and don't want to show anything when click the cell.
How to disable it?
Thank you very much.

Hi,
In your MGCMonthPlannerEKViewController subclass, you can return NO from monthPlannerView:shouldSelectEventAtIndex:date:

thanks for reply


When is folia-1.21 compatible?

CreeperRecover-1.0.12-all.jar doesn't work in folia-1.21

This plugin is great!


Document NSAllowsArbitraryLoadsInWebContent

There's a permission key in iOS called NSAllowsArbitraryLoadsInWebContent which generally speaking should be added to your iOS project if you're using RN WebView. Let's document that.

You can find more about this key here: https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsallowsarbitraryloadsinwebcontent

From what I understand, you're not able to load http (insecure) web pages without this key. I could be wrong.

From what I understand, you're not able to load http (insecure) web pages without this key. I could be wrong.

Yes! You are right, its a common mistake on IOS. I see on some places that Android 9 will need a similar config to enable HTTP calls


POST http://api.zinger.pw/platform/v1/auth/login net::ERR_NAME_NOT_RESOLVED

Hi @harshavardhan98
This issue is regarding ERR_NAME_NOT_RESOLVED when I try to login on http://platform.zinger.pw:4200/auth/login.
It is hitting an API endpoint - http://api.zinger.pw/platform/v1/auth/login and ERR_NAME_NOT_RESOLVED. How do I host this API backend? is it available in zinger-core? If so can you please point me to to the link for postman collection for these API endpoints?

The backend for this project: https://github.com/zinger-framework/zinger-rails
The documentation for this project is not yet completed, but please follow these 2 steps to get your setup done:

Add the below line to your hosts file as you did here.

<IP_ADDRESS>	api.zinger.pw

Navigate to rails project root directory & run docker-compose up. The application will be running in api.zinger.pw:80

@jayanthrbharadwaj
FYI Updated documentation for Zinger Rails project here.
https://github.com/zinger-framework/zinger-rails#readme


Make blocking behavior configurable

See: https://github.com/Annoraaq/grid-engine/discussions/161

closed with 2.5.0


Allow launcher use while offline

Launcher should allow use while offline (as long as there is a UUID and name to use). Obviously the session token will be invalid (find a way to make the launcher okay with this).

You know, this is still an issue. On real hardware Windows XP, I cannot log in in spite of being logged in before online.

yeah I think it's a regression in 183b744ef9ccf63d0db1d9c8f16676d723f05ed5

yeah I think it's a regression in 183b744

When do you think a patch would be available in release? I'm only asking because I don't want to fight Multi-MC on Windows XP, And I want the nostalgic look to remain.

I'm not sure, however I promised that the next bug I would work on is the resource-mapped bug related to pre-1.6 game assets so this will have to be fixed after that.


Control Node Transform Field removed by parent container.

Godot version
4.0 alpha 10
System information
Windows 10
Issue description
When you have a control node who's transform is being controlled by a parent, Hbox, MarginContainer, ect, The transform field is removed. I understand not wanting the user to be able to change it but that is where information about the node's current size is. This is critical information for keeping track of UI scales across scenes and maintaining UI consistency.
Here is a basic use case. I want to make a button in a HBox perfectly square. I cannot do this if I don't know it's current size.
Please leave it and make it read only.
Steps to reproduce

Minimal reproduction project
No response

Isn't that what the Container Sizing group is for? Disable horizontal and vertical stretching, set an appropriate minimum size, and make sure that the contents of the button are properly clipped.

That would work I suppose, but it seems to be a workaround. Why do i need to do so many steps just to get basic info like rect size? How hard would it be to make it a read only field when a child? In the example you gave, container is greyed out so I know it is possible.

I am aware and specifically complaining about it.
My particular use case is that I try to match UI Proportions across nodes in a scene and between scenes. I also make heavy use of containers. Even though I may not be setting it directly, I often do my adjustments in the parent node as seems to be the expected behavior pattern, I am checking rect_size hundreds of times in a work session.
Because it obscures that data, Godot 4 will greatly reduce my ability to work efficiently. I made a complaint in the discord a few days ago and a few people were also frustrated with this UI behavior. Thus, I came here to post and hopefully get it changed while still in alpha.

So you need to be able to see and change those values at design time and not at runtime?

So you need to be able to see and change those values at design time and not at runtime?

I need to see them even if I cannot change them. I have no problem with the paradigm of child sizes being controlled by the parent container.

This is caused by PR #55157.
I'm trying to debug a UI, and part of this process is that I need to see how big something is at runtime. Since Godot no longer displays the position and size for children of a container in the remote debugger, this makes it impossible to debug my issue without lots of print statements in my code to print out the size.


Feature: Aggregate LNS logs by device on the portal

Is your feature request related to a problem? Please describe.
When I'm deploying a device, I'd like to have information about :

Is my device trying to connect?
Is my device sending some message?
Is the LNS authenticating the device well?
...

Describe the solution you'd like
In addition to LSNR and RSSI data that we already have, it is useful to get LNS logs per devices and provide the user a way to see these logs on the portail in order to diagnose some connection issues...
Describe alternatives you've considered
N/A
Additional context
Add any other context or screenshots about the feature request here.

@adcoly @Metal-Mighty, we discussed about that but as fare I know you didn't create an issue for that?

I don't think we ever created an issue about that indeed. We have partially implemented it in our stack with contextual information added to the logs of our IoT Edge modules, but we currently don't have these information in the LNS. I'd be glad to discuss it with you over the next week !

This issue will be discussed this week during our PBR, don't hesitate to join us at this moment ;)

We will take messages coming from the LoRaWanNetworkSrvModule.
The events will be filtered by the sender id (using the convention device-id/module-id.
We will assume by default that module-id is LoRaWanNetworkSrvModule and will let it configurable.
We will keep by default 100 event logs per device and let it also configurable.

During the implementation, we will take the same job that is actually reading events from IoT Hub (same consumer group).
Attention
In the logs it could have some secrets (the auth code to the Azure Function), take it in account during the documentation, to provide insights to the users and encourage to use v2.2 of the starterkit that might have fixed that)