[video_player] add buffering controls

I'd like the ability to pause/resume buffering on video_player, why?
We've got a situation where we have multiple videos loading offscreen and we'd like video_player to buffer only enough to keep up once the user navigates to view that video. As far as I know, video_player currently buffers 100% of the video as quickly as possible which is normally appropriate but not when you have 6-12 videos buffering slightly off screen.

Is there some new about this buffering ? ty

network video is still slow. buffering in not stop on video pause.

It's been 9 months! @lukepighetti please email me if you've found a solution for your use case, I have the same exact use case (jpiabrantes@gmail.com).

Sorry, I never found a good solution for this.

Any update ?
@lukepighetti , @jpiabrantes did you find any solution ?

Sorry, I never found a good solution for this.

This is a very critical issue actually. For use cases like mine where you have a feed of many videos, the buffering must be dependent on the current position of playback, in the current state it will buffer 100% of the video which increases the data consumed by the application a lot. In regions where people do not all have WiFi connections with unlimited data available, this causes them to hit their data limits within a very short span of time and provides a very negative impression of the app. Once this happens, they may never come back.
Can we do something to control the percentage of video that is buffered ahead?
cc: @TahaTesser

This is a very critical issue actually. For use cases like mine where you have a feed of many videos, the buffering must be dependent on the current position of playback, in the current state it will buffer 100% of the video which increases the data consumed by the application a lot. In regions where people do not all have WiFi connections with unlimited data available, this causes them to hit their data limits within a very short span of time and provides a very negative impression of the app. Once this happens, they may never come back.
Can we do something to control the percentage of video that is buffered ahead?
cc: @TahaTesser

This is needed, or the video player for mobile is not something we can use for a case like social media feed that loads lots of videos in scrolling up and down.

Just my opinion, this is likely to require a community PR with full test coverage before anyone will triage it.

No solution found yet to this issue?

The solution is to make a PR with a solution. :)

Is there any alternatives for this?

Please is there any update on this? @lukepighetti

Doesn't look like it

No solution found yet to this issue?

Any solution?

I too have the same issue now, Can anyone have the updates?

kinda sad tbh

Any updates?

how are you guys able to load lot of videos without the video player crashing?

Any update about this issue?

If you load lots of videos using a real device, it will definitely crash the video player. And for data consumption, what i did was write  an algorithm that initialize new videos, dispose videos while the user scroll. The number of videos i initialize are 5[]

If you load lots of videos using a real device, it will definitely crash the video player. And for data consumption, what i did was write an algorithm that initialize new videos, dispose videos while the user scroll. The number of videos i initialize are 5[2 before, 2 after and the current video playing]

@wanoghoco can you provide me a sample code of how this is done? As I am doing exact same thing

@Helocominc i will guide u what to do, tell me is a listview, pageview or what  u are using

I am using listview builder

@Helocominc i will try to upload a sample code to explain better

@wanoghoco that will be great . Looking forward to it.

@Helocominc i will try to upload a sample code to explain better

Hello please can you post the sample code here??

Any update about this implementation yet.

@Helocominc i will try to upload a sample code to explain better

Any update on this implementation yet ?

The truth is, it's still hitting a high bandwidth after controlling the number of initialized videos. Flutter video player keeps buffering the videos over and over again. That 1 issue. That is what is making me hit a high bandwidth. Right now exo player has a documentation for catching while buffering instead of downloading again. Am thinking on integrating that on the plugin

I noticed the buffering issue in the flutter video player. Try using working on the plugin and let me know what you got. This feature is very important.

The truth is, it's still hitting a high bandwidth after controlling the number of initialized videos. Flutter video player keeps buffering the videos over and over again. That 1 issue. That is what is making me hit a high bandwidth. Right now exo player has a documentation for catching while buffering instead of downloading again. Am thinking on integrating that on the plugin

@wanoghoco did you got anything? Seems this issue is really hard. Will be great if you share what you got so far.

Nothing yet, what am currently doing is initializing only 3videos and making sure it stays that way even while the user is scrolling to a new video, i dispose older once and remove from the list of initialized videos. But wat am planning to do is to use flutter cache manager, once the app is loaded, if the video does not exist, it start downloading it while streaming the video, when it fully downloaded switch to local  and download 5  videos ahead, to save bandwidth, instead of streaming the videos all the time.

That's a good a idea, but downloading the video and caching it, isn't it going to eat up device memory, like when downloading multiple videos?

I think I would have to wait for your solution, as of now I don't know how to tackle this issue.

I think I would have to wait for your solution, as of now I don't know how to tackle this issue.

It's easy 😂😂😂  here us the solution.
https://honesdev.com/playing-and-caching-videos-in-flutter/
Just try and think how to maintain the cache, it's not here, u can research on that

I just found an answer on SO about the disadvantage of video caching with cache manger

Another Solution: Is to stream the video normally using the official plugin and to cache the video file using flutter_cache_manager simultaneously.
But this will lead to fetch the video twice the first time (Once for streaming through the video_player, Another for downloading the video through the cachemanager)

Are you aware about this issue using cache manager?

I just found an answer on SO about the disadvantage of video caching with cache manger

Another Solution: Is to stream the video normally using the official plugin and to cache the video file using flutter_cache_manager simultaneously.
But this will lead to fetch the video twice the first time (Once for streaming through the video_player, Another for downloading the video through the cachemanager)

Are you aware about this issue using cache manager?

U didn't get what i said.
if it does not exist start downloading and stream online, after downloading switch to local or u can decide to stream without downloading the first video that the user is streaming.
but start downloading the rest videos in the list instead of of initialing the list if video controllers.
You just have to pattern everything properly for it to work.

Okay. Will be waiting for your solution.

In the video_player, I don't think it's currently available to control how many amounts of videos are to be buffered.
But I did it by changing the native iOS code into a dependency file.
for iOS we have one option to set a limit on buffering.
Property is preferredForwardBufferDuration.
For More info: https://developer.apple.com/documentation/avfoundation/avplayeritem/1643630-preferredforwardbufferduration

Any updates

Has there been any work on this lately? It would be great to have buffering controls and a way to stop buffering, de-initializing controller causes the video to slow down everytime we try to re-visit them.

Has anyone figured out any issue for this. My app which uses pageview builder for displaying videos loads 3 pageviews at time initializing 3 videos at a time on page change it loads another one. But the concern here is that for loading just 3-4 videos which are approximately 6-8 MB in size app consumes 80-90 MB of data which is huge for the users runnig app on mobile network.
@TahaTesser kindly help in this situation.

For people still looking for a solution, i've found a player that let's you configure the buffer. it's called better Player.
Documentation : https://jhomlala.github.io/betterplayer/#/README
buffering documentaion. https://jhomlala.github.io/betterplayer/#/bufferingconfiguration
ps: if you have other player packages such as video player or chiwie it will cause a problem so make sure to remove them and run flutter clean.

Any updates on this? @lukepighetti do you have any workarounds? would love to see some implementation that would cache HLS streams, like how instagram reels, tiktok does.

For people still looking for a solution, i've found a player that let's you configure the buffer. it's called better Player. Documentation : https://jhomlala.github.io/betterplayer/#/README buffering documentaion. https://jhomlala.github.io/betterplayer/#/bufferingconfiguration
ps: if you have other player packages such as video player or chiwie it will cause a problem so make sure to remove them and run flutter clean.

It doesn't seems to work on the web and non-mobile platforms though.

and it's only for android, plus having a function to call that actually stops/starts buffering would be amazing


Translation / i18n of Autotune?

Hello All,
we're quite happy using Autotune but one thing we are missing is a Spanish version. I'm looking at implementing it using something along the lines of Polyglot ( http://airbnb.io/polyglot.js/ ) or i18n-js ( https://github.com/fnando/i18n-js ).
Is anybody else interested or working on this too? I'd really like to have this functionality upstream to avoid having a fork just for this.
@ryanmark , @kavyasukumar any thoughts on this?
Regards

Hi! That would be fantastic.
Are you thinking of setting up the front-end UI to get translated? I think that is straight forward enough. I think it might start getting tricky if we need to translate the backend app or blueprints.
Also the js front-end is mostly processed by node and browserify so the rails i18n-js might not be appropriate. Polyglot.js looks appropriate tho. I think the rails side might just use rails builtin i18n stuff.
We will want to throughly review the PR, and I'm happy to discuss your implementation plan. I wouldn't want you to put a bunch of work in before filing the PR and then have us ask you to change a bunch of stuff. Or maybe get a basic proof of concept going and open a PR for us to see where you're going with it.

Hi! That would be fantastic.
Are you thinking of setting up the front-end UI to get translated? I think that is straight forward enough. I think it might start getting tricky if we need to translate the backend app or blueprints.

Glad you like it. For now we just need to translate the front-end as most of our expected users aren't English savvy and we'll have to write our blueprints anywayts.

Also the js front-end is mostly processed by node and browserify so the rails i18n-js might not be appropriate. Polyglot.js looks appropriate tho. I think the rails side might just use rails builtin i18n stuff.

Perhaps I'm not understanding well i18n-js but from what I looked it would let us have all the translations in one place as it generates another js file from the Rails i18n mechanism to be used by the front-end.

We will want to throughly review the PR, and I'm happy to discuss your implementation plan. I wouldn't want you to put a bunch of work in before filing the PR and then have us ask you to change a bunch of stuff. Or maybe get a basic proof of concept going and open a PR for us to see where you're going with it.

I don't mind being asked to change things if that leads to a proper solution. Let me implement this with both libraries and then we'll see what fits better the project.
Thanks for answering so fast.

@ryanmark I set up a couple of templates using the i18n-js library here: https://github.com/LNtools/autotune/tree/i18n-js
This is still not complete, there are lots of things to translate yet and language switching is not implemented.
I'm not sure if I did something wrong when building application.js, the file grew a lot.
Do you think this can work?
Thanks.


ADD: Add new fields for AQ cookbook data portals

Add in new fields required for AQS and NASA Earthdata environment variables

@brian-rose - this adds new environment variables required for the air quality cookbook. Can you please review + merge here? So we can test downstream to ensure the secrets are passed properly in the build?

@ktyle feel free to take a look as well!

@mgrover1 this looks good but I'm a bit ignorant of how secrets should work in our workflows and actions. To test things, I created a fork of cookbook-actions based on your branch, and then added the AQS_USERNAME and AQS_KEY as secrets (using the email address and key that I received from the AQS site), based on https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions. I then forked the api-cookbook and created a PR (ktyle/api-cookbook#1) whose trigger-book-build action points to my fork of cookbook-actions. However, the build failed and it didn't seem like these two secrets were applied.
Additionally, in our ProjectPythia/cookbook-actions repo, I don't see any secrets set.
What might I be missing?

As far as I understand things, @ktyle's build failure should be expected because builds triggered by a PR from a different org (in this case, an individual user) won't have access to secrets set in the upstream repository.

I will merge so @mgrover1 can test functionality! We can go back and fix things as necessary.
There's a clear need for a more modular system for declaring secrets so we don't have to specify every possibility in these reusable workflow. But we're not there yet.

As far as I understand things, @ktyle's build failure should be expected because builds triggered by a PR from a different org (in this case, an individual user) won't have access to secrets set in the upstream repository.

Would this be the same issue with PR builds failing when sent from my fork to the Project Pythia cookbook? As far as I can tell, my builds are still failing when I push commits to my fork (and when PRs are initiated to the main repository).

@AMDeitsch Probably not, unless you are working with a cookbook that uses secrets for data access. Which cookbook repository are you opening PRs to?

Thanks, @brian-rose . It looks like @mgrover1 got the issue resolved in subsequent PRs.


Options -f [filesize] and --max-filesize are both broken with Pinterest

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.03.03
[x] I've checked that all provided URLs are alive and playable in a browser (kinda: I'm providing reproduction steps so the link is working, but it can't simply be opened on the browser as it's a .m3u8 link)
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-f', u'[filesize<10M]', u'https://v.pinimg.com/videos/mc/hls/eb/87/05/eb870543b922a1915f2d7e39f3cadd12.m3u8']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.03.03
[debug] Python version 2.7.16 (CPython) - Darwin-20.3.0-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] eb870543b922a1915f2d7e39f3cadd12: Requesting header
[generic] eb870543b922a1915f2d7e39f3cadd12: Downloading m3u8 information
ERROR: requested format not available
Traceback (most recent call last):
  File "/Users/mattiagiuffrida/Library/Python/2.7/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 806, in wrapper
    return func(self, *args, **kwargs)
  File "/Users/mattiagiuffrida/Library/Python/2.7/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 838, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/Users/mattiagiuffrida/Library/Python/2.7/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 872, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/Users/mattiagiuffrida/Library/Python/2.7/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 1671, in process_video_result
    expected=True)
ExtractorError: requested format not available

Description
I'm trying to download a video from Pinterest with multiple formats available, so I'm putting a constraint on the filesize to avoid downloading videos that are too big. However, when I do that, I get the error that you can see in the verbose log above: ExtractorError: requested format not available. I've been able to pinpoint the problem to the filesize format constraint because I did the following tests:

First, I checked for available formats, as you can see the best video is less than 4MB:

$ /usr/local/bin/youtube-dl -v -F https://v.pinimg.com/videos/mc/hls/eb/87/05/eb870543b922a1915f2d7e39f3cadd12.m3u8

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://v.pinimg.com/videos/mc/hls/eb/87/05/eb870543b922a1915f2d7e39f3cadd12.m3u8']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.03.03
[debug] Python version 2.7.16 (CPython) - Darwin-20.3.0-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] eb870543b922a1915f2d7e39f3cadd12: Requesting header
[generic] eb870543b922a1915f2d7e39f3cadd12: Downloading m3u8 information
[info] Available formats for eb870543b922a1915f2d7e39f3cadd12:
format code  extension  resolution note
642          mp4        240x424     642k , avc1.4d4015, 25.0fps, mp4a.40.5
1026         mp4        360x640    1026k , avc1.77.30, 25.0fps, mp4a.40.5
1186         mp4        480x852    1186k , avc1.77.30, 25.0fps, mp4a.40.5
1765         mp4        640x1136   1765k , avc1.4d401f, 25.0fps, mp4a.40.5
2867         mp4        720x1280   2867k , avc1.64001f, 25.0fps, mp4a.40.5 (best)

Then, I tried my command with a filesize limit of 10MB, which failed:

$ youtube-dl -v -f "[filesize<10M]" https://v.pinimg.com/videos/mc/hls/eb/87/05/eb870543b922a1915f2d7e39f3cadd12.m3u8

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-f', u'[filesize<10M]', u'https://v.pinimg.com/videos/mc/hls/eb/87/05/eb870543b922a1915f2d7e39f3cadd12.m3u8']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.03.03
[debug] Python version 2.7.16 (CPython) - Darwin-20.3.0-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] eb870543b922a1915f2d7e39f3cadd12: Requesting header
[generic] eb870543b922a1915f2d7e39f3cadd12: Downloading m3u8 information
ERROR: requested format not available
Traceback (most recent call last):
  File "/Users/mattiagiuffrida/Library/Python/2.7/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 806, in wrapper
    return func(self, *args, **kwargs)
  File "/Users/mattiagiuffrida/Library/Python/2.7/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 838, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/Users/mattiagiuffrida/Library/Python/2.7/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 872, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/Users/mattiagiuffrida/Library/Python/2.7/lib/python/site-packages/youtube_dl/YoutubeDL.py", line 1671, in process_video_result
    expected=True)
ExtractorError: requested format not available

I then tried removing the constraint on the filesize, which resulted in the video being correctly downloaded.

$ youtube-dl -v https://v.pinimg.com/videos/mc/hls/eb/87/05/eb870543b922a1915f2d7e39f3cadd12.m3u8

# Log here is really long, so I won't put it all, but it ends with a video of 3.45MB being successfully downloaded
[ffmpeg] Downloaded 3614800 bytes
[download] 100% of 3.45MiB in 00:00

I even tried the max-filesize option, but in this case I got the opposite issue: the option is simply ignored...

$ /usr/local/bin/youtube-dl -v --max-filesize 2m https://v.pinimg.com/videos/mc/hls/eb/87/05/eb870543b922a1915f2d7e39f3cadd12.m3u8

# Same really long log as step 3, ending in
[ffmpeg] Downloaded 3614800 bytes
[download] 100% of 3.45MiB in 00:00
# Which is surprising, since I limited the `max-filesize` to `2MB`

I'm unsure if I'm doing something wrong here, but the issue is easily reproducible and I'd really like some help to understand what's going wrong 😄
Thanks!

the file size is not known before finishing the download process for HLS formats.

Thanks for the prompt reply @remitamine!
Do you have any suggestion on how to overcome this limitation? Maybe a less elegant but at least working solution?

passing webpage URL to youtube-dl, you should be able to access a direct mp4 URL that would work with -max-filesize option.


typed-rest-client

Experiment with using typed-rest-client typed REST and HTTP Client

Building result in errors like the following
The package "fs" wasn't found on the file system but is built into node. Are you trying to bundle
  for node? You can use "--platform=node" to do that, which will remove this error.

It does seems like this library is supposed to be functional in the browser / web apps without the NodeJS runtime?
Support using in a web page
Browser Support
Enable web usage without webpack


Type 'void' has no call signatures.

when i upgrade to v1.0.14 , i got some error.

v1.0.13 it's working fine. ↓↓↓↓↓↓↓↓↓↓↓↓↓

Can you reproduce this problem with vue-tsc v1.0.14?

yes ,it's still error.

Can you get this error with run npx vue-tsc --noEmit?

no error output, i'll upload case later.

Can you try to increase "volar.vueserver.maxFileSize": 20971520 in VSCode setting?

Thanks for the repro case!


Update ingestion-properties.md

make the creationTime example less granular (day bin, instead of a precise timestamp)

@vladikbr : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@vladikbr - thanks for the change. In the future, make sure you work on dataexplorer-docs-pr not dataexplorer-docs. See: here for more information and to install SpineEdit so changes are automatically made in the correct location.


Subscription id handling in ReadClient is weird

Problem
In ReadClient, we:

Read and try to verify SubscriptionId values even if we're not a subscription.  This is pretty broken.
Enforce that the SubscriptionId in a Subscribe Response matches the previous Report Data, and fail if it does not.  The spec does not say to do this.  This might need to be a spec issue instead.  It's really not clear to me why the spec has a SubscriptionId in Subscribe Response at all, given that it's in the Report Data bits....
Have an mIsPrimingReports that seems to be used only for mSubscriptionId management and does not have clearly defined semantics.  It does have a name that suggests it should be true during the entire priming read, though, which it's not....

Proposed Solution
Align with spec, make code understandable, add tests.

for 2, spec issue is here https://github.com/CHIP-Specifications/connectedhomeip-spec/issues/5525


feat(applepay): expose isAvailable

See Jira: https://commercetools.atlassian.net/browse/SCC-2362
This PR exposes the isAvailable method for Apple Pay so we can control when it should and should not be rendered in the checkout. It uses Adyen's isAvailable helper.

BTW maybe it would be nice to implemente the isAvailable for every payment method in adyen as all the component expose that function.

BTW maybe it would be nice to implemente the isAvailable for every payment method in adyen as all the component expose that function.

I ended up moving up the logic to base.ts so we cover all Adyen's payment methods.


Can't get it to run on windows with node 22.2.0

i installed get-windows@latest and @mapbox/node-pre-gpy always compalins that  "this module supports node-api version 9. this node instance cannot run this module. the node-api version of this node instance is 8
but node -p "process.versions.napi" says that 9 is active.
i already deleted all nodemodules updated global npm installation cleared cache... everything
on linux it runs fine with the same node and npm version
any ideas? what did i forget?

@valueerrorx - any update on this, experiencing the same issue.

@valueerrorx  - I'm also facing the same issue.

@sindresorhus duplicate of #184

It should already be using 9, so I'm not sure what the problem is: https://github.com/sindresorhus/get-windows/blob/6e84e52e43469e2836f5b11e9de5d239a9963b54/package.json#L32-L34

Oh sorry, I see I subtly misinterpreted the issue. I guess actually #184 could be closed in favor of this in that case?
@valueerrorx I wonder if you've tried 22.8.x since?


$.remove  element input and select

Hi. When trying to do a $.remove( selector ) it does not remove form elements input and select.
To Reproduce
https://codepen.io/nafurset/pen/QWNOoWJ
Expected behavior
I expected that the rendered object would be removed from DOM
Desktop (please complete the following information):

macOS Catalina 10.15.6
Chrome 84.0.4147.125
Metro Version  4.3.9

Additional context
I have also tried to uso Metro.destroyPlugin  and Metro.destroyPluginAll  with no success.
Regards.

most of the components have a complex structure, so it is not enough to remove the base element, its wrapper will remain. Example, To remove select you must call api method destroy, this method removes all event handlers and return base element. Then you must wrapper and remove it:
var sel = Metro.getPlugin(el, "select");
var el = sel.destroy();
var wrapper = $(el). closest (".select");
wrapper.remove();

Perfecto, yes I added a  wrapper around the element then just removed it.
Thanks !


Verify that the instructions for changing the icon font are still valid

https://github.com/softlayer/sl-ember-components/tree/v0.11.0#icons compared to reference to fonts in the .sl-ember-components-icons declaration in addons.less

@juwara0 please assign me this.

@erangeles please read my comment on icon classes issue. The instructions should stay the same I believe, as we will not be changing the icon classes to be dynamic.

@notmessenger please close this as icon classes will not be dynamic


Attempted fix for constant database operations

Attempted fix to remove the constant database operations triggered whenever the staff is updated. The way it works is that it caches the staff description into the spellcasting entry flags, then whenever the staff is updated, it compares the cached and current descriptions. If they match, nothing the module cares about should have changed, and so there's no need to update the spellcasting entry. This should prevent the constant database operations whenever the staff is otherwise changed, eg. the staff changes grip.

This looks like it would help a lot, and is a very minor change.


https://ngva-test.synxis.com/

its not working in IE11 please help me on this qucik on i dont know how to fix
https://ngva-test.synxis.com/

Unfortunately we can’t guess what the issue is without details about the error and the relevant source code.

Thanks for Reply, i fixed it
{hotel.HotelName}
{hotel.HotelName}
Not Implemented


V003

Handle empty directories in the poster folder.
Handle more than two colls. Also, name the first and second tags right and left for backward compatibility. This could be implemented inside the list comprehension to handle single-column posters but my poster is due this week :).
Ignore hidden directories in max_file_time. This ignores .git so I like it but it could go either way.

Okay all the issues you mentioned should be fixed


fix(customization): added microservice path to redirect uris

we only registered ..services/customization and ..hub-service/customization but not microservice/customization.

:tada: This PR is included in version 6.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


project with workspace fails to load local dependency and instead tries to compile against older version on crates.io

Problem
project root
Cargo.toml
[workspace]
members = ["core", "nonblocking", "blocking", "bindings/python"]
resolver = "2"

[workspace.package]
version = "0.2.0"
......

[workspace.dependencies]
links_core = { path = "./core" } # published on crates.io as 0.1.0, note that I have completely removed version
links_nonblocking = { path = "./nonblocking" } # published on crates.io as 0.1.0, note that I have completely removed version
links_blocking = { path = "./blocking" } # published on crates.io as 0.1.0, note that I have completely removed version
links_bindings_python = { path = "./bindings/python" } # not published
......

bindings/python/Crago.toml
[package]
name = "links_bindings_python"
version.workspace = true
authors.workspace = true
readme.workspace = true
license-file.workspace = true
edition.workspace = true
rust-version.workspace = true
repository.workspace = true
keywords.workspace = true
categories.workspace = true

[dependencies]
links_nonblocking = { workpace = true, optional = true }
......

[features]
default = ["dep:links_nonblocking"]
full = ["unittest"]
unittest = [
    "dep:pyo3-log",
    "dep:links_nonblocking",
    "links_nonblocking/unittest",
]

when running cargo tree in the bindings/python folder resolves to rm Cargo.lock;  pushd bindings/python && cargo tree | less ; popd
links_bindings_python v0.2.0 ( .... links/bindings/python)
├── cfg-if v1.0.0
├── links_nonblocking v0.1.0.  # still resolves to 0.1.0 instead of 0.2.0 which is the only available local version
│   ├── bytes v1.5.0
│   ├── byteserde v0.6.2
│   │   └── bytes v1.5.0
│   ├── chrono v0.4.31
│   │   ├── iana-time-zone v0.1.59
│   │   │   └── core-foundation-sys v0.8.6
│   │   └── num-traits v0.2.17
│   │       [build-dependencies]
│   │       └── autocfg v1.1.0
│   ├── lazy_static v1.4.0
│   ├── links_core v0.1.0

Steps

try building this https://github.com/softstream-link/links/tree/v2

Possible Solution(s)
The only workaround is to this patch to the workspace which according the documentation is not required for project with workspace
Root Cargo.toml patch
[patch.crates-io]
links_core = { path = "./core" }
links_nonblocking = { path = "./nonblocking" }

Documentation reference
https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#multiple-locations
It is possible to specify both a registry version and a git or path location. The git or path dependency will be used locally (in which case the version is checked against the local copy), and when published to a registry like [crates.io](https://crates.io/), it will use the registry version. Other combinations are not allowed. Examples:

Notes
No response
Version
No response

I ran
$ git clone https://github.com/softstream-link/links.git
...
$ cd links
$ git checkout v2
...
$ cargo check
warning: /home/epage/src/personal/dump/links/bindings/python/Cargo.toml: dependency (links_nonblocking) specified with out providing a local path, Git repository, version, or workspace dependency to use. This will be considered an error in future versions
warning: /home/epage/src/personal/dump/links/bindings/python/Cargo.toml: unused manifest key: dependencies.links_nonblocking.workpace
...
$ # ... fixed warning
$ rm Cargo.lock
$ cargo tree --manifest-path bindings/python/Cargo.toml
    Updating crates.io index
links_bindings_python v0.2.0 (/home/epage/src/personal/dump/links/bindings/python)
├── cfg-if v1.0.0
├── links_nonblocking v0.2.0 (/home/epage/src/personal/dump/links/nonblocking)
...

I assume you are in the latest stable. How did you fix warning? Can you please share.

Latest stable.
Notice the "unused manifest key" warning about workpace.  It is missing an s.  Changing that to workspace removed both warnings.

Thanks, that was the problem.


Postpone app rendering until app state is loaded

Care to comment?
// src/common/app/start.js
constructor(props) {
  super(props);
  this.state = {
    stateFromStorageLoaded: false
  };
}

componentDidMount() {
  const { dispatch } = this.props;
  // Client side changes must be dispatched after componentDidMount, aka
  // the first app render, to match client and server HTML. Otherwise,
  // React attempt to reuse markup will fail.
  dispatch(firebaseActions.watchAuth(logout));
  dispatch(updateAppStateFromStorage()).finally(() => {
    this.setState({ stateFromStorageLoaded: true });
  });
}

render() {
  if (!this.state.stateFromStorageLoaded) return null;
  // ...
}

Hmm, for some reason server logs Warning: React.createElement(...): Expected props argument to be a plain object. Properties defined in its prototype chain will be ignored.
Maybe it's not the right approach anyway. I think what we need is app reducer with stateFromStorageLoaded bool and native switch. In the browser reading from localStorage is sync, so we UI is updated immediately without flashes.

Not sure if it is the same thing, but I am using: https://github.com/makeomatic/redux-connect as an HOC (kind of like what you had before)

No, that's completely different issue. I don't like any HOC depending on react-router, because in Este components can fetch data even they are not route handlers. Thank you anyway.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target/pull/876 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-target.git

Docs Build status updates of commit 7bbc709:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error-GitBranchDeletedOrForcePushed] Cannot sync git repo to specified commit because branch Release_Archive_master_2021-06-02-07-58-43 has been deleted or has been force pushed

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


re-send class_prepared signal for self-referenced ForeignKeys

pr for issue #1395

Can we add some comments explaining what the new lines do?

in a new pr? or just add them here?
but as i mentioned in my initial issue, it not my code, and i linked to the source.
The way i understand how it works is like this
if (extra_fields):
prevents the class_prepared being fired unnecessarily
del fields[model_path]

removes an already processed field
class_prepared.send(sender=sender)

process any remaining fields

Re-sending class_prepared sounds wrong, wouldn't it potentially break other people's code if they relied on it only being sent once? Any thoughts @Kniyl ?

I refactored a bunch of code in this area for Django 1.9, and can confirm that referring to "self" works fine in 1.9 under Mezzanine's 1.9_compat branch (see #1481).
I don't mind taking a look at this after that's merged if somebody needs it for 1.8, but probably not before then.

I think having it fixed for 1.9 is good enough.

This feature would be extremely useful for me, and I'm not able to upgrade to 1.9 in the near term. @stephenmcd, would you reconsider merging this?

@jeffrand your best bet would be to apply the change  to a fork of your own, and deploy against that.


Fixes #6731 Added real-time update to the 'uploaded files' UI in the settings

This creates an event when a new attachment is added and the event is
handled on the client side by updating page_params.attachments.

Thanks for working on this @AasthaGupta!  I posted a comment on how to get this PR working more fully.

@timabbott  I have made the changes you asked for. Please let me know if anything needs to be changed.

@timabbott
I have added code to render list again.
Can you explain what do you mean by remove case because delete attachment is already being handled.

@AasthaGupta do the test I described with 2 browers open at the same time, and you'll see the delete attachment handling is only correct for the browser it's in.  As the new feature tutorial explains, there's a better way to do this that doesn't have that flaw.
@hackerkid can help if you need further support fixing this.

@timabbott  @hackerkid
Should I send event for removing attachments also and re-render attachments list just like I have done for adding attachments?

@AasthaGupta yep, exactly.

@AasthaGupta it looks like you posted a new version; is it ready for review?  (GitHub doesn't send a notification in these cases, so you need to post a comment letting folks know you've addressed the comments from the last review for someone to know)

@timabbott Please review it. I have made the changes according to the comment.

@hackerkid can you take over review for this?

@timabbott Sure

@AasthaGupta It looks the mentioned in column is not getting updated. Can you debug that?

@hackerkid I have updated the pull request. This error of mentioned in  column not updating is still persisting even after using the same function that is used to get the initial attachments list. The 'mentioned in' column of the latest attachment is returned empty from the database. Is there any other call that updates this because otherwise it should be there?

@hackerkid I have updated the pull request.
Also now the mentioned in column gets updated when the user sends message.

@timabbott This is ready for your review.

So, this works functionally, but I think it's going to cause major backend performance problems for a user who has 1000+ uploaded files.  We need to do this via a set of incremental events, not a mega-event that just sends the entire data set every time anything happens.

@hackerkid
Should I change it to the way it was being done before, because before it was done as incremental events?

@AasthaGupta Sorry for that. Can you please change it to the way it was before?

@hackerkid
I have updated the pull request.

@hackerkid I have updated the pull request. Please review it.

@AasthaGupta Thanks for working on this. Deleting attachment is throwing an error. Can you fix that? Also if I delete an attachment and open uploaded files it should not list the deleted file there.

@hackerkid It was not showing deleted files in uploaded files section, after you have deleted it was because it did get deleted in the backend but it showed failed as the event was not sent properly. I have fixed the error. Please review the changes.

@AasthaGupta This looks nice. I amended this commit a bit to remove add_attachment_to_list(The function was to update the uploaded files UI if it is opened in any other tab while a person uploads a file).  https://github.com/hackerkid/zulip/commit/31dd788c53a26d799ed728e49ea4541bbddfe069 @timabbott This is ready fora review from your side.

@AasthaGupta this doesn't work.  The correct way to test this feature is to open two browsers as the same user.  In the first, you look at the attachments UI.  In the second, you upload a file (or delete the upload).  The first tab's state should update without reloading.
I think you've done the state half -- the page_params data structures are being updated in the first tab.  But not the UI.
For deletion, what you want to do is replace the code in the success() handler for channel.del (which updates the UI with row.remove()) with doing the corresponding thing inside the server_events_dispatch block for a removed attachment (e.g. finding the row by ID and updating it, or probably better, calling the function we need to call to update the list_render state after updating that data structure).  Let's get that working correctly and then we can move onto doing the same with the upload code path.  Does that make sense?
@hackerkid make sure you test using that procedure when you next review this :)

@timabbott Apologies for the inconvenience.
@hackerkid I have updated the PR. I update the list_render state after updating the page_params.attachments. Please review it and let me know if any changes are required.
Also I think rather than re-rendering the whole list, a better approach would be to just add/remove a table row. Should I do it that way instead?

@AasthaGupta I also like the idea of re-rendering the appropriate row instead of re-rendering the entire list.

I think the list_render system should be able to take care of re-rendering just the row when we add/remove an element using its API.

@timabbott
list_render doesn't have the functionality to re-render just the row. It has the option to change the data associated with the list but after that you have to re-render the list.
I think one of the alternative can be to associate the 'attachment-id' with each row and then use jquery to delete that row or append the row.

@hackerkid I have updated the pull request. Please review it and let me know if any other changes are required.
cc: @timabbott

For implementing this feature, I merged a series of commits ending with 43fa0aacbfa4ac6e5caf4bc4eaf85e0f015555d5.  This implementation had a number of bugs, but I did find it quite useful in doing this correctly.  Thanks for your efforts on this @AasthaGupta!
I think you could learn something from reading the commit series I merged.


Crail Kubernetes Deployment on POD Network with revised docker configuration to acomodate Node Discovery

Crail Kubernetes yaml and Docker build for Automatic Node Discovery

Please stop copying files randomly for other repositories.
It seems to me that you don't know what you are doing. Why do you have a hard-coded IP <IP_ADDRESS>? Didyou really try it out? What is file crail-apache/conf/list being used for? Why do you have an endless while loop?
We cannot merge a PR with just randomly copied files.

Hi Adrian,
That code is commented .As i have rewritten the kubernetes to work with
dynamically assigned id from hence i am not using any hardcoded id.
I am using crail-master-service of kubernetes to get
the ips .Pocket does not any kubernetes artifact related to service.I have
created the kubernetes service so that master and slave endpoints are
collected dynamically.
Please check the kubernetes artifact of mine you will notice the difference
in approach.
Thanks,
Kaustav
On Mon, 10 Jun 2019 at 3:41 PM, Adrian Schuepbach<EMAIL_ADDRESS>wrote:

Please stop copying files randomly for other repositories.
It seems to me that you don't know what you are doing. Why do you have a
hard-coded IP <IP_ADDRESS>? Didyou really try it out? What is file
crail-apache/conf/list being used for? Why do you have an endless while
loop?
We cannot merge a PR with just randomly copied files.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-crail/pull/76?email_source=notifications&email_token=AE7YQAHBDCWN2OKILNGZYC3PZYSGZA5CNFSM4HWPT3PKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXJPMEI#issuecomment-500364817,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE7YQAHHOZYJJPMFPSEIKTLPZYSGZANCNFSM4HWPT3PA
.

--
Thanks & Regards,
Kaustav
Technical Envagelist
Embedded & Distributed Systems

Hi Adrian,
I will remove this python file as it is not getting used by my modules .
In fact the new the pull request i created i have removed this file if not
i will remove it asap .
I have created the two artifacts under deploy folder and changed the
crail-site.conf and the slaves where i pointed to my services
The python file you mentioned is not getting used by any of my code.
I can remove it in next update in case i forgot.
Thanks,
Kaustav
Thanks ,
Kaustav
On Mon, 10 Jun 2019 at 3:57 PM, Adrian Schuepbach<EMAIL_ADDRESS>wrote:

@asqasq requested changes on this pull request.
In kubernetes/dockerfiles/crail-apache/datanode.py
https://github.com/apache/incubator-crail/pull/76#discussion_r291958143:

@@ -0,0 +1,84 @@
+##############  Pocket resource utilization daemon  ####################

+#from jsonsocket import Client
+import time
+import psutil
+import ifcfg
+import threading
+import pickle
+import sys
+import socket
+import struct
+
+CONTROLLER_IP = "<IP_ADDRESS>"
This seems to be a hard-coded IP to me. Why?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-crail/pull/76?email_source=notifications&email_token=AE7YQAGSLAGOOECEMIVJK3TPZYT7PA5CNFSM4HWPT3PKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB3AUGWA#pullrequestreview-247546712,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE7YQAHQZ4N6PGM6NWVTRHTPZYT7PANCNFSM4HWPT3PA
.

--
Thanks & Regards,
Kaustav
Technical Envagelist
Embedded & Distributed Systems

Hi Adrian,
I will publish the manual shortly once you have a look at my readme you
will understand the flow .Then you will probably understand that my
approach neither has any hard coding nor it is restricted to only aws .
Thanks ,
Kaustav
On Mon, 10 Jun 2019 at 4:07 PM, Kaustav Mukherjee<EMAIL_ADDRESS>wrote:

Hi Adrian,
I will remove this python file as it is not getting used by my modules .
In fact the new the pull request i created i have removed this file if not
i will remove it asap .
I have created the two artifacts under deploy folder and changed the
crail-site.conf and the slaves where i pointed to my services
The python file you mentioned is not getting used by any of my code.
I can remove it in next update in case i forgot.
Thanks,
Kaustav
Thanks ,
Kaustav
On Mon, 10 Jun 2019 at 3:57 PM, Adrian Schuepbach <
<EMAIL_ADDRESS>wrote:

@asqasq requested changes on this pull request.
In kubernetes/dockerfiles/crail-apache/datanode.py
https://github.com/apache/incubator-crail/pull/76#discussion_r291958143
:

@@ -0,0 +1,84 @@
+##############  Pocket resource utilization daemon  ####################

+#from jsonsocket import Client
+import time
+import psutil
+import ifcfg
+import threading
+import pickle
+import sys
+import socket
+import struct
+
+CONTROLLER_IP = "<IP_ADDRESS>"
This seems to be a hard-coded IP to me. Why?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-crail/pull/76?email_source=notifications&email_token=AE7YQAGSLAGOOECEMIVJK3TPZYT7PA5CNFSM4HWPT3PKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB3AUGWA#pullrequestreview-247546712,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE7YQAHQZ4N6PGM6NWVTRHTPZYT7PANCNFSM4HWPT3PA
.

--
Thanks & Regards,
Kaustav
Technical Envagelist
Embedded & Distributed Systems

--
Thanks & Regards,
Kaustav
Technical Envagelist
Embedded & Distributed Systems

Hi Kaustav
Actually I do understand the flow.
This PR cannot be merged for several reasons:

In my opinion it is not the right way to just copy files from other people and claim that
they are youres (even more, you copied our files from our repositories)
The Pocket files do not work with the latest Crail version. Pocket was a research project
based on a older and especially a modified version of Crail.
It seems to me that you don't understand what you are doing
I doubt that you tested it, especially after your last change. Deleting few files
could potentially mean that something does not work anymore (I doubt that it worked before).
We cannot reference docker images from a non-trusted source in the yaml files.

Therefore, we cannot merge this PR.

Hi Adrian,
Thanks for your feedback .
I will close this PR.
Thanks
Kaustav
On Tue, 11 Jun 2019 at 1:16 PM, Adrian Schuepbach<EMAIL_ADDRESS>wrote:

Hi Kaustav
Actually I do understand the flow.
This PR cannot be merged for several reasons:

In my opinion it is not the right way to just copy files from other
people and claim that
they are youres (even more, you copied our files from our repositories)
The Pocket files do not work with the latest Crail version. Pocket
was a research project
based on a older and especially a modified version of Crail.
It seems to me that you don't understand what you are doing
I doubt that you tested it, especially after your last change.
Deleting few files
could potentially mean that something does not work anymore (I doubt
that it worked before).
We cannot reference docker images from a non-trusted source in the
yaml files.

Therefore, we cannot merge this PR.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-crail/pull/76?email_source=notifications&email_token=AE7YQABGGZHOKE5HDIUZ7ILPZ5J5VA5CNFSM4HWPT3PKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXMH44A#issuecomment-500727408,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE7YQAG3BOOBPOCJTF4H65DPZ5J5VANCNFSM4HWPT3PA
.

--
Thanks & Regards,
Kaustav
Technical Envagelist
Embedded & Distributed Systems

Closing the PR

..


Set jaxb-xjc back to scope test

Was switched in commit 5544a990e1272bf72fbbb7bb5104409d020328e3
https://github.com/apache/cxf/commit/5544a990e1272bf72fbbb7bb5104409d020328e3#diff-357e4854869b2e21c38b1b437f11095a
This was likely by mistake.  The result is the maven-war-plugin and maven-assembly-plugin is pulling jaxb-xjc and all dependencies such as ant into war files, etc.

@ffang Can you take a look?

With this change, we also need add  jaxb-xjc dependency in test scope for java11, otherwise java11 build would be broken, I did it on master, so this change in core/pom.xml can be merged.
Thanks!

Why is jaxb-xjc a dependency at all?   All the tests in core pass without it.

Why is jaxb-xjc a dependency at all? All the tests in core pass without it.

Hi @dkulp,
Yes, you are correct, actually we don't need it, removed and we can close this PR now.
Thanks!
Freeman

Hi @dblevins,
Thanks for your contribution, found this issue in the first place.
Please let us know do you have further concern about it.
Freeman


splitting out images

I don't remember what we discussed, but I took a first rough pass

cleaned up excess deps from saturn (should probably make sure we have enough to run examples-cpu)
created saturn-pytorch/tensorflow/rapids
should validate saturn-rapids can run examples-gpu

It'd be good to cut down on the channels as well. Especially if we are going to encourage people to build off these images in the future. It looks silly to have 6 two of which are our own.

closing for #139


link to Function.v and mention Equations

I think Function is essentially obsolete at this point (though still maintained) and the best replacement for most people is Equations.

@tchajed OK with you if I merge this?

Yup, go ahead


(closes #410) Finalize MDL => Material2 migration

By this change we are completely removing mdl as dependency.

introduce own popover and badge component implemented using angular cdk (material 2 hasn't got its own)
introduce home component (still open for naming suggestions) that separates app component from displaying login and the app itself.
use material 2 theming to style our compnents (2 themes for now). Now, because we don't pull predefined themes from cdn, the 'themeColors' config option was removed and the user chooses the theme with one color picker in the settings. Also, 'csui.user.primaryColor' and 'csui.user.accentColor' user tags are now noop and are replaced by 'csui.user.theme'

need to merge master before reviewing

Please add some space between controls to this dialog


Spanish (ES) version of the tutorial

Hi, I saw the Japanese version of the README and just translated the whole tutorial to Spanish, in case you're interested in adding it as well.
It's right now in my fork, but I can send you a PR if you prefer that way.
Great tutorial by the way, to all people involved. :)

I think it's always better to send a PR.

Sorry I did not reply earlier! I did not know how to handle translations at all at that time, and I wasn't sure what to do.
We still do not have an official process for translations, but the folks working on Japanese translations are active in the #guide-translation channel on Slack. I believe they have some tools and processes to help things go more smoothly. I think talking with folks there is the best way to try to coordinate on getting more translations available.
I am open to having a more official process, but I don't really know enough to plan it out myself.


[Bug report] In Busy.org's wallet, my transaction history loading will not end forever.

Hello
I'm contributing as translator to Busy.org, and it is one of my favorite steem tools.
But I'm in trouble with an issue on Busy's wallet.
My transaction history loading will not end forever!

Of course, I checked other account's wallet, but it works correctly.

It does not related that there are a lot of transactions.
This account has huge number of transactions, but its history is loaded.

I thought that the login state is related, but it was not.
Then I looked for other accounts that have similar problems.
It was found immediately.
My second account has same problem lol
Umm,,, I want to see it on another person's account.
I thought what is something in common between these accounts (except their owners).
In these days, I've been trying various things with those accounts for some translation work.
So I thought that the unusual history is that cause.
Then I search some words with Steemit's search box.

I found!
Some users who have "Cancel transfer from savings" in their recent history have same issue in their wallet.
These are my environment:
PC1:
OS: Windows 7 64bit
Web Browser: Google Chrome (62.0.3202.75) and Firefox (56.0.2)
(My Internet Exploere 11 dose not work in Busy.org. )
PC2:
OS: Windows 10 64bit
Web Browser: Google Chrome (62.0.3202.75), Microsoft Edge 40.15063.674.0 and Firefox (56.0.2)
I hope this issue will be fixed soon.
Thank you for reading!Open Source Contribution posted via Utopian.io

Thanks for reporting this issue, I'll be taking a look into it


Add unit test [Chenlei]

Hi @LeiChenIntel, thanks for the contribution! Could you please follow these instructions to sign the the DCO?

Hi @LeiChenIntel, thanks for the contribution! Could you please follow these instructions to sign the the DCO?

Hi @psfoley, I close this PR because the commits are merged in https://github.com/securefederatedai/openfl/pull/821.


In the form of grep where the 2nd argument is an Array, if one of the items in the Array points to a directory, ShellJS will crash

Node version (or tell us if you're using electron or some other framework):
10.17.0
ShellJS version (the most recent version/Github branch you see the bug on):
0.8.3
Operating system:
Mac OSX
Description of the bug:
When using grep(), if you're using the form where you hand in a file_array and one of the files is actually a directory, ShellJS will crash.
Example ShellJS command to reproduce the error:
This will crash when it reaches directoryname:
sh = require('shelljs');
let my_file_array = ['filename1', 'filename2', 'directoryname'];
sh.grep(/hi there/, my_file_array);

The trace I got was:
[PATH REDACTED]/shelljs/src/common.js:399
        throw e;
        ^

Error [ShellJSInternalError]: EISDIR: illegal operation on a directory, read
    at Object.readSync (fs.js:524:3)
    at tryReadSync (fs.js:349:20)
    at Object.readFileSync (fs.js:386:19)
    at [PATH REDACTED]/shelljs/src/grep.js:55:45
    at Array.forEach (<anonymous>)
    at Object._grep ([PATH REDACTED]/shelljs/src/grep.js:49:9)
    at Object.grep ([PATH REDACTED]/shelljs/src/common.js:384:25)
    at Object.<anonymous> (/home/nfischer/shelljscrash/index.js:4:4)
    at Module._compile (internal/modules/cjs/loader.js:1133:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10) {
  errno: -21,
  syscall: 'read',
  code: 'EISDIR'
}

I think this is WAI. grep does not support directory arguments (see https://github.com/shelljs/shelljs#grepoptions-regex_filter-file--file-). We don't currently support the -R flag, although I filed #998 to track this.


Cant Update the Server

Is there an existing issue for this?

[X] I have searched the existing issues

Used image version
proton_latest
Used docker compose file or docker command
version: "3"
services:
enshrouded:
image: mornedhels/enshrouded-server:stable-proton
container_name: enshrouded
hostname: enshrouded
restart: unless-stopped
stop_grace_period: 90s
ports:
- "15636-15637:15636-15637/udp"
volumes:
- ./game:/opt/enshrouded
environment:
- SERVER_NAME=****
- SERVER_PASSWORD=**
- UPDATE_CRON=*/30 * * * *
- PUID=1000
- PGID=1000
#Tried different PUID / PGID
What you expected to see, versus what you actually saw
Expected: Updated Server
What happens:
Logs from the container
docker logs enshrouded
2024-02-06 21:05:17.670 supervisord: enshrouded-bootstrap enshrouded-updater: started
2024-02-06 21:05:17.685 supervisord: enshrouded-updater INFO - Running enshrouded-updater
2024-02-06 21:05:17,743 INFO exited: enshrouded-bootstrap (exit status 0; expected)
2024-02-06 21:05:17.742 supervisord: enshrouded-bootstrap INFO - Bootstrap complete
2024-02-06 21:05:18.178 supervisord: enshrouded-updater INFO - Enshrouded server needs to be updated
2024-02-06 21:05:18.178 supervisord: enshrouded-updater WARN - STEAM_API_KEY and STEAM_API_PUBLIC_IP must be set to check if server is empty
2024-02-06 21:05:20.114 supervisord: enshrouded-updater tid(85) burning pthread_key_t == 0 so we never use it
2024-02-06 21:05:20.114 supervisord: enshrouded-updater Redirecting stderr to '/home/enshrouded/Steam/logs/stderr.txt'
2024-02-06 21:05:20.114 supervisord: enshrouded-updater Logging directory: '/home/enshrouded/Steam/logs'
2024-02-06 21:05:20.114 supervisord: enshrouded-updater [  0%] Checking for available updates...
2024-02-06 21:05:20.114 supervisord: enshrouded-updater [----] Verifying installation...
2024-02-06 21:05:20.114 supervisord: enshrouded-updater Steam Console Client (c) Valve Corporation - version<PHONE_NUMBER>
2024-02-06 21:05:20.114 supervisord: enshrouded-updater -- type 'quit' to exit --
2024-02-06 21:05:20.114 supervisord: enshrouded-updater Loading Steam API...
2024-02-06 21:05:20.448 supervisord: enshrouded-updater dlmopen steamservice.so failed: steamservice.so: cannot open shared object file: No such file or directory
2024-02-06 21:05:20.448 supervisord: enshrouded-updater OK
2024-02-06 21:05:20.448 supervisord: enshrouded-updater "@sSteamCmdForcePlatformType" = "windows"
2024-02-06 21:05:20.448 supervisord: enshrouded-updater #033[0m
2024-02-06 21:05:20.449 supervisord: enshrouded-updater
2024-02-06 21:05:20.449 supervisord: enshrouded-updater Connecting anonymously to Steam Public...
2024-02-06 21:05:22.621 supervisord: enshrouded-updater OK
2024-02-06 21:05:22.621 supervisord: enshrouded-updater Waiting for client config...OK
2024-02-06 21:05:22.621 supervisord: enshrouded-updater Waiting for user info...
2024-02-06 21:05:24.069 supervisord: enshrouded-updater OK
2024-02-06 21:05:24.069 supervisord: enshrouded-updater Success! App '1007' already up to date.
2024-02-06 21:05:24.069 supervisord: enshrouded-updater  Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
2024-02-06 21:05:24.069 supervisord: enshrouded-updater Error! App '2278520' state is 0x226 after update job.
2024-02-06 21:05:24.228 supervisord: enshrouded-updater WARN - Download of Enshrouded server failed - aborting update
2024-02-06 21:05:24,589 INFO spawned: 'enshrouded-server' with pid 104
2024-02-06 21:05:24.610 supervisord: enshrouded-server INFO - Running enshrouded-server
2024-02-06 21:05:24.820 supervisord: enshrouded-server esync: up and running.
2024-02-06 21:05:24.821 supervisord: enshrouded-server wine: RLIMIT_NICE is <= 20, unable to use setpriority safely
2024-02-06 21:05:25.978 supervisord: enshrouded-server 0070:err:wineusb:DriverEntry Failed to initialize Unix library, status 0xc0000135.
2024-02-06 21:05:25.978 supervisord: enshrouded-server 0070:err:ntoskrnl:ZwLoadDriver failed to create driver L"\Registry\Machine\System\CurrentControlSet\Services\wineusb": c0000135
2024-02-06 21:05:26,981 INFO reaped unknown pid 128 (exit status 0)
2024-02-06 21:05:27,983 INFO reaped unknown pid 113 (exit status 0)
2024-02-06 21:05:29.009 supervisord: enshrouded-server 00f0:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
2024-02-06 21:05:29.009 supervisord: enshrouded-server 00f0:err:winediag:nodrv_CreateWindow L"The explorer process failed to start."
2024-02-06 21:05:29.011 supervisord: enshrouded-server 00f0:err:systray:initialize_systray Could not create tray window
2024-02-06 21:05:29.799 supervisord: enshrouded-server 0128:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
2024-02-06 21:05:29.799 supervisord: enshrouded-server 0128:err:winediag:nodrv_CreateWindow L"Make sure that your X server is running and that $DISPLAY is set correctly."
2024-02-06 21:05:29.802 supervisord: enshrouded-server 0128:err:tabtip:wWinMain Failed to create hwnd!
2024-02-06 21:05:29,839 INFO reaped unknown pid 176 (exit status 255)
2024-02-06 21:05:29.838 supervisord: enshrouded-server 00e8:err:kerberos:kerberos_LsaApInitializePackage no Kerberos support, expect problems
2024-02-06 21:05:31.379 supervisord: enshrouded-server [holistic] Backup log to '/opt/enshrouded/server/logs/backup/enshrouded_server_2024-02-06T21-05-31.log'.
2024-02-06 21:05:31.381 supervisord: enshrouded-server [enshrouded] Create logfile
2024-02-06 21:05:31.383 supervisord: enshrouded-server enshrouded_server(detached HEAD) - version 0cf549fdadc2e9439f5e34a22917b7236310ec8e (master)
2024-02-06 21:05:31.383 supervisord: enshrouded-server CPUs:
2024-02-06 21:05:31.383 supervisord: enshrouded-server  Idx   Id  Group  LogIdx  CoreIdx  LLCIdx  NumaIdx  EffClass
2024-02-06 21:05:31.383 supervisord: enshrouded-server   0   256    0       0        0       3       0        0
2024-02-06 21:05:31.383 supervisord: enshrouded-server   1   257    0       1        1       3       0        0
2024-02-06 21:05:31.384 supervisord: enshrouded-server   2   258    0       2        2       3       0        0
2024-02-06 21:05:31.384 supervisord: enshrouded-server   3   259    0       3        3       3       0        0
2024-02-06 21:05:31.384 supervisord: enshrouded-server CPU Types:
2024-02-06 21:05:31.384 supervisord: enshrouded-server   Performance:   4<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 21:05:31.385 supervisord: enshrouded-server   Efficient:     0<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
2024-02-06 21:05:31.385 supervisord: enshrouded-server [task] TaskSystemSetup:
2024-02-06 21:05:31.385 supervisord: enshrouded-server [task]   mainWorkerCount:     2
2024-02-06 21:05:31.385 supervisord: enshrouded-server [task]   nextMainWorkerIndex: 1
2024-02-06 21:05:31.385 supervisord: enshrouded-server [task]   Group 'default'  worker:5  main:2
2024-02-06 21:05:31.385 supervisord: enshrouded-server [task]      0:  Task-default00<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
2024-02-06 21:05:31.386 supervisord: enshrouded-server [task]      1:  Task-default01<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
2024-02-06 21:05:31.386 supervisord: enshrouded-server [task]      2:  Task-default02<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 21:05:31.386 supervisord: enshrouded-server [task]      3:  Task-default03<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 21:05:31.386 supervisord: enshrouded-server [task]      4:  Task-default04<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 21:05:31.386 supervisord: enshrouded-server [task]   Group 'background'  worker:3  main:0
2024-02-06 21:05:31.387 supervisord: enshrouded-server [task]      0:  Task-background00<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 21:05:31.387 supervisord: enshrouded-server [task]      1:  Task-background01<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 21:05:31.387 supervisord: enshrouded-server [task]      2:  Task-background02<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 21:05:31.387 supervisord: enshrouded-server [app] start creation step Init
2024-02-06 21:05:31.387 supervisord: enshrouded-server [app] start creation step Init_Steamworks
2024-02-06 21:05:31.843 supervisord: enshrouded-server CAppInfoCacheReadFromDiskThread took 20 milliseconds to initialize
2024-02-06 21:05:31.907 supervisord: enshrouded-server dlmopen steamservice.so failed: steamservice.so: cannot open shared object file: No such file or directory
2024-02-06 21:05:31.914 supervisord: enshrouded-server Setting breakpad minidump AppID = 1203620#015
2024-02-06 21:05:31.914 supervisord: enshrouded-server [app] start creation step Init_FileSystem
2024-02-06 21:05:31.915 supervisord: enshrouded-server SteamInternal_SetMinidumpSteamID:  Caching Steam ID:<PHONE_NUMBER>0265728 [API loaded no]#015
2024-02-06 21:05:31.930 supervisord: enshrouded-server [app] start creation step Init_MountPackage
2024-02-06 21:05:31.946 supervisord: enshrouded-server +--"":
2024-02-06 21:05:31.946 supervisord: enshrouded-server     |
2024-02-06 21:05:31.946 supervisord: enshrouded-server     +--"":
2024-02-06 21:05:31.946 supervisord: enshrouded-server         |
2024-02-06 21:05:31.946 supervisord: enshrouded-server         +--"gamebuild":  (#0) Archive -> '/gamebuild_path/enshrouded_server'
2024-02-06 21:05:31.946 supervisord: enshrouded-server         |
2024-02-06 21:05:31.946 supervisord: enshrouded-server         +--"gamebuild_path":  (#1) Native -> 'Z:/opt/enshrouded/server/'
2024-02-06 21:05:31.947 supervisord: enshrouded-server [app] start creation step Init_Version
2024-02-06 21:05:31.949 supervisord: enshrouded-server Game Version (SVN): 490142
2024-02-06 21:05:31.962 supervisord: enshrouded-server [app] start creation step Init_UserSystem
2024-02-06 21:05:31.962 supervisord: enshrouded-server [user] Couldn't retrieve steam friends interface.
2024-02-06 21:05:31.962 supervisord: enshrouded-server [user] Couldn't retrieve steam user interface.
2024-02-06 21:05:31.963 supervisord: enshrouded-server [app] start creation step Init_OnlineSystem
2024-02-06 21:05:31.972 supervisord: enshrouded-server [OnlineProviderSteam] started transition from 'Uninitialized' to 'Initialized' (current='')!
2024-02-06 21:05:31.973 supervisord: enshrouded-server [app] start creation step Init_Session
2024-02-06 21:05:31.998 supervisord: enshrouded-server [app] start creation step Init_Tracking
2024-02-06 21:05:32.038 supervisord: enshrouded-server [app] start creation step Init_ResourceSystem
2024-02-06 21:05:32.043 supervisord: enshrouded-server [resource] Could not load resource cache file 'Z:/opt/enshrouded/server/enshrouded_server.krc'. error=file not found
2024-02-06 21:05:32.049 supervisord: enshrouded-server [app] start creation step Init_ResourcePreload_Start
2024-02-06 21:05:32.913 supervisord: enshrouded-server [app] start creation step Init_ResourcePreload_Wait
2024-02-06 21:05:34,918 INFO success: enshrouded-server entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2024-02-06 21:05:34.927 supervisord: enshrouded-updater enshrouded-server: started
2024-02-06 21:05:35.029 supervisord: enshrouded-updater INFO - Releasing PID file /var/run/enshrouded/enshrouded-updater.pid
2024-02-06 21:05:35,038 INFO exited: enshrouded-updater (exit status 0; expected)
2024-02-06 21:05:35.037 supervisord: enshrouded-updater INFO - enshrouded-updater complete
2024-02-06 21:05:40.027 supervisord: enshrouded-server [app] start creation step Init_SaveDataSystem
2024-02-06 21:05:40.028 supervisord: enshrouded-server [savedata] Couldn't retrieve steam remote storage interface.
2024-02-06 21:05:40.029 supervisord: enshrouded-server [savedata] Steam Cloud is disabled.
2024-02-06 21:05:40.032 supervisord: enshrouded-server [app] start creation step Load_ResourceObject
2024-02-06 21:05:40.033 supervisord: enshrouded-server [app] start creation step Init_ServerThread
2024-02-06 21:05:40.035 supervisord: enshrouded-server [app] start creation step StartGame
2024-02-06 21:05:40.036 supervisord: enshrouded-server [app] start creation step Init
2024-02-06 21:05:40.036 supervisord: enshrouded-server [app] start creation step StartGame
2024-02-06 21:05:40.041 supervisord: enshrouded-server [Session] started transition from 'Lobby' to 'Host_Online' (current='')!
2024-02-06 21:05:40.044 supervisord: enshrouded-server [Session] 'PrepareGame' (up)!
2024-02-06 21:05:40.050 supervisord: enshrouded-server [savedata] Start 'Open Container' on container 3ad85aea
2024-02-06 21:05:40.057 supervisord: enshrouded-server [savedata] Updating 'Open Container' on container 3ad85aea
2024-02-06 21:05:40.058 supervisord: enshrouded-server [savedata] Finished 'Open Container' on container 3ad85aea
2024-02-06 21:05:40.815 supervisord: enshrouded-server [online] Server connected to Steam successfully
2024-02-06 21:05:41.053 supervisord: enshrouded-server [online] Server is not VAC Secure!
2024-02-06 21:05:41.054 supervisord: enshrouded-server [online] Server SteamId:<PHONE_NUMBER>8441224
2024-02-06 21:05:41.055 supervisord: enshrouded-server [online] Public ipv4: ...
2024-02-06 21:05:41.056 supervisord: enshrouded-server [OnlineProviderSteam] 'Initialize' (up)!
2024-02-06 21:05:41.057 supervisord: enshrouded-server [OnlineProviderSteam] 'JoinOrCreateGame' (up)!
2024-02-06 21:05:41.058 supervisord: enshrouded-server [OnlineProviderSteam] 'SetLobbyData' (up)!
2024-02-06 21:05:41.058 supervisord: enshrouded-server [OnlineProviderSteam] 'JoinLocalPlayers' (up)!
2024-02-06 21:05:41.058 supervisord: enshrouded-server [OnlineProviderSteam] 'ConnectToPeers' (up)!
2024-02-06 21:05:41.059 supervisord: enshrouded-server [OnlineProviderSteam] finished transition from 'Uninitialized' to 'InGame' (current='InGame')!
2024-02-06 21:05:41.060 supervisord: enshrouded-server [Session] 'HostOnline' (up)!
2024-02-06 21:05:41.060 supervisord: enshrouded-server [Session] finished transition from 'Lobby' to 'Host_Online' (current='Host_Online')!
2024-02-06 21:05:41.807 supervisord: enshrouded-server [ecss] TaskQueue:  workerCount:5   taskWorkerCount:4<PHONE_NUMBER>000000000000000000000000000000000000000000000000011110
2024-02-06 21:05:42.083 supervisord: enshrouded-server [TerraformingEfficiencyRegistry] Finished loading.
2024-02-06 21:05:42.164 supervisord: enshrouded-server [ecs] Nobuild zones changed
2024-02-06 21:05:42.165 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 21:05:42.165 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 21:05:42.176 supervisord: enshrouded-server [savexxx] LOAD 1 bases 287 entities
2024-02-06 21:05:42.177 supervisord: enshrouded-server [ecs] readEntitySerializationContext: Templates: 39  Components: 123  Size: 46,112
2024-02-06 21:05:42.194 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry e284b901-c3dd-4282-8a4e-0a618e5763f5.
2024-02-06 21:05:42.195 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 27a9e067-4b9f-45cf-b966-208a5ecbd53b.
2024-02-06 21:05:42.196 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 85e3baea-7c93-4d6f-b1be-42f79b036761.
2024-02-06 21:05:42.196 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 28c4b2ca-4a01-4ba9-9e05-49c871df6904.
2024-02-06 21:05:42.197 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry ec28c5d5-9d23-413e-b238-781157fb2f65.
2024-02-06 21:05:42.197 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 5d9bd376-5b7f-4db4-8f2e-6a3e564194b1.
2024-02-06 21:05:42.198 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 34aa0c3c-9a88-4f8b-a070-64827a5fdabf.
2024-02-06 21:05:42.319 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 21:05:42.319 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 21:05:45.592 supervisord: enshrouded-server [server] Load deserialization took 3.42 s
2024-02-06 21:05:45.620 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 21:05:45.644 supervisord: message repeated 3 times: [ enshrouded-server [ecs] Build zones changed]
2024-02-06 21:06:42.177 supervisord: enshrouded-server [ecss] Stats:  Upd:3,568  (16.8ms)  Time:60,005ms  Total:9,776ms  Max:20ms  Avg:2.7ms  +:0  -:0  Ent:894  Pred:0
2024-02-06 21:07:42.194 supervisord: enshrouded-server [ecss] Stats:  Upd:3,599  (16.7ms)  Time:60,015ms  Total:10,793ms  Max:23ms  Avg:3.0ms  +:0  -:0  Ent:894  Pred:0
2024-02-06 21:08:42.196 supervisord: enshrouded-server [ecss] Stats:  Upd:3,600  (16.7ms)  Time:60,000ms  Total:10,881ms  Max:20ms  Avg:3.0ms  +:0  -:0  Ent:894  Pred:0
2024-02-06 21:09:42.197 supervisord: enshrouded-server [ecss] Stats:  Upd:3,599  (16.7ms)  Time:60,001ms  Total:10,236ms  Max:28ms  Avg:2.8ms  +:0  -:0  Ent:894  Pred:0
Exec -it bash into container then ./enshrouded-updater
root@enshrouded:/usr/local/etc/enshrouded# ./enshrouded-updater
INFO - Running enshrouded-updater
INFO - Enshrouded server needs to be updated
WARN - STEAM_API_KEY and STEAM_API_PUBLIC_IP must be set to check if server is empty
tid(86) burning pthread_key_t == 0 so we never use it
Redirecting stderr to '/root/Steam/logs/stderr.txt'
Logging directory: '/root/Steam/logs'
[  0%] Checking for available updates...
[----] Verifying installation...
Steam Console Client (c) Valve Corporation - version<PHONE_NUMBER>
-- type 'quit' to exit --
Loading Steam API...OK
"@sSteamCmdForcePlatformType" = "windows"
Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
Success! App '1007' already up to date.
Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
Error! App '2278520' state is 0x226 after update job.
WARN - Download of Enshrouded server failed - aborting update
enshrouded-server: ERROR (spawn error)
INFO - Releasing PID file /var/run/enshrouded/enshrouded-updater.pid
INFO - enshrouded-updater complete

Hey, you probably run into a permission issue with your volume. Try chown your directory (recursive) and give the 1000:1000 user back the files. After that restart the container. The update should work afterwards.
Do not run the update script inside the container as root. You should run it via command described in the README.

Do you mean the 'game' directory or another one? I'm not an Docker Pro and are unsure where all the files are installed from Docker Container.

yes, update the game directory with the PUID and PGID and start the docker container

paddy@paddy-vps:~/enshrouded-mornedhels$ id -u
1000
paddy@paddy-vps:~/enshrouded-mornedhels$ id -g
1000
paddy@paddy-vps:~/enshrouded-mornedhels$ ls -la
total 16
drwxrwxr-x 3 paddy paddy 4096 Feb  6 22:20 .
drwxr-x--- 7 paddy paddy 4096 Feb  6 22:26 ..
-rw-r--r-- 1 root  root   446 Feb  6 22:20 compose.yml
drwxr-xr-x 3 paddy paddy 4096 Feb  6 19:43 game
This should fit the requirements now but if i use the command from the Readme you mentioned i still get the following Log

Docker Log
2024-02-06 22:30:04.105 supervisord: enshrouded-updater enshrouded-server: stopped
2024-02-06 22:30:05.721 supervisord: enshrouded-updater tid(588) burning pthread_key_t == 0 so we never use it
2024-02-06 22:30:05.721 supervisord: enshrouded-updater Redirecting stderr to '/home/enshrouded/Steam/logs/stderr.txt'
2024-02-06 22:30:05.721 supervisord: enshrouded-updater Logging directory: '/home/enshrouded/Steam/logs'
2024-02-06 22:30:05.721 supervisord: enshrouded-updater [  0%] Checking for available updates...
2024-02-06 22:30:05.721 supervisord: enshrouded-updater [----] Verifying installation...
2024-02-06 22:30:05.721 supervisord: enshrouded-updater Steam Console Client (c) Valve Corporation - version<PHONE_NUMBER>
2024-02-06 22:30:05.721 supervisord: enshrouded-updater -- type 'quit' to exit --
2024-02-06 22:30:05.721 supervisord: enshrouded-updater Loading Steam API...
2024-02-06 22:30:05.934 supervisord: enshrouded-updater dlmopen steamservice.so failed: steamservice.so: cannot open shared object file: No such file or directory
2024-02-06 22:30:05.934 supervisord: enshrouded-updater OK
2024-02-06 22:30:05.934 supervisord: enshrouded-updater "@sSteamCmdForcePlatformType" = "windows"
2024-02-06 22:30:05.934 supervisord: enshrouded-updater #033[0m
2024-02-06 22:30:05.936 supervisord: enshrouded-updater
2024-02-06 22:30:05.936 supervisord: enshrouded-updater Connecting anonymously to Steam Public...
2024-02-06 22:30:08.278 supervisord: enshrouded-updater OK
2024-02-06 22:30:08.278 supervisord: enshrouded-updater Waiting for client config...OK
2024-02-06 22:30:08.278 supervisord: enshrouded-updater Waiting for user info...
2024-02-06 22:30:09.615 supervisord: enshrouded-updater OK
2024-02-06 22:30:09.615 supervisord: enshrouded-updater Success! App '1007' already up to date.
2024-02-06 22:30:09.615 supervisord: enshrouded-updater  Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
2024-02-06 22:30:09.615 supervisord: enshrouded-updater Error! App '2278520' state is 0x226 after update job.
2024-02-06 22:30:09.767 supervisord: enshrouded-updater WARN - Download of Enshrouded server failed - aborting update
2024-02-06 22:30:10,140 INFO spawned: 'enshrouded-server' with pid 607
2024-02-06 22:30:10.166 supervisord: enshrouded-server INFO - Running enshrouded-server
2024-02-06 22:30:10.328 supervisord: enshrouded-server esync: up and running.
2024-02-06 22:30:10.329 supervisord: enshrouded-server wine: RLIMIT_NICE is <= 20, unable to use setpriority safely
2024-02-06 22:30:11.320 supervisord: enshrouded-server 0070:err:wineusb:DriverEntry Failed to initialize Unix library, status 0xc0000135.
2024-02-06 22:30:11.320 supervisord: enshrouded-server 0070:err:ntoskrnl:ZwLoadDriver failed to create driver L"\Registry\Machine\System\CurrentControlSet\Services\wineusb": c0000135
2024-02-06 22:30:12,323 INFO reaped unknown pid 631 (exit status 0)
2024-02-06 22:30:13,326 INFO reaped unknown pid 616 (exit status 0)
2024-02-06 22:30:13.633 supervisord: enshrouded-server 00f4:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
2024-02-06 22:30:13.633 supervisord: enshrouded-server 00f4:err:winediag:nodrv_CreateWindow L"The explorer process failed to start."
2024-02-06 22:30:13.634 supervisord: enshrouded-server 00f4:err:systray:initialize_systray Could not create tray window
2024-02-06 22:30:14.315 supervisord: enshrouded-server 012c:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
2024-02-06 22:30:14.315 supervisord: enshrouded-server 012c:err:winediag:nodrv_CreateWindow L"Make sure that your X server is running and that $DISPLAY is set correctly."
2024-02-06 22:30:14.317 supervisord: enshrouded-server 012c:err:tabtip:wWinMain Failed to create hwnd!
2024-02-06 22:30:14,346 INFO reaped unknown pid 680 (exit status 255)
2024-02-06 22:30:14.346 supervisord: enshrouded-server 00ec:err:kerberos:kerberos_LsaApInitializePackage no Kerberos support, expect problems
2024-02-06 22:30:15.523 supervisord: enshrouded-server [holistic] Backup log to '/opt/enshrouded/server/logs/backup/enshrouded_server_2024-02-06T22-30-15.log'.
2024-02-06 22:30:15.524 supervisord: enshrouded-server [enshrouded] Create logfile
2024-02-06 22:30:15.524 supervisord: enshrouded-server enshrouded_server(detached HEAD) - version 0cf549fdadc2e9439f5e34a22917b7236310ec8e (master)
2024-02-06 22:30:15.525 supervisord: enshrouded-server CPUs:
2024-02-06 22:30:15.525 supervisord: enshrouded-server  Idx   Id  Group  LogIdx  CoreIdx  LLCIdx  NumaIdx  EffClass
2024-02-06 22:30:15.525 supervisord: enshrouded-server   0   256    0       0        0       3       0        0
2024-02-06 22:30:15.525 supervisord: enshrouded-server   1   257    0       1        1       3       0        0
2024-02-06 22:30:15.526 supervisord: enshrouded-server   2   258    0       2        2       3       0        0
2024-02-06 22:30:15.526 supervisord: enshrouded-server   3   259    0       3        3       3       0        0
2024-02-06 22:30:15.526 supervisord: enshrouded-server CPU Types:
2024-02-06 22:30:15.526 supervisord: enshrouded-server   Performance:   4<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 22:30:15.526 supervisord: enshrouded-server   Efficient:     0<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
2024-02-06 22:30:15.527 supervisord: enshrouded-server [task] TaskSystemSetup:
2024-02-06 22:30:15.527 supervisord: enshrouded-server [task]   mainWorkerCount:     2
2024-02-06 22:30:15.528 supervisord: enshrouded-server [task]   nextMainWorkerIndex: 1
2024-02-06 22:30:15.528 supervisord: enshrouded-server [task]   Group 'default'  worker:5  main:2
2024-02-06 22:30:15.528 supervisord: enshrouded-server [task]      0:  Task-default00<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
2024-02-06 22:30:15.528 supervisord: enshrouded-server [task]      1:  Task-default01<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
2024-02-06 22:30:15.529 supervisord: enshrouded-server [task]      2:  Task-default02<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 22:30:15.529 supervisord: enshrouded-server [task]      3:  Task-default03<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 22:30:15.530 supervisord: enshrouded-server [task]      4:  Task-default04<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 22:30:15.530 supervisord: enshrouded-server [task]   Group 'background'  worker:3  main:0
2024-02-06 22:30:15.530 supervisord: enshrouded-server [task]      0:  Task-background00<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 22:30:15.530 supervisord: enshrouded-server [task]      1:  Task-background01<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 22:30:15.530 supervisord: enshrouded-server [task]      2:  Task-background02<PHONE_NUMBER>000000000000000000000000000000000000000000000000001111
2024-02-06 22:30:15.530 supervisord: enshrouded-server [app] start creation step Init
2024-02-06 22:30:15.531 supervisord: enshrouded-server [app] start creation step Init_Steamworks
2024-02-06 22:30:15.898 supervisord: enshrouded-server CAppInfoCacheReadFromDiskThread took 26 milliseconds to initialize
2024-02-06 22:30:15.965 supervisord: enshrouded-server dlmopen steamservice.so failed: steamservice.so: cannot open shared object file: No such file or directory
2024-02-06 22:30:15.975 supervisord: enshrouded-server Setting breakpad minidump AppID = 1203620#015
2024-02-06 22:30:15.976 supervisord: enshrouded-server SteamInternal_SetMinidumpSteamID:  Caching Steam ID:<PHONE_NUMBER>0265728 [API loaded no]#015
2024-02-06 22:30:15.976 supervisord: enshrouded-server [app] start creation step Init_FileSystem
2024-02-06 22:30:15.987 supervisord: enshrouded-server [app] start creation step Init_MountPackage
2024-02-06 22:30:16.007 supervisord: enshrouded-server +--"":
2024-02-06 22:30:16.007 supervisord: enshrouded-server     |
2024-02-06 22:30:16.007 supervisord: enshrouded-server     +--"":
2024-02-06 22:30:16.008 supervisord: enshrouded-server         |
2024-02-06 22:30:16.008 supervisord: enshrouded-server         +--"gamebuild":  (#0) Archive -> '/gamebuild_path/enshrouded_server'
2024-02-06 22:30:16.008 supervisord: enshrouded-server         |
2024-02-06 22:30:16.008 supervisord: enshrouded-server         +--"gamebuild_path":  (#1) Native -> 'Z:/opt/enshrouded/server/'
2024-02-06 22:30:16.008 supervisord: enshrouded-server [app] start creation step Init_Version
2024-02-06 22:30:16.010 supervisord: enshrouded-server Game Version (SVN): 490142
2024-02-06 22:30:16.011 supervisord: enshrouded-server [app] start creation step Init_UserSystem
2024-02-06 22:30:16.011 supervisord: enshrouded-server [user] Couldn't retrieve steam friends interface.
2024-02-06 22:30:16.012 supervisord: enshrouded-server [app] start creation step Init_OnlineSystem
2024-02-06 22:30:16.012 supervisord: enshrouded-server [user] Couldn't retrieve steam user interface.
2024-02-06 22:30:16.021 supervisord: enshrouded-server [OnlineProviderSteam] started transition from 'Uninitialized' to 'Initialized' (current='')!
2024-02-06 22:30:16.023 supervisord: enshrouded-server [app] start creation step Init_Session
2024-02-06 22:30:16.053 supervisord: enshrouded-server [app] start creation step Init_Tracking
2024-02-06 22:30:16.081 supervisord: enshrouded-server [app] start creation step Init_ResourceSystem
2024-02-06 22:30:16.084 supervisord: enshrouded-server [resource] Could not load resource cache file 'Z:/opt/enshrouded/server/enshrouded_server.krc'. error=file not found
2024-02-06 22:30:16.088 supervisord: enshrouded-server [app] start creation step Init_ResourcePreload_Start
2024-02-06 22:30:16.834 supervisord: enshrouded-server [app] start creation step Init_ResourcePreload_Wait
2024-02-06 22:30:20,844 INFO success: enshrouded-server entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2024-02-06 22:30:20.853 supervisord: enshrouded-updater enshrouded-server: started
2024-02-06 22:30:20.951 supervisord: enshrouded-updater INFO - Releasing PID file /var/run/enshrouded/enshrouded-updater.pid
2024-02-06 22:30:20.958 supervisord: enshrouded-updater INFO - enshrouded-updater complete
2024-02-06 22:30:20,961 INFO exited: enshrouded-updater (exit status 0; expected)
2024-02-06 22:30:22.949 supervisord: enshrouded-server [app] start creation step Init_SaveDataSystem
2024-02-06 22:30:22.950 supervisord: enshrouded-server [savedata] Couldn't retrieve steam remote storage interface.
2024-02-06 22:30:22.950 supervisord: enshrouded-server [savedata] Steam Cloud is disabled.
2024-02-06 22:30:22.954 supervisord: enshrouded-server [app] start creation step Load_ResourceObject
2024-02-06 22:30:22.954 supervisord: enshrouded-server [app] start creation step Init_ServerThread
2024-02-06 22:30:22.956 supervisord: enshrouded-server [app] start creation step StartGame
2024-02-06 22:30:22.957 supervisord: enshrouded-server [app] start creation step Init
2024-02-06 22:30:22.957 supervisord: enshrouded-server [app] start creation step StartGame
2024-02-06 22:30:22.962 supervisord: enshrouded-server [Session] started transition from 'Lobby' to 'Host_Online' (current='')!
2024-02-06 22:30:22.964 supervisord: enshrouded-server [Session] 'PrepareGame' (up)!
2024-02-06 22:30:22.970 supervisord: enshrouded-server [savedata] Start 'Open Container' on container 3ad85aea
2024-02-06 22:30:22.971 supervisord: enshrouded-server [savedata]
2024-02-06 22:30:22.972 supervisord: enshrouded-server Updating 'Open Container' on container 3ad85aea
2024-02-06 22:30:22.975 supervisord: enshrouded-server [savedata] Finished 'Open Container' on container 3ad85aea
2024-02-06 22:30:23.471 supervisord: enshrouded-server [online] Server connected to Steam successfully
2024-02-06 22:30:23.720 supervisord: enshrouded-server [online] Server is not VAC Secure!
2024-02-06 22:30:23.723 supervisord: enshrouded-server [online] Server SteamId:<PHONE_NUMBER>8348424
2024-02-06 22:30:23.725 supervisord: enshrouded-server [online] Public ipv4: ...
2024-02-06 22:30:23.725 supervisord: enshrouded-server [OnlineProviderSteam] 'Initialize' (up)!
2024-02-06 22:30:23.728 supervisord: enshrouded-server [OnlineProviderSteam] 'JoinOrCreateGame' (up)!
2024-02-06 22:30:23.729 supervisord: enshrouded-server [OnlineProviderSteam] 'SetLobbyData' (up)!
2024-02-06 22:30:23.729 supervisord: enshrouded-server [OnlineProviderSteam] 'JoinLocalPlayers' (up)!
2024-02-06 22:30:23.729 supervisord: enshrouded-server [OnlineProviderSteam] 'ConnectToPeers' (up)!
2024-02-06 22:30:23.730 supervisord: enshrouded-server [OnlineProviderSteam] finished transition from 'Uninitialized' to 'InGame' (current='InGame')!
2024-02-06 22:30:23.731 supervisord: enshrouded-server [Session] 'HostOnline' (up)!
2024-02-06 22:30:23.733 supervisord: enshrouded-server [Session] finished transition from 'Lobby' to 'Host_Online' (current='Host_Online')!
2024-02-06 22:30:24.106 supervisord: enshrouded-server [ecss]
2024-02-06 22:30:24.107 supervisord: enshrouded-server TaskQueue:  workerCount:5   taskWorkerCount:4<PHONE_NUMBER>000000000000000000000000000000000000000000000000011110
2024-02-06 22:30:24.366 supervisord: enshrouded-server [TerraformingEfficiencyRegistry] Finished loading.
2024-02-06 22:30:24.431 supervisord: enshrouded-server [ecs] Nobuild zones changed
2024-02-06 22:30:24.432 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 22:30:24.432 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 22:30:24.440 supervisord: enshrouded-server [savexxx] LOAD 1 bases 287 entities
2024-02-06 22:30:24.441 supervisord: enshrouded-server [ecs] readEntitySerializationContext: Templates: 39  Components: 123  Size: 46,112
2024-02-06 22:30:24.444 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry e284b901-c3dd-4282-8a4e-0a618e5763f5.
2024-02-06 22:30:24.444 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 27a9e067-4b9f-45cf-b966-208a5ecbd53b.
2024-02-06 22:30:24.444 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 85e3baea-7c93-4d6f-b1be-42f79b036761.
2024-02-06 22:30:24.444 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 28c4b2ca-4a01-4ba9-9e05-49c871df6904.
2024-02-06 22:30:24.446 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry ec28c5d5-9d23-413e-b238-781157fb2f65.
2024-02-06 22:30:24.446 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 5d9bd376-5b7f-4db4-8f2e-6a3e564194b1.
2024-02-06 22:30:24.446 supervisord: enshrouded-server [guid_registry] VoxelBlueprintRegistry: Finished loading entry 34aa0c3c-9a88-4f8b-a070-64827a5fdabf.
2024-02-06 22:30:24.563 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 22:30:24.564 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 22:30:27.655 supervisord: enshrouded-server [server] Load deserialization took 3.22 s
2024-02-06 22:30:27.677 supervisord: enshrouded-server [ecs] Build zones changed
2024-02-06 22:30:27.699 supervisord: message repeated 3 times: [ enshrouded-server [ecs] Build zones changed]
2024-02-06 22:31:24.448 supervisord: enshrouded-server [ecss] Stats:  Upd:3,577  (16.8ms)  Time:60,011ms  Total:10,144ms  Max:23ms  Avg:2.8ms  +:0  -:0  Ent:894  Pred:0

ok, well if its not a permission issue. Do you have enough space left on the drive? I never had a 0x226 error with steamcmd and I can't find anything about it.
If you also have enough space left on the drive, you can try and delete the enshrouded files inside ./game/servers. But keep the savegame directory. You should probably make a backup for that anyways.

 Big shout out to @cp-fabian-pittroff for the work!
Back on topic:
I ran in to an update issue when I was on different user on my ubuntu box.
Decided to just use the root user to install/setup docker, and I'm also running docker compose up via the root user. I started the server before the Enshrouded update hit Steam yesterday, and to my surprise the server auto updated over night.
I think the permission issue might be the problem, and like cp-fabian-pittroff said if it's not that storage would be my 2nd guess.


HDDS-2950. Upgrade jetty to the latest 9.4 release

What changes were proposed in this pull request?
The jetty which is used by web interfaces of Ozone is from the September of 2018.
Since then many bug and security fixes added to the Jetty project. I would suggest to use the latest jetty (from January of 2020).
As HttpServer2 (hadoop 3.2 class) has a strong dependency on the older version of Jetty (it uses SessionManager which is removed), it seems to be easier to clone HttpServer2 and move it to the Ozone project.
It provides us the flexibility:

To upgrade jetty independent from the Hadoop releases
To remove unused features and make our HTTP stack more reliable (eg. Yarn, WebHDFS part of the code)
To customize our HTTP server usage (eg. SPNEGO should be ignore for S3 REST)
To support ozone style configuration and maintain all our configuration est
To add additional insights/metrics to our own HttpServer
To simplify the current server initialization (current BaseHttpServer class of hdds/ozone is a wrapper around the Hadoop class, but we can combine the two functionalities)

What is included in this patch

HttpServer2 and lightweight dependencies are cloned from Hadoop 3.2
HADOOP-16152 (Jetty upgrade since 3.2 release)
Checkstyle / findbugs (to follow our coding standards)
As we have >5 http related classes I moved them to a dedicated package in the framework project.
**The main logic has not been modified by this patch ** This is mainly code organization and code import.

What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2950
How was this patch tested?
There are related unit tests, but also checked with docker-compose based cluster + checking the ui pages from browser.

Thank you very much @smengcl
As a background: as of now we use hadoop 3.2. The most safest import is to use exactly the same HttpServer2.java what we have in the latest hadoop 3.2 release.
I tried to use trunk HttpServer2 from the Hadoop trunk but found multiple changes related the authorization broke S3 gateway.
After that I decided to keep the same HttpServer2 which is used now (with Jetty patch included).
Therefore (as a rule of thumb) I prefer to import here just minimal changes and import newer patches in separated jira (where we can carefully test s3g).
I checked the suggested patches (thanks for the suggestions):

HADOOP-16727 It's a small null check, can be good to add it as soon as possible. I added it to the patch.
HADOOP-16398 Prometheus support is backported from Ozone to Hadoop. We don't need the patch as we already have the original solution. (Later we can simplify our initialization to make it more similar to the other default servlets but it's a bigger refactor.)
HADOOP-16718 I tried to understand why this is required and didn't find any information. If you have something, let me know. (Might be required for webhdfs which shouldn't be supported in our side.) For me it seems to be optional, and s3g and other ozone components work well based on the tests. Unless we have a strong reason I would prefer to keep the default Jetty behavior (and use one less config).

Thank you very much @smengcl
As a background: as of now we use hadoop 3.2. The most safest import is to use exactly the same HttpServer2.java what we have in the latest hadoop 3.2 release.
I tried to use trunk HttpServer2 from the Hadoop trunk but found multiple changes related the authorization broke S3 gateway.
After that I decided to keep the same HttpServer2 which is used now (with Jetty patch included).
Therefore (as a rule of thumb) I prefer to import here just minimal changes and import newer patches in separated jira (where we can carefully test s3g).
I checked the suggested patches (thanks for the suggestions):

HADOOP-16727 It's a small null check, can be good to add it as soon as possible. I added it to the patch.
HADOOP-16398 Prometheus support is backported from Ozone to Hadoop. We don't need the patch as we already have the original solution. (Later we can simplify our initialization to make it more similar to the other default servlets but it's a bigger refactor.)
HADOOP-16718 I tried to understand why this is required and didn't find any information. If you have something, let me know. (Might be required for webhdfs which shouldn't be supported in our side.) For me it seems to be optional, and s3g and other ozone components work well based on the tests. Unless we have a strong reason I would prefer to keep the default Jetty behavior (and use one less config).

Thanks for digging into this. I'm good with using same HttpServer2.java as Hadoop 3.2.
The two new commits lgtm.
As for HADOOP-16718, I believe the scope is larger than WebHDFS. Web UIs with HTTPS using jetty may all be impacted under certain cases. In short, this can cause connection failure if:

... the server's JKS file has a private/public key/cert pairing that is valid but it also has another trustedCertEntry certificate that has the hostname in subjectAltName extension, the trusted cert gets picked.
This triggers an internal failure to determine a common cipher to use and the server will return the following exception to the client:
fatal error: 40: no cipher suites in common

For details, please take a look at the link I sent over to you on Slack.

I am good with backporting HADOOP-16718 in another jira. So far lgtm+1
Thanks @elek @arp7

Thanks @elek for the update. The latest change LGTM, +1.


refactor! Change ComputeExtendedDataSquare argument from TreeConstructorFn to string

Overview

As mention in this comment and this comment
This PR help track the breaking changes of the code

Changes

Replace ComputeExtendedDataSquare TreeConstructorFn type argument to a string type represent the treeName

Checklist

[ ] New and updated code has appropriate documentation
[ ] New and updated code has new and/or updated testing
[ ] Required CI checks are passing
[ ] Visual proof for any user facing features like CLI or documentation updates
[ ] Linked issues closed with keywords

close in favor of #287