Return the files listed so far instead of null even in case of archive error in getFileList()

This allows to process incomplete RAR files: a partial buffer from stream for example. In my case I have like 50 MiB archive loading slower than 1 MiB/s and being able to continuously list files while archive is still loading would be a really nice user experience.
I don't insist on this change, but if you agree it is useful, it would be cool if it got accepted
If something else is needed for this PR to go through, just let me know, I have no life luckily ^_^

In the new release v1.0.0, the files will not be extracted in one function all.
The files will be returned as an iterator and you can traverse the list by for of so partial of the archive can be extracted even some files are broken:
const extracted = [];
try{
  for (const file of files) {
    extracted.push(file.extraction);
  }
} catch (unRarErr) {
  // ...
}

Hi. If you are still going to implement the iterator, I'd like to share some info I found after few days of struggling.
https://github.com/klesun-misc/node-unrar.js-issue3/commit/cdeb62acb511a0c081e834da5e4976f062c68751
There is this QOpenMode parameter that is by default QOPEN_AUTO because of which rar lib jumps over all file for integrity check before listing files. When archivce size is 1+ GiB this always results in READ_ERROR. I changed the default value from QOPEN_AUTO to QOPEN_NONE and 1+ GiB archives now open like a charm ^_^

Hi. If you are still going to implement the iterator, I'd like to share some info I found after few days of struggling.
https://github.com/klesun-misc/node-unrar.js-issue3/commit/cdeb62acb511a0c081e834da5e4976f062c68751
There is this QOpenMode parameter that is by default QOPEN_AUTO because of which rar lib jumps over all file for integrity check before listing files. When archivce size is 1+ GiB this always results in READ_ERROR. I changed the default value from QOPEN_AUTO to QOPEN_NONE and 1+ GiB archives now open like a charm ^_^

Hi. If you are still going to implement the iterator, I'd like to share some info I found after few days of struggling.
klesun-misc@cdeb62a
There is this QOpenMode parameter that is by default QOPEN_AUTO because of which rar lib jumps over all file for integrity check before listing files. When archivce size is 1+ GiB this always results in READ_ERROR. I changed the default value from QOPEN_AUTO to QOPEN_NONE and 1+ GiB archives now open like a charm ^_^

Thanks for your comment. The migration is already done, you can try if the big archive is working in the new version. I can add your fix into the repo (by override the  original file) if this is required.

Hi. If you are still going to implement the iterator, I'd like to share some info I found after few days of struggling.
klesun-misc@cdeb62a
There is this QOpenMode parameter that is by default QOPEN_AUTO because of which rar lib jumps over all file for integrity check before listing files. When archivce size is 1+ GiB this always results in READ_ERROR. I changed the default value from QOPEN_AUTO to QOPEN_NONE and 1+ GiB archives now open like a charm ^_^

Thanks for your comment. The migration is already done, you can try if the big archive is working in the new version. I can add your fix into the repo (by override the  original file) if this is required.

Thanks, trying out!
A small offtopic, I tried to run npm run build:release, and the output /esm/ directory holds index.esm.js and the rest unbundled files that are imported without .js extension:
import { ExtractorData } from './js/ExtractorData';

Considering I'm not magically resolving them in my web-server code, that results in such errors:

Thanks, trying out!
A small offtopic, I tried to run npm run build:release, and the output /esm/ directory holds index.esm.js and the rest unbundled files that are imported without .js extension:
import { ExtractorData } from './js/ExtractorData';

Considering I'm not magically resolving them in my web-server code, that results in such errors:

Oh, sorry, never mind, this /esm/ is probably intended for node environment, not for browser

Oh, sorry, never mind, this /esm/ is probably intended for node environment, not for browser

Hm, not sure I'm doing it right, but now I get the File read error even when I try to open a normal rar file of around 100 MiB...

Hm, not sure I'm doing it right, but now I get the File read error even when I try to open a normal rar file of around 100 MiB...

Oh, sorry, never mind, this /esm/ is probably intended for node environment, not for browser
Upd.: or is it...

The /esm/ is ESModule which used for webpack (browser) and the /dist/ is commonjs module which used for nodejs.

Hm, not sure I'm doing it right, but now I get the File read error even when I try to open a normal rar file of around 100 MiB in the browser (after monkey-patching the missing .js extensions in imports and path to unrar.wasm)...

I have no chance to test on the large file. Have you tried the demo in /demo/web? Is it working for small file but not for bigfile?

Oh, sorry, never mind, this /esm/ is probably intended for node environment, not for browser
Upd.: or is it...

The /esm/ is ESModule which used for webpack (browser) and the /dist/ is commonjs module which used for nodejs.

Hm, not sure I'm doing it right, but now I get the File read error even when I try to open a normal rar file of around 100 MiB in the browser (after monkey-patching the missing .js extensions in imports and path to unrar.wasm)...

I have no chance to test on the large file. Have you tried the demo in /demo/web? Is it working for small file but not for bigfile?

Yeah, sorry, only now noticed the demo, researching it.
In case you need a big rar file, I can share this guy: ;)
http://kunkka-torrent.online/kunkka_host/handmade/torrent_downloads/DLraw.net-Reincarnation Kaben vol 01-12.rar
Will write back once make the demo work.

Yeah, sorry, only now noticed the demo, researching it.
In case you need a big rar file, I can share this guy: ;)
http://kunkka-torrent.online/kunkka_host/handmade/torrent_downloads/DLraw.net-Reincarnation Kaben vol 01-12.rar
Will write back once make the demo work.

Yeah, sorry, only now noticed the demo, researching it.
In case you need a big rar file, I can share this guy: ;)
http://kunkka-torrent.online/kunkka_host/handmade/torrent_downloads/DLraw.net-Reincarnation Kaben vol 01-12.rar
Will write back once make the demo work.

The file is 800 MB!
Anyway, it's decompressed in the web demo on browser.
You may need to add some waitting function and remove content form the demo:
--- a/demo/web/src/index.ts
+++ b/demo/web/src/index.ts
@@ -68,9 +68,10 @@ document.querySelector<HTMLDivElement>('#file')?.addEventListener(
         filesDom.appendChild(
           fillTemplate('content', {
             fileName: file.fileHeader.name,
-            content: new TextDecoder('utf-8').decode(file.extraction),
+            content: '',//new TextDecoder('utf-8').decode(file.extraction),
           }),
         );
+        await new Promise<void>(r=>setTimeout(()=>r(), 10));
       }
     } catch (err) {
       if (err instanceof UnrarError) {

Yeah, sorry, only now noticed the demo, researching it.
In case you need a big rar file, I can share this guy: ;)
http://kunkka-torrent.online/kunkka_host/handmade/torrent_downloads/DLraw.net-Reincarnation Kaben vol 01-12.rar
Will write back once make the demo work.

The file is 800 MB!
Anyway, it's decompressed in the web demo on browser.
You may need to add some waitting function and remove content form the demo:
--- a/demo/web/src/index.ts
+++ b/demo/web/src/index.ts
@@ -68,9 +68,10 @@ document.querySelector<HTMLDivElement>('#file')?.addEventListener(
         filesDom.appendChild(
           fillTemplate('content', {
             fileName: file.fileHeader.name,
-            content: new TextDecoder('utf-8').decode(file.extraction),
+            content: '',//new TextDecoder('utf-8').decode(file.extraction),
           }),
         );
+        await new Promise<void>(r=>setTimeout(()=>r(), 10));
       }
     } catch (err) {
       if (err instanceof UnrarError) {

Hi again, thank you very much, I managed to make demo page working.
Yeah, I do experience the READ_ERROR when I try to load my large rar file that I cut after first 10 MiB before first file is listed.

If I open same file with an archive manager, it gets open ok:

And similarly, it opens ok in my patched version of node-unrar.js with QOpenMode changed from QOPEN_AUTO to QOPEN_NONE.
Sorry, I was not clear enough, the problem is not in trying to parse large file itself, but in trying to parse an incomplete part of it (like when you want to progressively show the buffer what you downloaded so far during a lengthy http retrieval).

Hi again, thank you very much, I managed to make demo page working.
Yeah, I do experience the READ_ERROR when I try to load my large rar file that I cut after first 10 MiB before first file is listed.

If I open same file with an archive manager, it gets open ok:

And similarly, it opens ok in my patched version of node-unrar.js with QOpenMode changed from QOPEN_AUTO to QOPEN_NONE.
Sorry, I was not clear enough, the problem is not in trying to parse large file itself, but in trying to parse an incomplete part of it (like when you want to progressively show the buffer what you downloaded so far during a lengthy http retrieval).

Here is the cut file in question:
https://kunkka-torrent.online/kunkka_host/handmade/torrent_downloads/dlraw_huj_10mb.rar
Opens in archive manager, does not open with node-unrar.js

Here is the cut file in question:
https://kunkka-torrent.online/kunkka_host/handmade/torrent_downloads/dlraw_huj_10mb.rar
Opens in archive manager, does not open with node-unrar.js

Got it.
So you are trying to make it works for incomplete file? By change the QOPEN define to bypass the integrity checking of unrar?
I'd not prefer to change this option in the official build, what about fork it and try to recompile for your case?
This version should be easier than the last one to for compilation.

Got it.
So you are trying to make it works for incomplete file? By change the QOPEN define to bypass the integrity checking of unrar?
I'd not prefer to change this option in the official build, what about fork it and try to recompile for your case?
This version should be easier than the last one to for compilation.

Well, I already have such fork ;)
https://github.com/klesun-misc/node-unrar.js-issue3/tree/partial-streaming-support-change-cpp
Though I'd love to get rid of it in favor in the main repo.
Maybe it could be specified by option somehow? I'm not a big emscripten guru, but this QOPEN thing seems to be one of "Command Arguments" or something like that...

Well, I already have such fork ;)
https://github.com/klesun-misc/node-unrar.js-issue3/tree/partial-streaming-support-change-cpp
Though I'd love to get rid of it in favor in the main repo.
Maybe it could be specified by option somehow? I'm not a big emscripten guru, but this QOPEN thing seems to be one of "Command Arguments" or something like that...

Not that I'm willing to force you to waste your time on that, just wanted to let you know about such a use case.

Not that I'm willing to force you to waste your time on that, just wanted to let you know about such a use case.

Yes; Yes; Well, I already have such fork ;)
https://github.com/klesun-misc/node-unrar.js-issue3/tree/partial-streaming-support-change-cpp
Though I'd love to get rid of it in favor in the main repo.
Maybe it could be specified by option somehow? I'm not a big emscripten guru, but this QOPEN thing seems to be one of "Command Arguments" or something like that...
If I get it right, this cmddata.cpp can even be specified in the rar archive buffer itself by prepending some special characters...

The cpp code is not working like this. The QOPEN is a flag which set by another cpp header and fixed in the building artifacts.
The only way to make it configurable is: copy the file, apply your change, build two different wasm files (one with the original file and another with the modified one), and load the different file based on the option.
I'd prefer to keep the changes on the cpp source minimum so I'd not go to that way.

Not that I'm willing to force you to waste your time on that, just wanted to let you know about such a use case.

Anyway thanks you for investigation on this topic. I believe it will be useful for other users with the same requirements :)

Yes; Yes; Well, I already have such fork ;)
https://github.com/klesun-misc/node-unrar.js-issue3/tree/partial-streaming-support-change-cpp
Though I'd love to get rid of it in favor in the main repo.
Maybe it could be specified by option somehow? I'm not a big emscripten guru, but this QOPEN thing seems to be one of "Command Arguments" or something like that...
If I get it right, this cmddata.cpp can even be specified in the rar archive buffer itself by prepending some special characters...

The cpp code is not working like this. The QOPEN is a flag which set by another cpp header and fixed in the building artifacts.
The only way to make it configurable is: copy the file, apply your change, build two different wasm files (one with the original file and another with the modified one), and load the different file based on the option.
I'd prefer to keep the changes on the cpp source minimum so I'd not go to that way.

Not that I'm willing to force you to waste your time on that, just wanted to let you know about such a use case.

Anyway thanks you for investigation on this topic. I believe it will be useful for other users with the same requirements :)

Ok, thanks to you as well.
One more small note for history: one nasty thing about this integrity check is that it does not run on any rar file, but only on files with MHEXTRA_LOCATOR flag in the extra fields data of the main header, so this issue will only reproduce on some rar files, but not on all of them...

Ok, thanks to you as well.
One more small note for history: one nasty thing about this integrity check is that it does not run on any rar file, but only on files with MHEXTRA_LOCATOR flag in the extra fields data of the main header, so this issue will only reproduce on some rar files, but not on all of them...


tflite toco build failed in version of 3daa07aa2dde379388beb2a557a78bc5dd1b86ba

System information

Have I written custom code : No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 17.10
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): tensorflow 1.8
Python version: 2.7
Bazel version (if compiling from source): 0.13.0
GCC/Compiler version (if compiling from source): 6.4
CUDA/cuDNN version: None
GPU model and memory: GeForce GTX 750
Exact command to reproduce: tensorflow code :
commit 3daa07aa2dde379388beb2a557a78bc5dd1b86ba
Author: Courtial Florian<EMAIL_ADDRESS>Date:   Wed Jun 6 01:02:07 2018 +0200
Add C++ no gradient for Floor operation. (#19662)

Describe the problem
Use cmd below:
bazel build --config android_arm --cxxopt=-std=c++11  //tensorflow/contrib/lite/toco:toco   --config monolithic
My WORKSPACE as below:
WORKSPACE
android_sdk_repository(
name = "androidsdk",
api_level = 22,
# Ensure that you have the build_tools_version below installed in the
# SDK manager as it updates periodically.
build_tools_version = "27.0.3",
# Replace with path to Android SDK on your system
path = "/home/hwh/Android/Sdk",
)
android_ndk_repository(
name="androidndk",
path="/home/hwh/Android/Ndk/android-ndk-r14b",
# This needs to be 14 or higher to compile TensorFlow.
# Please specify API level >= 21 to build for 64-bit architecture
# otherwise the Android NDK will automatically select the latest
# API level it does support without notice.
# Note that the NDK version is not the API level.
api_level=14)
build failed as below:
INFO: Found 1 target...
ERROR: /home/hwh/.cache/bazel/_bazel_hwh/a5d93fc2a8e4cfbb4b042a3b705d2c38/external/gif_archive/BUILD.bazel:8:1: C++ compilation of rule '@gif_archive//:gif' failed (Exit 1)
external/gif_archive/lib/openbsd-reallocarray.c:33:19: error: use of undeclared identifier 'SIZE_MAX'
nmemb > 0 && SIZE_MAX / nmemb < size) {
^
1 error generated.
Target //tensorflow/contrib/lite/toco:toco failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 167.962s, Critical Path: 29.34s
INFO: 374 processes, local.
FAILED: Build did NOT complete successfully

Now I can build toco by the command below:
bazel build tensorflow/contrib/lite/toco:toco

I don't believe that the toco binary is meant to be compiled with --config=android_arm, so I think this is working as intended.
CC @suharshs @aselle

Yes, I found I needn't set to arm platform as you figured out this.
And I had a mistake to used this opt of  --config=android_arm because the
comment below:
commit 320d8056af7799ab20e339757cf379963148425a
Author: freedom" Koan-Sin Tan<EMAIL_ADDRESS>Date:   Mon Jun 4 12:49:17 2018 +0800
make toco build for android (#17885)

* make toco build for android

2018-06-07 17:35 GMT+08:00 Asim Shankar<EMAIL_ADDRESS>
I don't believe that the toco binary is meant to be compiled with
--config=android_arm, so I think this is working as intended.
CC @suharshs https://github.com/suharshs @aselle
https://github.com/aselle
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/19798#issuecomment-395356230,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AcPuIUWZYz8vDXjpvzg8DHsjz1tc6syQks5t6PPWgaJpZM4UcRqK
.

Great, so if I understand correctly, there is no issue now?

Yes, No issue for me, thanks a lot.
2018-06-09 0:41 GMT+08:00 Asim Shankar<EMAIL_ADDRESS>
Great, so if I understand correctly, there is no issue now?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/19798#issuecomment-395817378,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AcPuIe9h8S8BwtxCWi4lDtkooGpjUzB0ks5t6qlVgaJpZM4UcRqK
.


GetReply(Sync+Async)Internal enters an infinite loop

FTP Server OS: Unix / Windows
FTP Server Type: FTPS
Client Computer OS: Windows
FluentFTP Version: <IP_ADDRESS>
Framework: .NET 6
The method GetReplyAsyncInternal enters an infinite loop.
if (!exhaustNoop || elapsedMilliseconds <= (long) timeOut)
{
    if (!exhaustNoop)
    {
        if (timeOut >= 0)
        {
            client.m_stream.ReadTimeout = __nonvirtual (client.Config).ReadTimeout;
            text = await client.m_stream.ReadLineAsync(__nonvirtual (client.Encoding), token);
        }

await client.m_stream.ReadLineAsync(__nonvirtual (client.Encoding), token); always returns null.
Logs :
# Connect(False)
Status:   FluentFTP <IP_ADDRESS>
Status:   Connecting to IP #1= ***:990
Warning:  SSL Buffering force disabled, is .NET 5.0 and later
Status:   FTPS authentication successful, protocol = Tls12
Status:   Time to activate encryption: 0h 0m 0s.  Total Seconds: 0.0248618.
Status:   SslStream: Tls12 (Aes256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, 44550, 256)
Status:   Waiting for a response
Response: 220-== Virtual Benefits Administrator SFTP Server -============================
Response: 220-== This is a private network.
Response: 220-== Unauthorized access and/or actions will result iin prosecution.
Response: 220 == For assistance, contact<PHONE_NUMBER>. =============================== [738539.427d]
Command:  USER ***
Status:   Waiting for response to: USER ***
Response: 331 User ***, password please [23ms]
Command:  PASS ***
Status:   Waiting for response to: PASS ***
Response: 530 Not logged in. Username/password incorrect, user disabled, or user logged in too many times [3.02s]

# Dispose()
Status:   Disposing FtpClient object...
Status:   Closing/Disposing FtpSocketStream(control connection)

# Connect(False)
Status:   FluentFTP <IP_ADDRESS>
Status:   Connecting to IP #1= ***:990
Warning:  SSL Buffering force disabled, is .NET 5.0 and later
Status:   FTPS authentication successful, protocol = Tls12
Status:   Time to activate encryption: 0h 0m 0s.  Total Seconds: 0.0268872.
Status:   SslStream: Tls12 (Aes256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, 44550, 256)
Status:   Waiting for a response
Response: 220-== Virtual Benefits Administrator SFTP Server -============================
Response: 220-== This is a private network.
Response: 220-== Unauthorized access and/or actions will result iin prosecution.
Response: 220 == For assistance, contact<PHONE_NUMBER>. =============================== [738539.431d]
Command:  USER ***
Status:   Waiting for response to: USER ***
Response: 331 User ***, password please [25ms]
Command:  PASS ***
Status:   Waiting for response to: PASS ***

I can see a potential for an infinite loop when stream read returns null, which can happen if the socket is no longer connected and the timeout does not fire. I tested it just now on sync, so the issue is on both, I think.

Virtual Benefits Administrator SFTP Server??? Buddy we don't even support SFTP, use SSH.NET for that.

Virtual Benefits Administrator SFTP Server??? Buddy we don't even support SFTP, use SSH.NET for that.
Also, what type of server is this? It is not identifying itself in the welcome message?

FTP Server Type: FTPS
For clarity I can connect to this server via FluentFTP.

Doesn't change anything, @robinrodricks
I can reproduce an infinite loop in GetReply, when:
While waiting for a response and not reached the socket.timeout, using ReadLine, and then if socket is closed due to external reasons. In this case we don't get the timeout and return a null. Which then causes the loop.

Also, what type of server is this? It is not identifying itself in the welcome message?

I dunno, maybe the company is called "Virtual Benefits", and the welcome message has been customized (sure looks like it) and the geek that did it can't spell FTPS (used SFTP instead).

@timurposhva Can you test the current master concerning this issue of yours?

@timurposhva Can you test the current master concerning this issue of yours? Works on my scenario.

Sure. I will do it tomorrow.

@timurposhva Can you test the current master concerning this issue of yours? Works on my scenario.

The problem is not reproduced.

So does that mean it is fixed or it is not fixed?

So does that mean it is fixed or it is not fixed?

fixed

FYI, was able to reproduce via nuget FluentFTP 44.0.1, pure-ftpd (docker stilliard/pure-ftpd) via a FTPS FtpClient.GetListing call.
Confirmed GetReplyInternal enters infinite loop when the FtpSocketStream.BaseStream is null, via the FtpSocketStream.Read method constantly returning 0.
Have not tested master yet, just providing extra info. If any additional info would be helpful about reproduction, just let me know.

Thanks for this further confirmation of the observed problem.

Have not tested master yet

Should be ok with that code level.

Hi,
I can reproduce this one too quite often and is killing us.
Is there a rollback version which is not affected?
Thanks
A

It's fixed (in master), but pending release.

Is there a rollback version which is not affected?

You would need to rollback to prior to the GetReply rewrite, which is quite far (many months). If you are experiencing this in such a frequent manner, I am surprised that you didn't report this months ago?

Is there a rollback version which is not affected?

You would need to rollback to prior to the GetReply rewrite, which is quite far (many months). If you are experiencing this in such a frequent manner, I am surprised that you didn't report this months ago?

Thanks for the kind reply @FanDjango .
Many things have changed both on my side and server side, let's say. So it may be that some other change worsened the situation.
I switched to version 40.0.0 on september 14th, then stayed with that until January 17th, when I updated to 44.0.1
Thanks
A

I understand. Unless you can implement current master, then the next Nuget will be your solution.
Either @robinrodricks produces a quick interim release or it will be another week or so, because there is something big in the pipeline.

Ok thanks. But the GetReply has been rewritten in 40.x big bang?
A

Use master, so much is clear from the above. And if it does not help, I am here to fix it ASAP.

What about generating a 44.0.2-beta package and publish on github as it was done for 40.x?
A

What about generating a 44.0.2-beta package and publish on github as it was done for 40.x?

I will defer that decision to @robinrodricks

Ok, I guess it's a no.
No worries though, I'll wait for the official release by the end of the week otherwise I'll create a beta locally here.

Do you have clearer plan for an official release now?
If not, I'll make the beta from master on my own.
Thanks a lot
Cheers
A

Released! V45.0.3

Sigh, still seems present with 45.0.4 (nuget). Hangs on client.GetListing(). Not sure if separate (new) issue or not though. Let me know if I can provide any additional information.
[Info] >         Connect(False)
[Verbose] FluentFTP <IP_ADDRESS>
[Info] Connecting to IP #1= <IP_ADDRESS>:2121
[Verbose] Waiting for a response
[Verbose] Response: 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
Response: 220-You are user number 1 of 50 allowed.
Response: 220-Local time is now 00:05. Server port: 21.
Response: 220-This is a private system - No anonymous login
Response: 220-IPv6 connections are also welcome on this server.
[Info] Response: 220 You will be disconnected after 15 minutes of inactivity. [738568.004d]
[Info] Status:   Detected FTP server: PureFTPd
[Info] Command:  AUTH TLS
[Verbose] Waiting for response to: AUTH TLS
[Info] Response: 234 AUTH TLS OK. [10ms]
[Info] FTPS authentication successful, lib = .NET SslStream, cipher suite = Tls12 (Aes128, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, DiffieHellman, 0) [290ms]
[Info] Command:  USER testuser
[Verbose] Waiting for response to: USER testuser
[Info] Response: 331 User testuser OK. Password required [1ms]
[Info] Command:  PASS ***
[Verbose] Waiting for response to: PASS ***
[Info] Response: 230 OK. Current directory is / [12ms]
[Info] Command:  PBSZ 0
[Verbose] Waiting for response to: PBSZ 0
[Info] Response: 200 PBSZ=0 [1ms]
[Info] Command:  PROT P
[Verbose] Waiting for response to: PROT P
[Info] Response: 200 Data protection level set to "private" [8ms]
[Info] Command:  FEAT
[Verbose] Waiting for response to: FEAT
[Verbose] Response: 211-Extensions supported:
Response: EPRT
Response: IDLE
Response: MDTM
Response: SIZE
Response: MFMT
Response: REST STREAM
Response: MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
Response: MLSD
Response: AUTH TLS
Response: PBSZ
Response: PROT
Response: UTF8
Response: TVFS
Response: ESTA
Response: PASV
Response: EPSV
Response: SPSV
[Info] Response: 211 End. [9ms]
[Info] Text encoding: System.Text.UTF8Encoding+UTF8EncodingSealed
[Info] Command:  OPTS UTF8 ON
[Verbose] Waiting for response to: OPTS UTF8 ON
[Info] Response: 200 OK, UTF-8 enabled [1ms]
[Info] Command:  SYST
[Verbose] Waiting for response to: SYST
[Info] Response: 215 UNIX Type: L8 [2ms]
[Verbose] Listing parser set to: Machine
[Info] Command:  PWD
[Verbose] Waiting for response to: PWD
[Info] Response: 257 "/" is your current location [5ms]
[Info] >         GetListing(null, Auto)
[Info] Command:  TYPE I
[Verbose] Waiting for response to: TYPE I
[Info] Response: 200 TYPE is now 8-bit binary [<1ms]
[Info] >         OpenDataStream("MLSD /", 0)
[Info] >         OpenPassiveDataStream(AutoPassive, "MLSD /", 0)
[Info] Command:  EPSV
[Verbose] Waiting for response to: EPSV
[Info] Response: 229 Extended Passive mode OK (|||21223|) [6ms]
[Info] Connecting to IP #1= <IP_ADDRESS>:21223
[Info] Command:  MLSD /
[Verbose] Waiting for response to: MLSD /
[Info] Response: 150 Accepted data connection [3ms]
[Info] FTPS authentication successful, lib = .NET SslStream, cipher suite = Tls12 (Aes128, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, DiffieHellman, 0) [28ms]
[Verbose] +---------------------------------------+
[Verbose] -----------------------------------------
[Verbose] Closing/Disposing FtpSocketStream(data connection)
[Info] >         CloseDataStream()
[Verbose] Waiting for a response
<...hangs...>

using FluentFTP;

public class Program
{
    private class FtpClientLogger : IFtpLogger
    {
        public void Log(FtpLogEntry entry)
        {
            Console.WriteLine($"[{entry.Severity}] {entry.Message}");
            if (entry.Exception != null) Console.WriteLine(entry.Exception);
        }
    }

    public static void Main(string[] args)
    {
        var client = new FtpClient(
            host: "<IP_ADDRESS>",
            port: 2121,
            user: "testuser",
            pass: "testPass1!",
            logger: new FtpClientLogger(),
            config: new FtpConfig
            {
                LogHost = true,
                LogUserName = true,
                LogPassword = false,
                DataConnectionReadTimeout = 1000 * 30,
                EncryptionMode = FtpEncryptionMode.Auto,
                SslBuffering = FtpsBuffering.Off,
                ValidateAnyCertificate = true,
                ConnectTimeout = 1000 * 30,
                ReadTimeout = 1000 * 30,
            }
        );

        client.Connect();

        var items = client.GetListing() ?? Array.Empty<FtpListItem>();
        Console.WriteLine($"Found {items.Length} items");
    }
}

docker-compose.yaml
version: "3.5"
services:
  ftps:
    container_name: ftps
    image: stilliard/pure-ftpd:latest
    restart: unless-stopped
    ports:
      - "2121:21"
      - "21200-21299:21200-21299"
    environment:
      PUBLICHOST: "<IP_ADDRESS>"
      FTP_USER_NAME: testuser
      FTP_USER_PASS: testPass1!
      FTP_USER_HOME: /home/testuser
      FTP_PASSIVE_PORTS: 21200:21299
      FTP_MAX_CLIENTS: 50
      FTP_MAX_CONNECTIONS: 50
      ADDED_FLAGS: "--tls=2"
      TLS_USE_DSAPRAM: "true"
      TLS_CN: "localhost"
      TLS_ORG: "Test"
      TLS_C: "US"

@maxrunsoftware This one is slightly different although the place where it happens and the sympoms are nearly the same.
I wonder how you achieve this situation. Have tried to recreate with a good copy of your docker, using the same pure-ftpd. It doesn't happen, sigh.
Up to now, I was assuming the loop occurs on account of ReadLine (et. al.) returning null because the control connection has disconnected (for whatever reason) - and those cases above now react as expected. Your case, as can be seen from the verbose log (thanks for that, by the way), seems to be: null returned although control connection is still connected. Of course this is not caught by the check for disconnect, and although I still wonder how it happens, I can check for this.
So I have re-done the coding. Are you able to test this case using a special branch I will make available to you?

@FanDjango Sure! Perhaps issue with .Net6 Socket implementation on OSX. Dev box is OSX 13.2.1 using Rider, Docker VM is Debian GNU/Linux 11 (bullseye) (kernel 5.10.0-18-amd64). I will try same code on Windows box later today against same docker VM to see if issue is present or is specific to OSX.
If you have a specific branch you want me to test though, I can compile and test to see if issue is still present.

The GetListing output you showed in the log, was that redacted or is the directory actually empty?

So we are waiting for a 226.... or a failure response, no response ever arrives and it hangs. You should now get a timeout (after some time) with the new branch and not a hang.
That still leaves the question open what is causing the total silence on the control connection... check the server log in the container maybe to see what the server says...

Will do. Original log was with empty / root directory. Tested 45.0.4 with creating a empty subdirectory "TestDirectory" and with "TestDirectory" and "TestFile.txt" in / and all produces same log result.
I'll try and test the new branch later today and also the Windows client tests. Here are the server logs with 45.0.4 with test program vs FileZilla (OSX) logs if it helps any.
Log of docker image when running test program...
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[INFO] New connection from <IP_ADDRESS>
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220-You are user number 2 of 50 allowed.
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220-Local time is now 16:38. Server port: 21.
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220-This is a private system - No anonymous login
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220-IPv6 connections are also welcome on this server.
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220 You will be disconnected after 15 minutes of inactivity.
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [auth] [TLS]
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 234 AUTH TLS OK.
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[INFO] TLS: Enabled TLSv1.2 with ECDHE-RSA-AES128-GCM-SHA256, 128 secret bits cipher
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [user] [testuser]
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 331 User testuser OK. Password required
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [pass] [<*>]
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[INFO] testuser is now logged in
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 230 OK. Current directory is /
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [pbsz] [0]
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 200 PBSZ=0
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [prot] [P]
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 200 Data protection level set to "private"
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [feat] []
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 211-Extensions supported:#015#012 EPRT#015#012 IDLE#015#012 MDTM#015#012 SIZE#015#012 MFMT#015#012 REST STREAM#015#012 MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;#015#012 MLSD#015#012 AUTH TLS#015#012 PBSZ#015#012 PROT#015#012 UTF8#015#012 TVFS#015#012 ESTA#015#012 PASV#015#012 EPSV#015#012 SPSV#015
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 211 End.
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [opts] [UTF8 ON]
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 200 OK, UTF-8 enabled
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [syst] []
Feb 18 16:38:19 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 215 UNIX Type: L8
Feb 18 16:38:20 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [pwd] []
Feb 18 16:38:20 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 257 "/" is your current location
Feb 18 16:38:20 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [type] [I]
Feb 18 16:38:20 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 200 TYPE is now 8-bit binary
Feb 18 16:38:20 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [epsv] []
Feb 18 16:38:20 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 229 Extended Passive mode OK (|||21285|)
Feb 18 16:38:20 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [mlsd] [/]
Feb 18 16:38:20 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 150 Accepted data connection

Server log with FileZilla connection...
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[INFO] New connection from <IP_ADDRESS>
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220-You are user number 1 of 50 allowed.
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220-Local time is now 16:44. Server port: 21.
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220-This is a private system - No anonymous login
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220-IPv6 connections are also welcome on this server.
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 220 You will be disconnected after 15 minutes of inactivity.
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [auth] [TLS]
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 234 AUTH TLS OK.
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[INFO] TLS: Enabled TLSv1.3 with TLS_AES_256_GCM_SHA384, 256 secret bits cipher
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [user] [testuser]
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 331 User testuser OK. Password required
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [pass] [<*>]
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[INFO] testuser is now logged in
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 230 OK. Current directory is /
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [opts] [UTF8 ON]
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 200 OK, UTF-8 enabled
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [pbsz] [0]
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 200 PBSZ=0
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [prot] [P]
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 200 Data protection level set to "private"
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] Command [pwd] []
Feb 18 16:44:02 dd378aedb51e pure-ftpd<EMAIL_ADDRESS>[DEBUG] 257 "/" is your current location

FileZilla (OSX) log...
Status:      	Connecting to <IP_ADDRESS>:2121...
Status:      	Connection established, waiting for welcome message...
Response: 	220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
Response: 	220-You are user number 1 of 50 allowed.
Response: 	220-Local time is now 16:44. Server port: 21.
Response: 	220-This is a private system - No anonymous login
Response: 	220-IPv6 connections are also welcome on this server.
Response: 	220 You will be disconnected after 15 minutes of inactivity.
Command:	AUTH TLS
Response: 	234 AUTH TLS OK.
Status:      	Initializing TLS...
Status:      	TLS connection established.
Command:	USER testuser
Response: 	331 User testuser OK. Password required
Command:	PASS **********
Response: 	230 OK. Current directory is /
Command:	OPTS UTF8 ON
Response: 	200 OK, UTF-8 enabled
Command:	PBSZ 0
Response: 	200 PBSZ=0
Command:	PROT P
Response: 	200 Data protection level set to "private"
Status:      	Logged in
Status:      	Retrieving directory listing...
Command:	PWD
Response: 	257 "/" is your current location
Status:      	Directory listing of "/" successful

Interesting. I don't see any directory listing happening in your "Server log with FileZilla connection...". Althoug the filezilla client log says "Retrieving directory listing...", none of that to be seen either. Incomplete logs?

Can you move this to a new issue? We are cluttering this one.

Also, does GetListing work at all? I mean, with a non-empty directory?

Can do. Opened https://github.com/robinrodricks/FluentFTP/issues/1203
GetListing does not appear to function at all regardless of directory contents. Tested with empty directory, single file in directory, single subdirectory in directory, and both file and subdirectory in directory. Will post new testing info in other ticket.

@Adhara3 Is this ok for you now? Then I will close.

@FanDjango I stress tested the system with this latest version and everything seems fine.
I'm ok with closing.
Thanks a lot
A

@FanDjango it happens again. I don't know if that's the same, but here we go.
02-23 09:32:21.701  Since we are connected, we update ECU status
02-23 09:32:21.959 Telnet Client <IP_ADDRESS>: Disconnected from <IP_ADDRESS>:23

The above is my application code, I do try to issue a custom command (status) via FTP, but I also have a Telnet connection open for other reasons. So I start issuing the command, then I probably lose connection with the device as you can see from the logs. This is just for context and timestamps
On the FTP only side
The status command before the disconnection failed "gracefully"
02-23 09:32:20.921 INFO  FtpLogger
02-23 09:32:20.921 INFO  FtpLogger Executing stashed command
02-23 09:32:20.921 INFO  FtpLogger
02-23 09:32:20.921 INFO  FtpLogger Command:  XDAQ daq status
02-23 09:32:20.921 DEBUG FtpLogger Waiting for response to: XDAQ daq status
02-23 09:32:21.422 DEBUG FtpLogger Closing/Disposing FtpSocketStream(control connection)
02-23 09:32:21.422 ERROR FtpLogger Exception while executing FTP command.
System.TimeoutException: Timed out trying to read data from the socket stream!
   at FluentFTP.FtpSocketStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at FluentFTP.FtpSocketStream.ReadLine(Encoding encoding)
   at FluentFTP.Client.BaseClient.BaseFtpClient.GetReplyInternal(String command, Boolean exhaustNoop, Int32 timeOut)
   at FluentFTP.Client.BaseClient.BaseFtpClient.FluentFTP.IInternalFtpClient.ExecuteInternal(String command)

then
02-23 09:32:21.702 WARN  FtpLogger Reconnect needed due to disconnected control connection (SslSessionLength: 0)
02-23 09:32:21.702 INFO  FtpLogger Command stashed: XDAQ daq status
02-23 09:32:21.702 INFO  FtpLogger >         Connect(True)
02-23 09:32:21.702 WARN  FtpLogger Reconnect (Count: 2)
02-23 09:32:21.702 DEBUG FtpLogger FluentFTP <IP_ADDRESS>
02-23 09:32:21.702 INFO  FtpLogger Connecting to IP #1= ***:21
02-23 09:32:21.702 DEBUG FtpLogger Waiting for a response
02-23 09:32:21.704 INFO  FtpLogger Response: 220 Daq FTP Server ready. [782ms]
02-23 09:32:21.704 INFO  FtpLogger Command:  USER ***
02-23 09:32:21.704 DEBUG FtpLogger Waiting for response to: USER ***
02-23 09:32:21.707 INFO  FtpLogger Response: 331 Password required for ***. [2ms]
02-23 09:32:21.707 INFO  FtpLogger Command:  PASS ***
02-23 09:32:21.707 DEBUG FtpLogger Waiting for response to: PASS ***
02-23 09:32:21.709 INFO  FtpLogger Response: 230 User *** logged in. [1ms]
02-23 09:32:21.709 INFO  FtpLogger Text encoding: System.Text.ASCIIEncoding
02-23 09:32:21.709 INFO  FtpLogger Command:  SYST
02-23 09:32:21.709 DEBUG FtpLogger Waiting for response to: SYST
02-23 09:32:21.710 INFO  FtpLogger Response: 215 DAQ, Release 6.00.07 [1ms]
02-23 09:32:21.710 DEBUG FtpLogger Listing parser set to: Custom
02-23 09:32:21.710 INFO  FtpLogger >         SetWorkingDirectory("/mnt/mmc/_daq2Data")
02-23 09:32:21.710 INFO  FtpLogger Command:  CWD /mnt/mmc/_daq2Data
02-23 09:32:21.710 DEBUG FtpLogger Waiting for response to: CWD /mnt/mmc/_daq2Data
02-23 09:32:21.711 INFO  FtpLogger Response: 250 Command forbidden [1ms]
02-23 09:32:21.711 INFO  FtpLogger Command:  PWD
02-23 09:32:21.711 DEBUG FtpLogger Waiting for response to: PWD
02-23 09:32:21.716 INFO  FtpLogger Response: 257 "/mnt/mmc/_daq2Data" is current directory. [5ms]
02-23 09:32:21.716 INFO  FtpLogger
02-23 09:32:21.716 INFO  FtpLogger Executing stashed command
02-23 09:32:21.716 INFO  FtpLogger
02-23 09:32:21.716 INFO  FtpLogger Command:  XDAQ daq status
02-23 09:32:21.716 DEBUG FtpLogger Waiting for response to: XDAQ daq status

And here we got stuck.
What is SslSessionLength? What is stashed command?
I will rollback to v 36 or 40 or switch to a simpler library, FluentFTP is doing great but for a custom system like this all the evolutions you are working on risk to break. Please, could you confirm that 36 or 40 are versions that do not contain the connection block hang?
Thanks
A

@Adhara3
Let's go through this post of yours step by step:

it happens again. I don't know if that's the same, but here we go.

It is not the same, first of all, so it certainly would merit a separate new issue. A pity you just hung this one on here without further analysis.

The status command before the disconnection failed "gracefully"

You got a System.TimeoutException. Was it not possible to catch that and discontinue operations on that server?

then

Well, the automatic reconnect logic in Execute has been in there for ages. It now actually works, whereas it didn't work before. For example, it didn't restore the working directory after reconnecting. It failed entirely on encrypted connections (not your issue, I know), etc.
The automatic reconnect will attempt to "reissue" the failing command (aka "stashed" command). On the way there, it will attempt to relogon, restore the path and the redo the command (i.e. the "remembered" command, aka the "stashed" command).
You can see all this clearly in the log.
Intersting, but probably not important, is the following observation: This server does not accept a CWD command (to get back to the previous directory. The subsequent PWD command shows that the current directory is the one that is wanted, so that failure had no adverse effect (one would hope that the server is gracious).
After all the previous commands have worked fine and gotten good responses only Deity himself knows why this server is not responding to a "XDAQ daq status" command. It is not responding. The socket is still connected, but it is not blocking and receives null data.

And here we got stuck.

It is a known bug of ours, and it is being worked on right now, that we do not time out correctly in this situation. The fix for this is currently being tested and looks good. You will get a System.TimeoutException once that fix finishes final testing and gets merged.
Although in your code you seem to ignore these exceptions and just continue onwards (Is that correct?). That is not a problem, as then the automatic reconnect will kick in and things will restart, again, again and again, I suppose, until this little server will respond correctly.

What is SslSessionLength?

I will suppress reporting the current SslSessionLenght counter on connections that are not encrypted, to avoid confusion on your side. The value is shown to aid determining at what point the session was reconnected.

What is stashed command?

See the above explanation of automatic reconnect.

I will rollback to v 36 or 40 or switch to a simpler library

You had a version that worked for you?
Perhaps a simple server  actually needs a simple library :-)

Please, could you confirm that 36 or 40 are versions that do not contain the connection block hang?

No. 36 and 40 were versions that had their own set of problems and if you are honest, you will accept that the set of possible current problems is smaller in magnitude that the one in the past.

I will rollback to v 36 or 40 or switch to a simpler library, FluentFTP is doing great but for a custom system like this all the evolutions you are working on risk to break. Please, could you confirm that 36 or 40 are versions that do not contain the connection block hang?

If you are using it for such a mission critical purpose, then maybe better to fork and use your own version.
"Risk to break" is really funny, I mean this is free and open source. @FanDjango  has also worked excessively hard (too much, really) to make the recent versions more reliable. If you don't feel the newer version is reliable then you are free to use any older version.
If the direction of the project does not suit you, you are always free fork and handle further development yourself.
If you use the latest version, you are subject to (breaking) changes, which are inevitable, as is the process of everlasting growth.

Foreword
Guys @FanDjango and @robinrodricks ,
please do not misunderstand me. As I said I appreciate and support all the efforts you do for this library. I am perfectly aware that this is an open source project and it has the right to fail, add regressions etc. It's all perfectly fine and I am in no way whatsoever criticizing you, the project or what you are doing or the directions you are taking. Really.
I am also aware that my case is a special one and while FluentFTP main goal is to match as many (standard) server types as possible and face issues as proxies and encryption I am working with an unpolite embedded system not designed to be an FTP server. And this is my responsibility.
The reason why I tried to stick with latest version and not forking, is because it contains bug fixes and because the basic file operations are more or less working fine in my environment (i.e. my environment is kind enough to behave quite in standard way when it come to file management).
So bottom line: I'm not asking or expecting anything from you, except some help to find a safe place where to land which could be an older version, another library or some bug fixing. But I really do come in peace.
Why FluentFTP
My code was initially using an edited copy of .NET Framework System.Net.FtpClient but I wanted to use something community supported, less customized, etc. This is why I switched to FluentFTP, 2 years ago, maybe more. I was aware of all the pros and cons of going with a library, including regressions etc. because I am old enogh to know that this is part of the game. But my belief was that I did not want to reinvent the wheel (FTP) and that you guys are more experienced than I am on this topic.
The usage is quite simple but the server is not stable, meaning that by design it goes offline then reappears.
The whole point here is: do I expect FluentFTP to work is such a weird scenario? Do I expect you to solve my architectural issues? No, to both questions.
I am here only because I would expect to get a timeout and my thread is stuck instead. Maybe I was just missing a configuration setting, I don't know.
Some context

You got a System.TimeoutException. Was it not possible to catch that and discontinue operations on that server?

No. The status request is issued every second so that I can spot the system status and react. I stop polling only if network goes down (i.e. ping) but a failed status may be just a reboot, so a temporary fail.
The pattern is broken, I am aware of that. I should use FTP for file stuff, I should not poll. But that's what I have

Intersting, but probably not important, is the following observation: This server does not accept a CWD command (to get back to the previous directory. The subsequent PWD command shows that the current directory is the one that is wanted, so that failure had no adverse effect (one would hope that the server is gracious).

The reason is that they jailed users in specific directories, as soon as I log the server puts me in that dir and I cannot move from ther (with this user). I have no power on this, that-'s how it's working.

It is a known bug of ours, and it is being worked on right now, that we do not time out correctly in this situation. The fix for this is currently being tested and looks good. You will get a System.TimeoutException once that fix finishes final testing and gets merged.

I though that this was the original issue with the GetReply, this is why I posted here instead of opening another separate issue. If I was wrong about it, I apologize, I really thought that here too I was entering the infinite loop.

You had a version that worked for you?

It's not so easy. I had a version that was working with a specific server version back in december. Since then, they updated server software and they made some changes that probably broke. Of course, this is not your fault and I am not expecting you to fix it. The new server software is doing dirty tricks that trigger this infinite loop&hang situation that would occur also with previous FluentFTP versions. This is why I was asking when the infinite loop regression was added, not to criticize the project, but to understand which versions I could test. Going back to version 40 or 36 is not free at all because of API changes. So again, I was just looking for an advice.

Perhaps a simple server actually needs a simple library :-)

This is not a simple server, but a bad working one in a bad architecture. :-(

36 and 40 were versions that had their own set of problems and if you are honest, you will accept that the set of possible current problems is smaller in magnitude that the one in the past.

Overall, the library improved a lot in terms of API and design and this is why I tried to stick to latest versions, because I do recognize there is a huge effort and value. Nevertheless, the hang stuff that I am experiencing right now is deadly. And again, I am not saying that the problem is FluentFTP, the context is probably triggering this issue now. But I need to come up with a solution, any solution. So if the library has an issue, I'm just providing some feedback not urging any fix or action, just asking for advice on how to solve or mitigate.
I hope this clarifies a bit
Thanks
A

I am here only because I would expect to get a timeout and my thread is stuck instead

You had a timeout. But I agree, you should be getting another one, and another one and so on. WE HAVE BUG and we are fixing it

The status request is issued every second so that I can spot the system status and react. I stop polling only if network goes down (i.e. ping) but a failed status may be just a reboot, so a temporary fail.
The pattern is broken, I am aware of that. I should use FTP for file stuff, I should not poll. But that's what I have

My turn to say no :-) I think it is perfectly ok to poll like that if there is no other way.
The real question is how to avoid a hang so that your program can muddle along doing whatever it does.
As I said, in edge cases such as this one, we still have a possibility for a hang, I have found it and we have a different issue currently open where it happens. That guy is happy, it works. It does not solve the real problem, in that the connection fails, but it does not hang. Let's just solve it in your environment too.
To do that, I need feedback on the mentioned branch. We cannot just put tentative fixes in there and put them out to Nuget, even as Beta, it is just too much work, unless the issue is totally clear.
Looking at GitHub and past versions can be time consuming and tedious - I have made some spot checks and I believe in the case we have here, previous versions would also fail you - many other users do actually go to the trouble of checking that sort of thing by bisecting. What is stopping you? I can't do it, unless I have access to that server.

And now I would like to get back to work...


InputAccessoryView not working in react-native 0.55

None of the provided examples related to InputAccessoryView feature in 0.55 are not working.
Environment
Environment:
OS: macOS High Sierra 10.13.4
Node: 8.5.0
Yarn: 1.5.1
npm: 5.3.0
Watchman: 4.9.0
Xcode: Xcode 9.3 Build version 9E145
Android Studio: 2.3 AI-162.4069837
Packages: (wanted => installed)
react: 16.3.1 => 16.3.1
react-native: 0.55.3 => 0.55.3
Steps to Reproduce

react-native init sample
copy and paste the code from either here or here
run the code

Expected Behavior
Should work and not presented with red screen
Actual Behavior
provided an error in red screen as follows

I had to use:
const InputAccessoryView = require('InputAccessoryView')
Rather than:
import { InputAccessoryView} from 'react-native'

Thanks. Closing this issue.

@alinz Having a workaround doesn't mean the issue is resolved. The underlying issue is that InputAccessoryView hasn't been made available as an export on the main 'react-native' module, this contradicts the documentation about how to use it.

Looks like it's fixed in master though! 🎉


[Enhancement] Add multiple results per Submission

Checklist

[ ] I tested all changes and all related features with different users (student, tutor, instructor, admin) on the test server https://artemistest.ase.in.tum.de.
[x] Server: I followed the coding and design guidelines.
[x] Server: I implemented the changes with a good performance and prevented too many database calls
[x] Server: I documented the Java code using JavaDoc style.

Motivation and Context
There is currently no supported way to allow for a second correction, see #2015. As a first step towards this goal, this PR aims to change the relation between the Submission and Result from:

to:

Note: This PR only includes server changes.
Description

Change relation between Submission and Result from @OneToOne to @OneToMany.

Add results_order column to Result to track the order of the results. Because of this change we also had to follow the save procedure described in https://artemis-platform.readthedocs.io/en/latest/dev/guidelines/database/#ordered

Add 0 to all existing results in their results_order column for backwards compatibility

Perform the necessary code changes across the application to support the new relation

Add custom serialization and deserialization tag for Jackson to serialize to a single result (latestResult) for the client, and to multiple results for the server. This is a temporary solution until we also change the relationship on the client.

Steps for Testing
There are no functional changes. The goal is to have everything working as usual. Therefore, we must test every aspect of the application that was affected by these changes. These include:

Creating Text-, Modeling-, File Upload-, Quiz-, and Programming Exercises

Add Example Submission with solution

Creating Exam

With all exercise types

Participating in Text-, Modeling-, File Upload-, Quiz-, and Programming Exercises

Configure automatic programming assessment

Participating in Exam

Conducting a Test Run

Participate

Assessing Text-, Modeling-, File Upload-, Quiz-, and Programming Exercises

Locking
Overriding
Submitting
Also for exam exercises

Complaining about the Assessment

Complaint
More Feedback Request

Reviewing Complaint

Accepting Complaint
Rejecting Complaint

Test Coverage
There are no changes on the test coverage.

I got the following error when trying to edit a modeling exercise that has an example submission attached:

https://artemistest.ase.in.tum.de/course-management/331/modeling-exercises
detail":"Could not write JSON: failed to lazily initialize a collection of role: de.tum.in.www1.artemis.domain.Submission.results, could not initialize proxy - no Session; nested exception is com.fasterxml.jackson.databind.JsonMappingException: failed to lazily initialize a collection of role: de.tum.in.www1.artemis.domain.Submission.results, could not initialize proxy - no Session (through reference chain: de.tum.in.www1.artemis.domain.modeling.ModelingExercise[\"exampleSubmissions\"]->org.hibernate.collection.internal.PersistentSet[0]->de.tum.in.www1.artemis.domain.ExampleSubmission[\"submission\"]->de.tum.in.www1.artemis.domain.modeling.ModelingSubmission[\"result\"])","path":"/api/modeling-exercises/4837","message":"error.http.500"}

Got the following error upon reloading the quiz page after the timer ran out:

I can not create example submissions for text exercises
https://artemistest.ase.in.tum.de/course-management/331/text-exercises/4845/example-submissions/274

{"type":"https://www.jhipster.tech/problem/problem-with-message","title":"Internal Server Error","status":500,"detail":"Cannot invoke "de.tum.in.www1.artemis.domain.Exercise.getCourseViaExerciseGroupOrCourseMember()" because "exercise" is null","path":"/api/exercises/4845/example-submissions","message":"error.http.500"}

When clicking the red refresh button:

Probably unrelated:

After failing a build the build logs are only shown in the build log output. The result should also show Build Failed and make the Build Logs available

Only for programming exam exercises:
{"type":"https://www.jhipster.tech/problem/problem-with-message","title":"Internal Server Error","status":500,"detail":"failed to lazily initialize a collection of role: de.tum.in.www1.artemis.domain.Result.feedbacks, could not initialize proxy - no Session","path":"/api/exercises/4846/programming-submission-without-assessment","message":"error.http.500"}

Also probably unrelated. This is for programming exercises with manual assessment:

Submit code that builds for a programming exercise in time
Submit code that fails to build in time

Course dashboard

Inside editor
It seems like the result of the failed build is ungraded, even though it was submitted in time.

The "Scores" view for the exams is not updated properly.

I got the following error when trying to edit a modeling exercise that has an example submission attached:

https://artemistest.ase.in.tum.de/course-management/331/modeling-exercises
detail":"Could not write JSON: failed to lazily initialize a collection of role: de.tum.in.www1.artemis.domain.Submission.results, could not initialize proxy - no Session; nested exception is com.fasterxml.jackson.databind.JsonMappingException: failed to lazily initialize a collection of role: de.tum.in.www1.artemis.domain.Submission.results, could not initialize proxy - no Session (through reference chain: de.tum.in.www1.artemis.domain.modeling.ModelingExercise[\"exampleSubmissions\"]->org.hibernate.collection.internal.PersistentSet[0]->de.tum.in.www1.artemis.domain.ExampleSubmission[\"submission\"]->de.tum.in.www1.artemis.domain.modeling.ModelingSubmission[\"result\"])","path":"/api/modeling-exercises/4837","message":"error.http.500"}

Should be fixed after 11d67bf231a09b8c0f741f3512b1458c770813a3

The "Scores" view for the exams is not updated properly.

I analyzed this, this is actually not a bug. This happens because we filter for student participations only, and ignore instructors/administrator participations. This is because in reality users with instructor capabilities should not be able to participate in the exam. Instructors can only participate in the exam as part of test runs, the scores of which are excluded from the scores page. As you all participated with your admin accounts, your scores were disregarded in the scores page.
See:

I think long term we should not allow users to be registered to exams which have instructor or tutor access level in order to make sure that more people dont stumble on this behavior.

@kloessst @FuchsDominik we should have now fixed the issues that you guys found during the testing session today. We can retest tomorrow all fixes in a testing session again :)

Issues found so for:
You cannot save a sample submission for modeling exercise (click save after everything was created)
question answers are not displayed even though correctly answered

Tested on TS1. Everything worked as intended except the example submissions. During assessing the example submission there is an issue if you set field to 0 Points you cannot save your example submission. You get a translation not found exception.

The only way to save your submission is when you put another value !=0 in the fields.

Fixed in the last commit

Tested in testing session. A bug with sample assessments where an error happens when updating the assessment(e.g. adding an explanation).

Fixed in ac0e1f6


How to use the sendMediaGroup method with local media (and not URL)?

Hello,
I am having issues using the sendMediaGroup command, when I try to send album of "local" pictures (not from URL).
Documentation says to use the "attach" syntax ("attach://<file_attach_name>"), however after multiple tries, I am unable to get this working.
I am currently using a workaround by sending the pictures one by one, however sending all pictures in album would be better.
Could you explain how to use the sendMediaGroup command with local media?
thanks

Hi @TheSleepySlee
The attach:// method requires the key name of the multipart form that gets sent to Telegram, not the file name/path.
This is as easy as adding custom entries to the $data array being passed to Request::sendMediaGroup(...);. So all you need to do, is add custom elements that contain the file handlers and reference them :+1: Same thing applies for videos too!
(Be sure to not use any key that may be reserved by Telegram. It's very unlikely, but just check the documentation to be safe.)
Request::sendMediaGroup([
    'chat_id'  => $chat_id,
    'photo_1'  => Request::encodeFile('/path/to/some/photo.png'),
    'photo_2'  => Request::encodeFile('/path/to/another/photo.png'),
    'my_video' => Request::encodeFile('/path/to/my/video.mp4'),
    'media'    => [
        new InputMediaPhoto(['media' => 'attach://photo_1']),
        new InputMediaVideo(['media' => 'attach://my_video']),
        new InputMediaPhoto(['media' => 'attach://photo_2']),
    ],
]);

Thanks a lot. Clearly I am not an expert with this library.
Here is a working example, if it can help others.
pic1/2/3/4.jpg are located in same location where PHP file runs.
`
 $chat_id,
    'photo_1'  => Request::encodeFile('pic1.jpg'),
    'photo_2'  => Request::encodeFile('pic2.jpg'),
    'photo_3'  => Request::encodeFile('pic3.jpg'),
    'photo_4'  => Request::encodeFile('pic4.jpg'),
    'media'    => [
        new InputMediaPhoto(['media' => 'attach://photo_1']),
        new InputMediaPhoto(['media' => 'attach://photo_2']),
        new InputMediaPhoto(['media' => 'attach://photo_3']),
        new InputMediaPhoto(['media' => 'attach://photo_4']),
    ],
]);

?>
`

On my machine running (PHP 7.4) @TheSleepySlee code produces warnings:
[04-Jan-2020 15:18:33 UTC] PHP Warning:  file_exists(): Unable to find the wrapper "attach" - did you forget to enable it when you configured PHP? in /var/www/html/telegram/vendor/longman/telegram-bot/src/Request.php on line 443

It works, but by simply doing the warning is gone:
php
Request::sendMediaGroup([
    'chat_id'  => $chat_id,
    'media'    => [
        new InputMediaPhoto(['media' => '/path/to/photo_1']),
        new InputMediaPhoto(['media' => '/path/to/photo_2']),
        new InputMediaPhoto(['media' => '/path/to/photo_3']),
        new InputMediaPhoto(['media' => '/path/to/photo_4']),
    ],
]);


Use the installed version of wasm-opt by default

https://github.com/WebAssembly/binaryen's wasm-opt is installed on my machine, so it'd be great to use it with wasm-pack. Unfortunately, /usr/binwasm-opt --version fails with:
wasm-opt (unable to determine version)

Not sure how we can get around that. This tool has so many problems unfortunately.

I believe binaryen fixed this issue in https://github.com/WebAssembly/binaryen/commit/4953bcc59085c49343c334ccc852fc5c0339c938, but sadly they have not pushed a release, so some distros now seem to live with a broken build.

@attilaolah thanks for the heads up! i'll circle up with the binaryen team and see when they plan their next release!

Upgrading binaryen to 90-2 fixed this for me.


Horadric cube

Recipes:

2 oils => 1 oil same rarity
4 oils => 1 oil higher rarity
3 items => 1 item same rarity
5 items => 1 item higher rarity

Oils and items placed into the cube must be of same rarity.
Note that transmute obviously can't go above the unique rarity. Probably disable recipes for such cases?

Closed by 97418e3bc9999d863920c4cbb31069340735464d


Evaluation and performance

Hi,
you got some great numbers, but I have several questions regarding the comparison to Monodepth2 and Depth-VO-Feat (Zhan et al).
First of all, your depth and feature encoders are ResNet50 with input resolution of 320×1024, right? Why do you then compare to Monodepth2 with ResNet18 encoder in Table 2? Table 6 of Monodepth2 shows that changing encoder from ResNet18 to ResNet50 reduces RMSE by more than 0.22 even with lower resolution input. Is there any performance gain when compared to Monodepth2 in a fair way (same encoder, same resolution)?
Depth-VO-Feat introduce a very similar idea, but back then it was implemented using a significantly weaker baseline. How does FeatDepth compare to Depth-VO-Feat using a better reconstruction loss (e.g. from Monodepth2)?

Since Monodepth2 and Depth-VO-Feat are only 2 models from 20+ our baselines, there are plenty of other methods which use high resolution inputs and big models. For example, depthhint can be considered as an improved version of Monodepth2.

I am afraid that is not quite what I asked. Fair comparison to Monodepth2 is important for 2 reasons:

It is one of the best performing methods trained on monocular videos;
Your model is basically Monodepth2 + proposed improvement, so fair comparison is essential to see the effect of the proposed method.
I don't want to sound rude, but from the point of view of experienced reader, your performance gain seems to be the result of a bigger encoder. I am really curious why the reviewers at ECCV did not insist on the comparison to Monodepth2 with ResNet50 and 320x1024 input resolution.

As for Depth-Hints, I don't know what variant of Depth-Hints you compare to, since the numbers in your paper are different from the reported in the original paper. They report the following numbers for ResNet18 320x1024 MS + post-processing model:
Abs Rel    Sq Rel    RMSE    RMSE log    δ<1.25    δ<1.25^2    δ<1.25^3
0.098        0.702      4.398     0.183           0.887      0.963          0.983
You report the following numbers for their method with pp:
0.100        0.728      4.469     0.185           0.885      0.962          0.982
Looking forward to your response to the question about Monodepth2.

1、I think our ablation study can clearify this question, the base model without any improvement also got a very good performance, which is much higher than monodepth2. You can consider this gain comes from a bigger backbone and higher input resolutions (although we use a different architecture for depthdecoder). It not easy to make further imrpovement on such a high performance base model, but our technique really did.
2、The numbers of DepthHint in our paper is from table2 row 10 in its paper, the numbers you refer to is not using KITTI dataset. You can see that depthhint already use ResNet50 and 320*1024 input resolution (in table 3 with pp), so I afraid it is no need to make extra experiments.
I hope this explanation can help, looking forward to further discussion.

Unfortunately, your explanation only raises more questions. Anyway, thank you for the effort.
I do not see any point in further discussion, since our perception of what research is seems to be very different.

Also very currious about how FeatDepth performs with ResNet 18 on 640x192 input :) Wondering if you could provide the weight?

Sorry, I don't maintain pretrained weights of this settings.

I wonder if the reported performance choice the best weights from different epochs in training phase according to the loss on the test set? As I konw, monodepth2 used the last epoch for their reported results. @sconlyshootery @Yevkuzn


Customize colors of 2D plots in H5Web cell

H5Web allows customizing the colors in 2D plots, which is done in the css. It would be great to have a way to be able to inject this to invoking these widgets in Jupyter lab.
Originally posted by @tomio13 in https://github.com/silx-kit/jupyterlab-h5web/issues/34#issuecomment-1041231232

Hi @tomio13, could you please tell us more about what you're doing with H5Web? How you're using it (in a cell or in a tab), with what kind of data, etc.

Thank you for the fast response.
In one case we are using it within a jupyter lab notebook to visualize NEXUS data output files with several curves to be plotted together. The specific example I used was an ellipsometry curve containing 3 Psi and 3 Delta curves each at a different incident angles.

Okay, thanks, and so I take it from your issue in the H5Web repo https://github.com/silx-kit/h5web/issues/970 that you'd like the curves to be more easily identifiable and distinguishable from one another? Is this just a matter of changing the extension's default auxiliary curve colours so they're not all the same, or do you need actual control over the colours every time your run the H5Web widget?

Yes. As I see, the story has two aspects: on the one hand just changing the style such that a series of colors would be used is a good starting point. On the other hand, giving the possibility to set this by the user e.g. through setting an optional class, would be a really nice solution.

May be linked to https://github.com/silx-kit/h5web/issues/1209


Fix handling of spaces after for as

Fix https://github.com/roc-lang/roc/issues/6521

Oops, tests fixed


[Question] Can only add one custom header?

var baseConfig = {
baseURL: 'xxxx',
headers: {
'Origin': '*',
'access_token': 'xxxx',
}
};
axiosInstance = axios.create(baseConfig);
I added two custom headers, captured the request by Fiddler, only the "Origin" header in request header, but "access_token" is in Access-Control-Request-Headers: accept, access_token, and got error from server showed that "The collection of headers 'accept,access_token' is not allowed".
Does Axios only support to add one custom header?

Axios allows you to add as many custom headers as you need. When a cross-origin request is made with custom headers, the client will send the so-called preflight request (with the OPTIONS method) first. You should see it in Fiddler. The server needs to respond with Access-Control-Allow-Headers that lists your custom headers:
Access-Control-Allow-Headers: access_token, <other custom headers>

The client will then make the actual request.
This is explained in great detail here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
Hope this helps!

How does this work if you're authenticating using a custom header?

^ Same question here.
I am using the following headers to make a PATCH request:
headers = {'Content-Type': 'application/json', 'Authorization': 'Token <my_token'}

Yet Axios doesn't seem to recognize the Authorization header....

@BigChief45
try the following way:
axios.defaults.headers.Authorization = Token <my_token>

@liukan0211 Thanks! That seems to work. But it seems odd, why does it have to be set like this? It is not how it is described in the documentation.
I've read through a lot of similar issues about this. Some people keep saying it is not a bug, but it certainly looks like a bug to me.

@BigChief45
I reviewed the source code and found that axios will ignore the Authorization header if it's set in custom headers, you can refer to https://github.com/axios/axios/blob/master/lib/helpers/parseHeaders.js.

@liukan0211 Thanks for the help. Unfortunately I kept experiencing more problems with Axios, so I switched to got, works like a charm!


[bug] Bots don't work anymore

The bots on this project are not doing anything anymore. The gitconsensus bot doesn't mark or merge any pull requests. ScreepsAutoSpawner doesn't execute a respawn. The official Quorum bot has been out of rooms for days.

this was already identified to the host 2 days ago. he's busy irl, but he said he'd take a look at it when he could...

I'll leave this issue open. Information on this can be posted here.

Github changed the response to some of the API calls and this broke the github python client I was using. Upgrading the library resolved that problem.
Still need to look at why the autospawner isn't working through.

Quorum is here again so I'm closing this.


SSL support should be provided to support HTTPS based clients

Log traces below for https based client,
>>> print(client._test_connection())
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 344, in connect
    ssl_context=context)
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 347, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 412, in wrap_socket
    session=session
  File "/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 853, in _create
    self.do_handshake()
  File "/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)```

@cfournie : I have faced this issue once when I was using requests package. I believe providing an option to pass the cert parameter would work in this case.
https://2.python-requests.org/en/master/user/advanced/#client-side-certificates


Add A Footer Section

Footer Section Updated
Related Issue

Mention issue #265   for which this pull request is a part (please mention issue number # )

Proposed Changes
The UI part of the footer section
Additional Info
Make it easier for the user to connect to the website through official social media handles
Checklist

[x] ✅ My code follows the code style of this project.
[x] 📝 My change requires a change to the documentation.
[x] 🎀 I have updated the documentation accordingly.
[x] 🌟 ed the repo

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

I've redesigned the footer Icons, Added hover animations to them, and Improved the back button design.
Output Screenshots
Before

After

Great work
Did you checked for responsiveness ?
also, please make target="_blank" for all the links
rest all good

Yes, I have checked the responsiveness. It is responsiveness.
also, I have updated all the links used in the footer section and made all the links target blank you can check
https://deploy-preview-291--codesetgo.netlify.app/


API return null

Bug report
Description
All the endpoints of the API return null
Expected behavior
in the result parameter should be the response to the command
Current behavior
the result parameter contains null as value
Steps to reproduce
curl -X POST -d '' http://<IP_ADDRESS>:1242/Api/Command/version
curl -X POST -d '' http://<IP_ADDRESS>:1242/Api/Command/status
curl -X POST -d '' http://<IP_ADDRESS>:1242/Api/Command/2fa {bot name}
Full log.txt recorded during reproducing the problem
2018-08-12 23:39:43|ArchiSteamFarm-28737|INFO|ASF|InitASF() ArchiSteamFarm V<IP_ADDRESS> (linux-x64/61c03fef-7e4e-4e04-abbf-00d089ff014c | Linux 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u3 (2017-08-15))
2018-08-12 23:39:44|ArchiSteamFarm-28737|INFO|ASF|Start() Starting IPC server on http://************************/...
2018-08-12 23:39:44|ArchiSteamFarm-28737|INFO|ASF|Start() IPC server ready!
2018-08-12 23:39:44|ArchiSteamFarm-28737|INFO|ASF|CheckAndUpdateProgram() ASF will automatically check for new versions every 1 day.
2018-08-12 23:39:44|ArchiSteamFarm-28737|INFO|ASF|CheckAndUpdateProgram() Checking for new version...
2018-08-12 23:39:46|ArchiSteamFarm-28737|INFO|ASF|CheckAndUpdateProgram() Local version: <IP_ADDRESS> | Remote version: <IP_ADDRESS>
2018-08-12 23:39:46|ArchiSteamFarm-28737|INFO|Gaben|Start() Starting...
2018-08-12 23:39:46|ArchiSteamFarm-28737|INFO|Gaben|Connect() Connecting...
2018-08-12 23:39:47|ArchiSteamFarm-28737|INFO|TMarmol|Start() Starting...
2018-08-12 23:39:47|ArchiSteamFarm-28737|INFO|Principal|Start() Starting...
2018-08-12 23:39:47|ArchiSteamFarm-28737|INFO|mrmarblet4|InitStart() Not starting this bot instance because it's disabled in config file!
2018-08-12 23:39:47|ArchiSteamFarm-28737|INFO|mrmarblet3|InitStart() Not starting this bot instance because it's disabled in config file!
2018-08-12 23:39:47|ArchiSteamFarm-28737|INFO|mrmarblet2|InitStart() Not starting this bot instance because it's disabled in config file!
2018-08-12 23:39:47|ArchiSteamFarm-28737|INFO|mrmarblet1|InitStart() Not starting this bot instance because it's disabled in config file!
2018-08-12 23:39:47|ArchiSteamFarm-28737|INFO|Gaben|OnConnected() Connected to Steam!
2018-08-12 23:39:47|ArchiSteamFarm-28737|INFO|Gaben|OnConnected() Logging in...
2018-08-12 23:39:49|ArchiSteamFarm-28737|INFO|Gaben|OnLoggedOn() Successfully logged on as ******************************.
2018-08-12 23:39:49|ArchiSteamFarm-28737|INFO|Gaben|Init() Logging in to ISteamUserAuth...
2018-08-12 23:39:50|ArchiSteamFarm-28737|INFO|Gaben|Init() Success!
2018-08-12 23:39:50|ArchiSteamFarm-28737|INFO|Gaben|IsAnythingToFarm() Checking first badge page...
2018-08-12 23:39:52|ArchiSteamFarm-28737|INFO|Gaben|StartFarming() We don't have anything to idle on this account!
2018-08-12 23:39:56|ArchiSteamFarm-28737|INFO|TMarmol|Connect() Connecting...
2018-08-12 23:39:56|ArchiSteamFarm-28737|INFO|TMarmol|OnConnected() Connected to Steam!
2018-08-12 23:39:56|ArchiSteamFarm-28737|INFO|TMarmol|OnConnected() Logging in...
2018-08-12 23:39:56|ArchiSteamFarm-28737|INFO|TMarmol|OnLoggedOn() Successfully logged on as **********************************.
2018-08-12 23:39:56|ArchiSteamFarm-28737|INFO|TMarmol|Init() Logging in to ISteamUserAuth...
2018-08-12 23:39:57|ArchiSteamFarm-28737|INFO|TMarmol|Init() Success!
2018-08-12 23:39:57|ArchiSteamFarm-28737|INFO|TMarmol|IsAnythingToFarm() Checking first badge page...
2018-08-12 23:39:59|ArchiSteamFarm-28737|INFO|TMarmol|StartFarming() We don't have anything to idle on this account!
2018-08-12 23:40:05|ArchiSteamFarm-28737|INFO|Principal|Connect() Connecting...
2018-08-12 23:40:05|ArchiSteamFarm-28737|INFO|Principal|OnConnected() Connected to Steam!
2018-08-12 23:40:05|ArchiSteamFarm-28737|INFO|Principal|OnConnected() Logging in...
2018-08-12 23:40:05|ArchiSteamFarm-28737|INFO|Principal|OnLoggedOn() Successfully logged on as **************************.
2018-08-12 23:40:05|ArchiSteamFarm-28737|INFO|Principal|Init() Logging in to ISteamUserAuth...
2018-08-12 23:40:06|ArchiSteamFarm-28737|INFO|Principal|Init() Success!
2018-08-12 23:40:06|ArchiSteamFarm-28737|INFO|Principal|IsAnythingToFarm() Checking first badge page...
2018-08-12 23:40:08|ArchiSteamFarm-28737|INFO|Principal|IsAnythingToFarm() Checking other badge pages...
2018-08-12 23:40:10|ArchiSteamFarm-28737|INFO|Principal|StartFarming() We don't have anything to idle on this account!
2018-08-12 23:40:40|ArchiSteamFarm-28737|ERROR|Gaben|Response() steamID || message is null!
2018-08-12 23:41:24|ArchiSteamFarm-28737|ERROR|Gaben|Response() steamID || message is null!
2018-08-12 23:41:35|ArchiSteamFarm-28737|ERROR|Gaben|Response() steamID || message is null!
2018-08-12 23:41:57|ArchiSteamFarm-28737|ERROR|Gaben|Response() steamID || message is null!
2018-08-12 23:42:00|ArchiSteamFarm-28737|ERROR|Gaben|Response() steamID || message is null!
2018-08-12 23:42:15|ArchiSteamFarm-28737|ERROR|Gaben|Response() steamID || message is null!
2018-08-12 23:44:39|ArchiSteamFarm-28737|ERROR|Gaben|Response() steamID || message is null!

Global ASF.json config
{
  "AutoRestart": false,
  "CurrentCulture": "en-US",
  "Headless": true,
  "IdleFarmingPeriod": 5,
  "IPC": true,
  "IPCPassword": "***********************",
  "IPCPrefixes": [
    "http://*******************/"
  ],
  "LoginLimiterDelay": 9,
  "UpdateChannel": 2
}

BotName.json config of all affected bot instances (if more than one)
{
  "AcceptGifts": true,
  "AutoSteamSaleEvent": true,
  "CustomGamePlayedWhileFarming": null,
  "CustomGamePlayedWhileIdle": null,
  "Enabled": true,
  "FarmingOrders": [
    0
  ],
  "GamesPlayedWhileIdle": [],
  "HandleOfflineMessages": false,
  "HoursUntilCardDrops": 1,
  "IdlePriorityQueueOnly": false,
  "IdleRefundableGames": true,
  "LootableTypes": [
    0,
    1,
    2,
    3,
    4,
    5,
    6
  ],
  "MatchableTypes": [
    0,
    1,
    2,
    3,
    4,
    5
  ],
  "OnlineStatus": 0,
  "PasswordFormat": 0,
  "Paused": false,
  "RedeemingPreferences": 1,
  "SendOnFarmingFinished": true,
  "SendTradePeriod": 0,
  "ShutdownOnFarmingFinished": false,
  "SteamTradeToken": "*****************",
  "SteamUserPermissions": {
    "************************": 3
  },
  "TradingPreferences": 1,
  "UseLoginKeys": true,
  "BotBehaviour": 8,
  "SteamLogin": "**************",
  "SteamMasterClanID": ++++++++++++++++++++++,
  "SteamParentalPIN": "0",
  "SteamPassword": "****************",
}

Additional info

Thanks to the commit I saw that I needed the SteamOwnerId in the configuration, There's a bug in the IPC GUI, it is using the s_SteamOwnerId instead of SteamOwnerId that's why it deleted mine, I'll make a PR


Add example for OpenTelemetry Tracer

Commit Message: Add example for OpenTelemetry Tracer
Additional Description: This is the Jaeger Tracing demo, but with OpenTelemetry tracing instead. Like the Jaeger Tracing demo, it pulls up 3 Envoy instances with OTLP tracing enabled, then also runs the OpenTelemetry Collector.
Risk Level: Very low (just an example)
Testing: Added verify.sh
Docs Changes: Added README.md here, but happy to add docs as well
Part of https://github.com/envoyproxy/envoy/issues/9958

hi @AlexanderEllis thans for working on this - very useful - headsup that these sandboxes can take quite a bit of back and forth
we will definitely need some rst docs for this - im happy to help putting these together
i see this is WIP still - do you want me to give it a first pass now ?

Hi @phlax , sounds good to me! Last thing I'm fixing is just ensuring the verify script passes in the pipeline as well — I think I was running into permissions issues with the yaml config for the collector, but I've added a bit to (hopefully) fix that part.
A first pass would be great! I'll mark this as review, and happy to iterate as we go :) Thanks!

@phlax Still running into trouble with the verify script in the CI pipeline, though things are working locally 🤔  I'm currently seeing the following:
opentelemetry_collector_1  | Error: failed to get config: cannot resolve the configuration: unable to read the file file:/etc/otel-collector-config.yaml: open /etc/otel-collector-config.yaml: permission denied
opentelemetry_collector_1  | 2022/08/18 20:03:29 collector server run finished with error: failed to get config: cannot resolve the configuration: unable to read the file file:/etc/otel-collector-config.yaml: open /etc/otel-collector-config.yaml: permission denied

I think I'm adding this file correctly in the Dockerfile:
  opentelemetry_collector:
    build:
      context: .
      dockerfile: Dockerfile-opentelemetry-collector
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
    - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
    - "${PORT_COLLECTOR_ZPAGE:-55679}:55679"

Any suggestions on this? I'm far from a docker expert, and any help would be appreciated.

Any suggestions on this? I'm far from a docker expert, and any help would be appreciated.

i will review tomorrow - but off hand my guess is that the permissions in CI are different.
before running it the permissions/umask are restricted - in ci/do_ci.sh
 191   umask 027
 192   chmod -R o-rwx examples/

it does this to simulate an env (ie google) where home dirs are shared and perms are stricter

following up on this - im just looking at flakes in other examples
the shared/tracing Docker image is probably best avoided - its really not an example of best practice as it has 2 processes running inside the container
im just looking at how we can unpick this from existing examples

im realizing that this PR is copying out some of the issues in other tracing examples
it was on my agenda to rewrite these - esp for zipkin as it can flake - but also for the jaeger ones
im most of the way through rewriting the zipkin example - which can be seen here #22771, altho its going to take me a while to complete

zipkin sandbox update is complete - i would use that (ie in PR #22771 as it hasnt yet landed) as the basis for this sandbox
updated docs for the zipkin sandbox can be seen here https://storage.googleapis.com/envoy-pr/87b2474/docs/start/sandboxes/zipkin_tracing.html

With OpenTelemetry now being supported (thanks @AlexanderEllis!), I think we can / should deprecate the Jaeger and OpenTracing extensions and sandboxes. Discussion at https://github.com/envoyproxy/envoy/issues/9958. @cgilmour is planning to update the Datadog tracer to remove the OpenTracing dependency which will allow for its removal - https://github.com/envoyproxy/envoy/issues/21083#issuecomment-1196172567

I think we can / should deprecate the Jaeger and OpenTracing extensions and sandboxes

it would be good to clarify which sandboxes these are - there are a few other tracing sandboxes (eg skywalking, zipkin)
either way - i think the pattern that the updated zipkin sandbox uses is roughly the pattern we want for the tracing sandboxes

@AlexanderEllis was hoping to ping you on slack - but didnt find you there - so i guess ill post here
@nguyenptk has been working on a Kafka sandbox and has got a pretty good idea how the sandbox docs/verify work
im wondering if it would be helpful for him and I to take over this PR - im thinking we can use the settings you have set up here but with the zipkin example as a template
I really appreciate the work you have done/are doing on the OT extension - so happy to work on this if it is helpful

Hey @phlax , sorry for the delay here (day-job got busy), but that would be great! It sounds like the new Zipkin example would be great to reuse here, and from the sounds of it, the only differences would be 1) the OpenTelemetry Tracer config 2) seeing the traces and 3) viewing the OpenTelemetry collector page.
Happy to help with any questions about the setup 👍  Thanks again!

closing this as #22988 has now landed


does RocksDict compatible with RocksDB-Python?

Here is my code, I am trying to save some sst files and read with rocksdb in python:
from rocksdict import Rdict, Options, SstFileWriter
import random
import rocksdb

# generate some rand bytes
rand_bytes1 = [random.randbytes(200) for _ in range(100000)]
rand_bytes1.sort()

# write to file1.sst
writer = SstFileWriter(options=Options(raw_mode=True))
writer.open("file1.sst")
for k, v in zip(rand_bytes1, rand_bytes1):
    writer[k] = v

writer.finish()

# bulid Rdict
d = Rdict("tmp", options=Options(raw_mode=True))
d.ingest_external_file(["file1.sst"])
d.close()

# rocksdb read
db = rocksdb.DB("tmp", read_only=True, opts=rocksdb.Options())

And I am getting the error as below:
---------------------------------------------------------------------------
Corruption                                Traceback (most recent call last)
remote%2B<IP_ADDRESS>/opt/huawei/data1/z00811450/code/test.ipynb#Y211sdnNjb2RlLXJlbW90ZQ%3D%3D?line=22) db = rocksdb.DB("tmp", read_only=True, opts=rocksdb.Options())

File /opt/huawei/wisebrain/anaconda3/lib/python3.9/site-packages/rocksdb/_rocksdb.pyx:1636, in rocksdb._rocksdb.DB.__cinit__()

File /opt/huawei/wisebrain/anaconda3/lib/python3.9/site-packages/rocksdb/_rocksdb.pyx:76, in rocksdb._rocksdb.check_status()

Corruption: b'Corruption: unknown checksum type 4 from footer of tmp/000019.sst, while checking block at offset 22672231 size 35'

Does this mean that rocksdict is not compatible with rocksdb in python? Looking forward to anyone's help!

rocksdict uses rocksdb 8.0 for the moment. So it maybe incompatible to rocksdb 6 which is used by python-rocksdb

But it should be compatible with other rocksdb clients, e.g., java, c++, rust


Investigate File > Open Folder experiences

As we investigate the File > Open Folder experiences, we should keep in mind the following open Qs
Extensibility

[ ] How do we register as an File > Open provider?

Performance

[ ] Why does it take so long to open a folder?
[ ] is there any way to mitigate this issue?

Project System

[ ] how to determine File > Open Folder project context?
[ ] What areas does our current featureset depend on the existing project system? For each of these areas, how do we extend File > Open Folder to make it happen?
[ ] How to determine Node.js project context?

Editing / IntelliSense

[ ] how to supply Node.js-specific tsconfig.json w/o explictly adding it to the project?

Telemetry

[ ] How to determine # Node.js projects?
[ ] Any current telemetry?

Debugging / Build

[ ] how do we set options / debug flags?

[ ] current options?
[ ] any future options we should consider?

[ ] msbuild as a "case study" - what would it take?

Roaming between VS / VSCode

[ ] what does that look like?
[ ] are there any existing efforts to bring them closer together?

implemented


response.is_valid? is return false even though email-id is fetched

i am able to fetch email from SAMLResponse from attributes by this way: response.attributes["urn:oid:0.9.2342.192003<IP_ADDRESS>"]
but when I check response.is_valid? it return false. What could be reason? how to see error message?
response.errors returns me empty. Any bug here?

Check response.errors after calling response.is_valid?. The reason for the error should be at errors attribute.

As I said, response.errors is empty

@RohitVenturit, were you able to find the issue?

i have the same issue. response.errors is empty but response.is_valid? return false. I use idp gluu server. Can you help me explain this issue?

All validations defined on is_valid? append an error when the validation fails, you will need to debug the validate method and figure out what's going on.

I'm having the same issue response.errors empty but response.is_valid? false