v1.0.0 causes crashes during execution

Ravi tests crash during execution, and I am not yet able to narrow it down. Initial investigation appears to show that if I isolate the code that fails, then it passes, but when run in the larger test program, it fails. This makes me think that its something to do with the size of the function being compiled.
I am flagging this for two reasons:

It is likely there is some bug in the code generation that needs to be fixed
I'd like some ideas on how to isolate the failure

The failure is basically causing runtime assertions to trigger in Ravi - that means some corruption somewhere.
Example of failure (there are other failures similar to this)
https://github.com/dibyendumajumdar/ravi/actions/runs/9374498129/job/25810662116

Maybe if you have the code that was working before using this mir version and dump the generated asm and comparing then can give a clue ?

The master branch runs the previous version of MIR (v0.x) which works fine.
I could try dumping out the asm for both.

I am not sure how to generate assembly output.
I generated the MIR output from the two versions, along with the C input file.
These are attached
test.mir_v1.0.txt
test.mir_v0.0.txt
test.c.txt

thank you @mingodad
I need to find a reduced example of the issue - I will try to see if I can minimize failing tests; but if the issue is size related then it may be difficult to minimize.

Probably what I said here Also in a several functions the type of some variables have changed from v0.0 to v1.0 is not correct but I noticed that in v0.0 there is less locals (64) than in v1.0 (65) but v0.0 call  alloca	fp, 176 and v1.0 call alloca	fp, 160 (all the above are for __ravifunc_159:	func	i32, u64:U0_L).

Running on Ubuntu-18.04 64bits with valgrind:
***** FILE 'errors.lua'*****
testing errors
==12900== Conditional jump or move depends on uninitialised value(s)
==12900==    at 0x543C8AA: vfprintf (vfprintf.c:1642)
==12900==    by 0x54688AF: vsnprintf (vsnprintf.c:114)
==12900==    by 0x5444F9E: snprintf (snprintf.c:33)
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==    by 0x126BA5: luaD_call (ldo.c:604)
==12900==
==12900== Use of uninitialised value of size 8
==12900==    at 0x543883B: _itoa_word (_itoa.c:179)
==12900==    by 0x543BEDD: vfprintf (vfprintf.c:1642)
==12900==    by 0x54688AF: vsnprintf (vsnprintf.c:114)
==12900==    by 0x5444F9E: snprintf (snprintf.c:33)
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==
==12900== Conditional jump or move depends on uninitialised value(s)
==12900==    at 0x5438845: _itoa_word (_itoa.c:179)
==12900==    by 0x543BEDD: vfprintf (vfprintf.c:1642)
==12900==    by 0x54688AF: vsnprintf (vsnprintf.c:114)
==12900==    by 0x5444F9E: snprintf (snprintf.c:33)
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==
==12900== Conditional jump or move depends on uninitialised value(s)
==12900==    at 0x543BFE4: vfprintf (vfprintf.c:1642)
==12900==    by 0x54688AF: vsnprintf (vsnprintf.c:114)
==12900==    by 0x5444F9E: snprintf (snprintf.c:33)
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==    by 0x126BA5: luaD_call (ldo.c:604)
==12900==
==12900== Conditional jump or move depends on uninitialised value(s)
==12900==    at 0x543CB1C: vfprintf (vfprintf.c:1642)
==12900==    by 0x54688AF: vsnprintf (vsnprintf.c:114)
==12900==    by 0x5444F9E: snprintf (snprintf.c:33)
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==    by 0x126BA5: luaD_call (ldo.c:604)
==12900==
+
testing stack overflow

Here is my modified script to run the tests:
#!/bin/sh

LUA=$1
RUN_TRACEHOOK_TESTS=$2

if [ "$LUA" = "" ]
then
  echo "Please pass path to Ravi executable"
  exit 1
fi

run_lua53_tests() {
  arg=$1
  errmsg=$2
  cd lua53
  valgrind $LUA -e"$arg" all.lua
  rc=$?
  cd ..
  if [ $rc != 0 ]
  then
    echo "$errmsg"
    exit 1
  fi
}

run_ravi_tests() {
  dir=$1
  script=$2
  arg=$3
  cd $dir
  valgrind $LUA -e"$arg" $script
  rc=$?
  cd ..
  if [ $rc != 0 ]
  then
    echo "Test $script failed"
    exit 1
  fi
}

run_lua53_tests "_port=true ravi.jit(false)" "Lua53 interpreter test failed"
run_lua53_tests "_port=true ravi.auto(true)" "Lua53 auto JIT test failed"
run_lua53_tests "_port=true ravi.auto(true,1)" "Lua53 auto JIT all test failed"

run_ravi_tests language ravi_tests1.ravi "ravi.jit(false)"
run_ravi_tests language ravi_tests1.ravi "ravi.auto(true,1)"
run_ravi_tests language ravi_tests2.ravi "ravi.jit(false)"
run_ravi_tests language ravi_tests2.ravi "ravi.auto(true,1)"
run_ravi_tests language defer_tests.ravi "ravi.jit(false)"
run_ravi_tests language defer_tests.ravi "ravi.auto(true,1)"
run_ravi_tests language ravi_tests3.ravi "ravi.auto(true,1)"
run_ravi_tests language ravi_errors.ravi "ravi.auto(true,1)"
run_ravi_tests language basics.lua "ravi.auto(true,1)"
run_ravi_tests extra gaussian2.lua "ravi.auto(true,1)"
run_ravi_tests extra bittest.lua "ravi.auto(true,1)"
run_ravi_tests performance fannkuchen.ravi "ravi.auto(true,1)"
run_ravi_tests performance mandel1.ravi "ravi.auto(true,1)"
run_ravi_tests performance matmul1_ravi.lua "ravi.auto(true,1)"
run_ravi_tests performance sieve.ravi "ravi.auto(true,1)"
run_ravi_tests performance pisum.ravi "ravi.auto(true,1)"
run_ravi_tests performance md5test.lua "ravi.auto(true,1)"
run_ravi_tests comptests test.lua "ravi.jit(false)"
exit 0

thank you

One more piece of info - I wasn't setting an opt level, so it was using the default level.
Opt level 0 and 1 do not appear to have the bug.
Opt level 2 fails.

It seems related to size somehow.
One of the failures is in a file containing many tests; test 88 fails.
If I remove first 10 tests then test 98 fails.
If I remove first 20 tests then all tests pass.

Literally, if I remove one test, it fails after 1 test.

I wonder if the source file is too large causing something to overflow; maybe an integer value is overflowing? Seems to be related to opt level 2.

for hard to debug stuff like this git bisect could be the right tool.

@rofl0r thanks for the suggestion; it may indeed by be a good idea to look back at the commits to see when it starts to fail. But the problem is that the bbv branch had many commits, and I not even sure if there were points in the commit history where the branch was usable.

just go back to where it branched off. if it works there, you can tag it good, and start bisecting. the good thing about bisecting is that it needs only a few attempts to find the culprit even for hundreds or thousands of commits, because you always split the candidate commits in half.

Thank you for working on this bug.  Even providing the test case requires a lot of efforts. I really appreciate your involvement.   Although I tested the release well but one person is not enough.
In order to work on this issue I need some standalone C program (the bug can be in C code, e.g. C undefined behavior can be used, the bug can be in c2mir compiler, although I don't see it in your comparison of different mir versions).  The C program should be executable with some result which says that the program works or not.  The C code can be very big.  It does not matter for me.  Having this program I can bisect optimizations and functions of the program and find the bug origin.
I understand that providing the standalone program requires even more efforts from you but without such program it is practically impossible to find the bug reason and fix the bug.

I guess the issue might be related to the size of a function; Lua code that is all part of a single script goes into a single top level function, making it large. It is hard to find such examples in regular C code.
Its impossible to create a standalone C program.
I think the best approach is as suggested by @rofl0r  - but even this is a lot of effort, but I will have a go at it over the weekend. If we can narrow down the commit from when the bug started then it might help find the issue.

But have you fixed the problems that valgrind found ?

@mingodad  don't think those are real problems - its reporting issues with standard C functions.
I have ASAN enabled and no failures reported there - but of course I don't think ASAN covers JIT output.

    Make add_ssa_edge return the edge.

 mir-gen.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

I realize that this may be an interim issue that may have been fixed - but I would require patch to continue bisecting.

I am going to do a different bisect run to see if I can find a later point with a different issue

okay so my second bisect run ignores other errors and focuses on the issue with "large" functions.
git bisect good
3f693641b509a14a1e527600ce848696ff1d626e is the first bad commit
commit 3f693641b509a14a1e527600ce848696ff1d626e
Author: Vladimir N. Makarov<EMAIL_ADDRESS>Date:   Tue Jun 13 10:23:56 2023 -0400

    Constrain conflict matrix size:
      Add MIR_MAX_COALESCE_VARS.  Add scan_collected_moves.  Call scan_collected_moves in consider_move_vars_only.
      Extract new func collect_moves from coalesce.

 mir-gen.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------------
 1 file changed, 50 insertions(+), 32 deletions(-)

https://github.com/vnmakarov/mir/commit/3f693641b509a14a1e527600ce848696ff1d626e

It seems that you didn't understood the valgrind output:
***** FILE 'errors.lua'*****
testing errors
==12900== Conditional jump or move depends on uninitialised value(s)
...
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==    by 0x126BA5: luaD_call (ldo.c:604)
==12900==
==12900== Use of uninitialised value of size 8
...
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==
==12900== Conditional jump or move depends on uninitialised value(s)
...
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==
==12900== Conditional jump or move depends on uninitialised value(s)
==12900==    at 0x543BFE4: vfprintf (vfprintf.c:1642)
==12900==    by 0x54688AF: vsnprintf (vsnprintf.c:114)
==12900==    by 0x5444F9E: snprintf (snprintf.c:33)
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==    by 0x126BA5: luaD_call (ldo.c:604)
==12900==
==12900== Conditional jump or move depends on uninitialised value(s)
...
==12900==    by 0x131722: tostringbuff (lobject.c:377)
==12900==    by 0x131CFE: addnum2buff (lobject.c:481)
==12900==    by 0x131FC4: luaO_pushvfstring (lobject.c:512)
==12900==    by 0x1324F0: luaO_pushfstring (lobject.c:564)
==12900==    by 0x123D75: luaG_addinfo (ldebug.c:677)
==12900==    by 0x1241D4: luaG_runerror (ldebug.c:702)
==12900==    by 0x149E9F: luaV_idiv (lvm.c:892)
==12900==    by 0x153D05: luaV_execute (lvm.c:1635)
==12900==    by 0x126BA5: luaD_call (ldo.c:604)
==12900==
+
testing stack overflow

Valgrind is interpreting the machine code and detected use of not initialized memory in libc but that has the origin in your code just around formatt %a %A that ultimately probably is calling printf like in libc, I'm not confirm that this is the problem but they are highly correlated and I'll not be surprised if it's indeed the causing of the failure (and other not known yet bugs).
testing strings and string library
testing 'format %a %A'
............./home/runner/work/ravi/build/ravi: strings.lua:366: bad argument #1 to 'assert' (value expected)
>>> closing state <<<

stack traceback:
	[C]: in function 'assert'
	strings.lua:366: in main chunk
	[C]: in function 'dofile'
	all.lua:167: in main chunk
	[C]: in ?
Lua53 auto JIT test failed

It is possible that there is a separate bug (notice that the call stack has luaG_runerror - so it is already in the process of reporting an error). I will have a look at it, but right now I don't think it is related.

Well, I checked this, and actually what I said above does not apply in Lua tests, because they are compiled from Lua bytecodes.
  Add scan_collected_moves.  Call scan_collected_moves in consider_move_vars_only.
      Extract new func collect_moves from coalesce.

 mir-gen.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------------
 1 file changed, 50 insertions(+), 32 deletions(-)

3f69364
To me this looks like a bug::
  scan_vars_num = 0;
  scan_collected_moves (gen_ctx);
  return scan_vars_num > 0;

I can confirm that reverting this commit fixes the issue.

To me this looks like a bug::
  scan_vars_num = 0;
  scan_collected_moves (gen_ctx);
  return scan_vars_num > 0;

Yes, you are absolutely right.  I fixed this by https://github.com/vnmakarov/mir/commit/19d4a622c525428048da221ef0282f903c4bd6f3 .
Although I have a question: how is important generated code speed for the test case on which the problem occurs?
W/o this optimization (coalescing), the generated code will be quite bad.  It will have a lot of moves as for each phi result and operand a copy is added.  When there are a lot vars (and consequently conflict matrix is big), I could do coalescing on live range basis.  The current coalescing based on conflict graph has more quality than coalescing based on live-range uses.

I can confirm the fix worked, thank you for looking into it. I assume this was size related; how is this feature tested? Presumably you do not have a test case that has a large enough function to hit the limit?
Re impact, I do not know yet as I haven't measured, but its okay I think to penalize very large functions, rather than fail completely or fail at runtime. I can of course limit JIT compilation based on function size. Usually such large functions are the top level script and the performance of these are unlikely to be important.
Another question is that in v0 there was no limit, but this limit was added in v1, is that right? What was the reason for it?

Another question is that in v0 there was no limit, but this limit was added in v1, is that right? What was the reason for it?

v0 finds conflicts based on live-ranges.  It is difficult to evaluate complexity of the algorithm but I would say it is "close to" linear.  Therefore there is no limit.
v1 finds conflicts based on conflict graph.  It is more superior.  For example, v0 considers p1 and p2 conflicting in the following case (it is a pretty frequent case after out-of-SSA pass):
p2 = p1
...
use p1 and p1 becomes dead
...
use p2

when v1 recognizes that there is no conflict.  Complexity (in time and space) of conflict graph based algorithm can be quadratic in the worst case.  Therefore I introduced a limit.
Using v0 algorithm as a backup when we hit the limit of v1 algorithm could be a solution.  I'll work on it.  It will be not a big addition to the MIR-generator code.


Issue with displaying the table

I am trying to replicate the example shown under "How do I use the code?", but I am unable to get the table displaying properly. It is as if there is no css applied to the data. I'm probably missing something, but what? Code used below:
<!DOCTYPE html>
<html>
<head>
	<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.0/jquery.min.js"></script>
	<script src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.11.4/jquery-ui.min.js"></script>
	<script src="https://cdnjs.cloudflare.com/ajax/libs/pivottable/2.1.0/pivot.min.css"></script>
	<script src="https://cdnjs.cloudflare.com/ajax/libs/pivottable/2.1.0/pivot.min.js"></script>
	<style>
		body{ font-family: Verdana; }
	</style>
	</head>
	<body>
		<div id="pivotTest"> </div>

		<script>$(function(){
			$("#pivotTest").pivotUI(
				[
					{color: "blue", shape: "circle"},
					{color: "red", shape: "triangle"}
				],
				{
					rows: ["color"],
					cols: ["shape"]
				}
			);
		});</script>
	</body>
</html>

Result:

Thanks in advance,

Yeah I was missing something - my head! All I had to do was change
<script src="https://cdnjs.cloudflare.com/ajax/libs/pivottable/2.1.0/pivot.min.css"></script>
to
<link rel="stylesheet" type="text/css" href="https://cdnjs.cloudflare.com/ajax/libs/pivottable/2.1.0/pivot.min.css" />
:)


problem with C decl

(Original issue 1032 created by None on 2014-09-19T20:58:08.921597+00:00)
With this fragment of code :
int (*f(int))(int);
int g(int);
g and f doesnt appear the same color althought they only differ by their types.

(Original issue 1032 last updated on 2014-09-19T22:00:29.136406+00:00)
(Issue automaticaly closed due to status in Bitbucket: wontfix)

(Original comment by birkenfeld on 2014-09-19T22:00:29.150034+00:00)
Highlighting C function types correctly is very hard. Remember we don't do anything close to a full parse.
These are minor warts, which have to be expected in a highlighter.


ReadAsync throws open SSL error for specific table on Linux on Azure

Describe the bug
When connecting to SQL Server on Azure with Linux client we get a SQLException that has a error linked to OpenSSL. We tried both Ubuntu 18x and Ubuntu 19x and same error.
Oddly enough, this error happens on a very specific table.
Also same error with older client System.Data.SqlClient
This is on net core preview 7
Exception message:
Stack trace:
 ---> System.Data.SqlClient.SqlException (0x80131904): A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 35 - An internal exception was caught)
 ---> System.IO.IOException: The decryption operation failed, see inner exception.
 ---> Interop+OpenSsl+SslException: Decrypt failed with OpenSSL error - SSL_ERROR_SSL.
 ---> Interop+Crypto+OpenSslCryptographicException: error:1408F10B:SSL routines:ssl3_get_record:wrong version number
   --- End of inner exception stack trace ---
   at Interop.OpenSsl.Decrypt(SafeSslHandle context, Byte[] outBuffer, Int32 offset, Int32 count, SslErrorCode& errorCode)
   at System.Net.Security.SslStreamPal.EncryptDecryptHelper(SafeDeleteContext securityContext, ReadOnlyMemory`1 input, Int32 offset, Int32 size, Boolean encrypt, Byte[]& output, Int32& resultSize)
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer)
   at System.Data.SqlClient.SNI.SNIPacket.<ReadFromStreamAsync>g__ReadFromStreamAsync|27_0(SNIPacket packet, SNIAsyncCallback cb, ValueTask`1 valueTask)
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParserStateObject.ReadAsyncCallbackCaptureException(TaskCompletionSource`1 source)
--- End of stack trace from previous location where exception was thrown ---
   at Apption.DBConnectors.SQLServer.SQLServerTableMetadata.ReadTable(SQLDatabase database, SQLTable table, ITargetBlock`1 target)

Expected behavior
SQL Clients reads data correctly from source data.
Further technical details
Microsoft.Data.SqlClient version: 1.0.19189.1
SQL Server version: Azure SQL Server
Operating system:  Ubuntu 18.04 docker container
Additional context
Error seems to be related to SSL version https://stackoverflow.com/questions/29627991/1408f10bssl-routinesssl3-get-recordwrong-version-number-call-on-indy

Oddly enough, this error happens on a very specific table.

Could you provide more specific details so we can debug better?

Oddly enough, this error happens on a very specific table.

Could you provide more specific details so we can debug better?

Yes, we could provide you a private access to that dataset and the repro code. How can we send you these details?

You may send me email with necessary details.

same problem here on preview7
works fine with 4.7.0-preview6.19303.8

I also see this error. The app is running on the mcr.microsoft.com/dotnet/core/aspnet:3.0.0-preview8-alpine docker image

I have now tried with both System.Data.SqlClient 4.7.0 Previiew8 and Microsoft.Data.SqlClient 1.0.19221.1-Preview.
It turns out that System.Data throws this error but Microsoft.Data does not. It, however, returns corrupted data at some positions in the returned result!!!

@rmja Do you have a repro available that you can provide here?

I am working on it...

Done....
Code is here: https://github.com/rmja/repro145
I have mailed the connection string to you. Please let me know when you are done with it, so that I can shut down the database instance.
The sample will run/fail depending on the launch configurration:

@cheenamalhotra were you able to reproduce the issue on your side?

I have investigated this further and these are my findings:
The following very simple program produces the issue:

Program.cs
class Program
    {
        private const string ConnectionString = "Server=tcp:xxx.database.windows.net,1433;Initial Catalog=xxx;Persist Security Info=False;User ID=xxx;Password=xxx;MultipleActiveResultSets=True;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;";

        static void Main(string[] args)
        {
            using (var connection = new Microsoft.Data.SqlClient.SqlConnection(ConnectionString))
            {
                connection.Open();

                //using (var command = new Microsoft.Data.SqlClient.SqlCommand("SELECT TOP(5811) Id, DomainId, Timezone, DeviceNumber FROM Devices ORDER BY Id", connection)) // Works
                //using (var command = new Microsoft.Data.SqlClient.SqlCommand("SELECT TOP(5812) Id, DomainId, Timezone, DeviceNumber FROM Devices ORDER BY Id", connection)) // Times out
                //using (var command = new Microsoft.Data.SqlClient.SqlCommand("SELECT TOP(5813) Id, DomainId, Timezone, DeviceNumber FROM Devices ORDER BY Id", connection)) // Times out
                //using (var command = new Microsoft.Data.SqlClient.SqlCommand("SELECT TOP(5814) Id, DomainId, Timezone, DeviceNumber FROM Devices ORDER BY Id", connection)) // Times out
                //using (var command = new Microsoft.Data.SqlClient.SqlCommand("SELECT TOP(5815) Id, DomainId, Timezone, DeviceNumber FROM Devices ORDER BY Id", connection)) // Throws if MultipleActiveResultSets=True
                using (var command = new Microsoft.Data.SqlClient.SqlCommand("SELECT Id, DomainId, Timezone, DeviceNumber FROM Devices ORDER BY Id", connection)) // Throws if MultipleActiveResultSets=True
                {
                    using (var reader = command.ExecuteReader())
                    {
                        while (reader.Read())
                        {
                            reader.GetInt32(0);
                            reader.GetString(1);
                            reader.GetString(2);
                            reader.GetString(3);
                        }
                    }
                }

                connection.Close();
            }
        }
    }

SQL Script seeding table
CREATE TABLE Devices (
	Id int identity(1,1) primary key,
	DomainId nvarchar(50) not null,
	Timezone nvarchar(50) not null,
	DeviceNumber nvarchar(50) not null
)

BEGIN TRANSACTION
DECLARE @i int = 0

WHILE @i < 10000
BEGIN
	INSERT INTO Devices (DomainId, Timezone, DeviceNumber) VALUES ('domain', 'Europe/Copenhagen', @i)
    SET @i = @i + 1
END

COMMIT TRANSACTION

I have only been able to reproduce it on SQL Azure, and not on my local SQL Server Developer instance.
The following two cases triggers the error:

Run in a linux docker container with a package reference to

<PackageReference Include="Microsoft.Data.SqlClient" Version="1.0.19239.1" />

Dockerfile
FROM mcr.microsoft.com/dotnet/core/runtime:3.0.0-preview8-buster-slim AS base
WORKDIR /app

FROM mcr.microsoft.com/dotnet/core/sdk:3.0-buster AS build
WORKDIR /src
COPY ["ConsoleApp3/ConsoleApp3.csproj", "ConsoleApp3/"]
RUN dotnet restore "ConsoleApp3/ConsoleApp3.csproj"
COPY . .
WORKDIR "/src/ConsoleApp3"
RUN dotnet build "ConsoleApp3.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "ConsoleApp3.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "ConsoleApp3.dll"]

Alternatively, run in Windows with a debug build of Microsoft.Data.SqlClient and set the Microsoft.Data.SqlClient.UseManagedSNIOnWindows environment variable to true.

I have been unable to debug into the SqlClient inside docker, but as I have been able to reproduce it in Windows, the following is what I see when the error occurs:

This ultimately results in the following exception being thrown:

Some notes are:

The error only occurs if the number of rows/data received exceeds some threshold. With the example dataset selecting 5811 rows works fine but 5815 rows throws.
The errors seems to only happen if MARS is enabled.
I intuition (take it with a grain of salt) tells me that this is somehow related to https://github.com/dotnet/corefx/issues/40476

If you want to debug into the source in docker you can add the pdb to the nuget package and it'll let you trace in as long as JustMyCode isn't enabled. That's how i was looking at the bug in the linked PR.
I agree with your intuition that it may be related to the TdsOverSslStream error. I should just be able to enable mars on my test project and see if it fails/succeeds at the same time.

@Wraith2 got it, here is the exception thrown from within the docker container:

Note to myself: I disabled "Just My Code" and copied C:\Users\rmja\.nuget\packages\microsoft.data.sqlclient\1.0.19239.1\runtimes\unix\lib\netcoreapp2.1 to C:\Source\ConsoleApp3\ConsoleApp3\bin\Debug\netcoreapp3.0\runtimes\unix\lib\netcoreapp2.1 before starting the debug session.

It's going to take some poking at the TDS data to see what's wrong with it. I'll see if i can look into it this evening. It isn't going to be the same problem but it could easily be a related issue.
The managed implementation has a few rough areas which is one of the reasons i've made so many changes to it in corefx. Hopefully with the faster release cadance here we can identify and rectify them independently of corefx releases and in doing so improve non-windows reliability and performance.

@Wraith2 year, they are for sure not related. This is a different issue.
The problem here has to do with reading the length of a string when the length (2 bytes) is split across two packets:
This is a dump of what I see where a row corresponds to a SQL datarow:
// Packet header
04 00 1F 40 00 72 2E 00
// Last portion of row continued from previous packet
                                    00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 35 00 36 00 35 00
// A full row
D1 BF 15 00 00 0C 00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 35 00 36 00 36 00
// A full row
D1 C0 15 00 00 0C 00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 35 00 36 00 37 00
... // Many more rows
// A full row
D1 37 16 00 00 0C 00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 36 00 38 00 36 00
// A full row
D1 38 16 00 00 0C 00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 36 00 38 00 37 00
// Start of a row
D1 39 16 00 00 0C 00 64 00

// Next packet header
04 00 1F 40 00 72 2F 00
// Continuation of previous data
                           6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 36 00 38 00 38 00
// A full row
D1 3A 16 00 00 0C 00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 36 00 38 00 39 00
... // Many more rows
// A full row
D1 B2 16 00 00 0C 00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 38 00 30 00 39 00
// A full row
D1 B3 16 00 00 0C 00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 38 00 31 00 30 00
// A part of a row. NOTICE: 0C is the first out of 2 bytes describing the length of the string
D1 B4 16 00 00 0C

// The next header
04 00 1F 40 00 72 30 00
// Continuation of the previous packet. The first byte 00 is the last of the two length bytes
                  00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 38 00 31 00 31 00
D1 B5 16 00 00 0C 00 64 00 6F 00 6D 00 61 00 69 00 6E 00 22 00 45 00 75 00 72 00 6F 00 70 00 65 00 2F 00 43 00 6F 00 70 00 65 00 6E 00 68 00 61 00 67 00 65 00 6E 00 08 00 35 00 38 00 31 00 32 00
...

Clearly for this to work, the 0C has to be "saved" while reading the next packet. The code that should handle this is:

The following shows _bTmp after the 0C is read into the buffer:

However, after a single step over TryPrepareBuffer() _bTmp suddenly looks like this:

This is of cause causing a bad interpretation. _bTmp is somehow being used in between which is unfortunate. I have been unable to determine which code overwrites the buffer. It is somehow done from a different thread.

One more thing. It seems that if I skip the SendAckIfNecessary() call:

while reading invoking TryPrepareBuffer() during the read of the 2-byte length field then _bTmp does not get corrupted. Whether this is the cause or just a coincidence, I don't know.

It might be this https://github.com/dotnet/corefx/pull/37270 it looks like the change but not the fix got applied and it's the only way i know of to corrupt that buffer, it's also string specific. If so applying the same fix should work.

@Wraith2 you are absolutely right. I can confirm that it fixes the issue. I created #171 which applies https://github.com/dotnet/corefx/pull/37270 to M.D.SqlClient.
Thank you so much for helping me with this!

Well I'm sorry I broke it in the first place. It's been a bad day for my quality of my work.

@David-Engel I just want to point out the severity of this bug. Basically hitting the exception is a good thing because it is quite obvious that something is wrong. However, subtle reads of bad data can happen and is why I found the bug in the first place. Getting it to throw is a special, lucky case.
I would say that 1.0.19239.1 is dangerous to use in a linux/osx invironment as incorrect reads of values across packets can and will happen without any errors shown to the user.
Because of this I would strongly recommend to get the fix out in a 1.0.1 patch release asap instead of waiting on a 1.1.0.
cc @Wraith2 @cheenamalhotra

I'm pretty sure that conversation is being had already.
I agree that the possibility of silent bad data warrants a release.

Sorry for my impatience to get this fix out in a hurry, but I really believe that it is urgent now where ef core relies on M.D.SqlClient. It's release is imminent and it will have a dependency with this fatal issue. I really hope that a patch release will be out asap.
I created https://github.com/aspnet/EntityFrameworkCore/issues/17592 to track the update of the reference if this gets a patch release before the ef core release
Cc @cheenamalhotra @David-Engel @Divega


[CXF-6695] Adding JAX-RS 2 extension for handling @BeanParam and @Mat…

…rixParam

I have created swagger-api/swagger-core#1558 at Swagger with purpose of exploring the possibility to create a new JAX-RS 2.0 module there, instead of accepting this PR in CXF.
If they are keen to work that way, I will remove this PR; otherwise I will propose to merge this PR.

Jenkins keeps failing, but failures don't seem to be related in any way with the changes contained here.
Given the discussion at swagger-api/swagger-core#1558 I am going to merge this PR right now.
We can always remove it should they decide to do something about it.

Merged as

https://github.com/apache/cxf/commit/3ff04d2b458b867ea6437a7ed9949d834b16e346
https://github.com/apache/cxf/commit/cc591f4a3b205a5dae58cadb79828d524a424513
https://github.com/apache/cxf/commit/ec15e35805cf21c03d9eb922241ed4c02419c7e2


Remove wkhtmltopdf package which caused OSX builds to fail

Fixes your issue @martenson, per discussion in https://github.com/galaxyproject/training-material/pull/2040#issuecomment-706173427

Thanks!

xref https://github.com/conda/conda/issues/8089

xref https://github.com/conda/conda/issues/8089


Shell cleanup

Reopening (closed due to cli error)

I don't agree about run.sh.  You're making a lot of unrelated changes at once -- it would be better to have separate pull requests and then we can untangle the issues.  And also squash some of these changes together.  Don't delete things just because you haven't used them.

Done. See #74

I added the subdirectory and docs.sh

There's an empty file in this branch: gh-pages/test. Drop it?

The file readme.md constains an outdate directory structure (misses gh-pages/, pipeline/, files docs.sh, util.sh, contains run.sh).

Done and done.


Erase all items Failed to execute

Hi,
I use ArcRest Toolbox. I have a problem with Erase all items tool since several month. i have this message
scriptDeleteUserContent...
error on line: line 73
error in file name: C:\Users\jpdeboos\Documents\ArcGIS\ArcREST-July15Release\tools\src\deleteUserContent.py
with error message: KeyError: 'items'
Completed script scriptDeleteUserContent...
Failed to execute (scriptDeleteUserContent).
I try with the new version (july15) but it's the same.
Someone has a solution ?

I am not able to reproduce the issue.  Can you provide more details?  Can you verify you have the most current ArcRest installed.
Are you using Portal or Online?

I have this message. I use Online.

Tks

I am not able to reproduce it, can you run the process in debug mode and let me know what 'c' is on line 72.  It should look similar to this:
{'username': 'USERNAME', 'currentFolder': {'username': 'USERNAME', 'title': 'segg', 'id': 'e01be9b1feb14f8d985dea3bfa814edf', 'created':<PHONE_NUMBER>000L}, 'nextStart': -1, 'items': [], 'start': 1, 'num': 0, 'total': 0}

I use the last version (july 2015). We use ArcGIS Online in french.


add acl-token to ttl

resubmit for #571 to fix #559 .
@spencergibb I'm sorry for a new PR.

Hello
What about this PR ? I always have a 403 :
http: Request PUT /v1/agent/check/pass/service:casconfigserver-8888, error: Permission denied
Thanks.

There are ci errors, and I think the sleep is too short in the remove test. But I do not know how to trigger the ci again.

Not sure what to do, tests continue to fail.

@spencergibb The ci error is "This job has been blocked because resource-class:large is not available on your plan. Please upgrade to continue building.", not compile error or test error.

This old issue, how is it still not repaired? Do you need to wait? @spencergibb
What can I help?

I think this is a very serious bug, should not be dragged for so long.

这个bug需要赶紧修复了。大哥们。

Hello guys, do you plan to fix this bug?

3 years, this problem is still.... i wonder why

Hi! As you probably know, this PR introduces some breaking changes. Our academic tool, BreakBot, helps developers better assess the impact of these changes on client projects. BreakBot checked the top 100 most starred clients of spring-cloud-consul on GitHub and found out that only one is impacted by one of the breaking changes.
You can find the full BreakBot report in our fork repository breakbot-playground/spring-cloud-consul.
We hope this information about the impact of this PR's changes is valuable to you, and apologize otherwise. If you're willing to help, we would kindly ask for your help to fill in a 5-minutes survey about the report. Your feedback will help us improve the tool and help us in our research!

3 years, this problem is still.... i wonder why

There is no reason, because technology is really rubbish! Four years of bugs, think about it

This was fixed in 226e08a8eb4cb77ee36861195a234020ee12d665

这已在226e08a中修改

Is there any release version available? If you submit code and then close the bug, have you tried testing or released an available version. Version 4.0.1 still fails

这已在226e08a中修改

Can you teach me how to introduce the main branch from maven? The main branch reference

Main will be released in two weeks. See https://github.com/spring-cloud/spring-cloud-release/milestones?direction=asc&sort=due_date&state=open


Language of sample/snippet included in path redundantly

Issue description
In many places such as in the example below, the implementation language of a snippet or sample is redundantly stated in more than one segment of the directory path.
Example specifying both csharp and CS in the path:
snippets/csharp/VS_Snippets_CLR_Classic/classic Console Example/CS/source.cs
The problem with this way of organizing the source files is that it makes it difficult to compare and manage alternate language examples of similar source. For example if one were to look to see if there are VB and F# counterparts for a C# example one cannot simply look under snippets/VS_Snippets_CLR_Classic/classic Console Example/VB or snippets/VS_Snippets_CLR_Classic/classic Console Example/FS accordingly. Likewise it is more difficult to create a duplicate directory structure for each language and then maintain it consistently after it is created.
Proposed Solution
My proposal would be to remove the high level language specific directories for such API doc samples such as the csharp, fsharp, and visualbasic and use those only for documentation about language unique features when necessary. By having the language specific folder only included the leaf most section for these examples it more inherent and consequently naturally enforced consistency between language implementations. Using the proposed method of organization should also allow newer contributors like myself to more efficiently contribute to fill gaps in missing documentation.

Thanks for your feedback @jpierson. This weird structure was introduced by migrating the samples from MSDN to docs.microsoft.com. But I agree that it makes hard to find the equivalent samples.
Tagging @Thraka and @BillWagner to this issue since they're looking into our CI in the samples repo.
The reason why we split them that way is because we thought that setting up the CI per language would be easier at the time.

@jpierson We're aware of how annoying this... 😁 we're working on redesigning this samples structure and projects over the coming months.

@Thraka, thanks for your response. Please be aware that I'm currently working on helping to fill some gaps with missing F# samples. It would be great to get a little heads up before any restructuring begins so I can wrap up my merge request with my progress so far in the case that I'm incomplete.
Please see:

https://github.com/dotnet/samples/pull/1809
dotnet/dotnet-api-docs/issues/3571

I've gone ahead with requesting review for the first round of additional F# samples in https://github.com/dotnet/samples/pull/1809. I'll try to keep the set of samples granular to the particular documentation page that I'm covering per merge request to avoid a long lived branch.

This looks like a related discussion in this past merge request.
https://github.com/dotnet/samples/pull/1418

@Thraka, is there any issue or PR that I could follow regarding the restructuring plans you mentioned above or are these discussion outside of GitHub as of now?

This one is probably good. We had a meeting about this around the holidays.

The plan for structuring snippet files is published in the contributor guide, added by https://github.com/MicrosoftDocs/Contribute/pull/394-- see https://docs.microsoft.com/en-us/contribute/dotnet/dotnet-contribute

We could probably close this issue now since we have a plan in place.


problem syntax array php with library mongodb with php version 5.3.1  ( Note I must use this version , old version use Array and new version use [ ] , but I need use array , is not [ ] )

( Note I must use this version ) . please help me. thank you

( Note I must use this version , old version use Array and new version use [ ] , but I need use array , is not [ ] )


pty is not able to write to stdout

Modified the local example a little, but I do not see any output on my terminal
package main

import (
	"github.com/iyzyi/aiopty/pty"
	"github.com/iyzyi/aiopty/term"
	"github.com/iyzyi/aiopty/utils/log"
	"io"
	"os"
	"sync"
)

func main() {
	// open a pty with options
	opt := &pty.Options{
		Path: "echo",
		Args: []string{"hello world"},
		Dir:  "",
		Env:  nil,
		Size: &pty.WinSize{
			Cols: 120,
			Rows: 30,
		},
	}
	p, err := pty.OpenWithOptions(opt)

	// You can also open a pty simply like this:
	// p, err := pty.Open(path)

	if err != nil {
		log.Error("Failed to create pty: %v", err)
		return
	}
	defer p.Close()

	// When the terminal window size changes, synchronize the size of the pty
	onSizeChange := func(cols, rows uint16) {
		size := &pty.WinSize{
			Cols: cols,
			Rows: rows,
		}
		p.SetSize(size)
	}

	// enable terminal
	t, err := term.Open(os.Stdin, os.Stdout, onSizeChange)
	if err != nil {
		log.Error("Failed to enable terminal: %v", err)
		return
	}
	defer t.Close()

	// start data exchange between terminal and pty
	var wg sync.WaitGroup
	wg.Add(2)
	go func() { io.Copy(p, t); wg.Done() }()
	go func() { io.Copy(t, p); wg.Done() }()
	wg.Wait()

}

The program just prints a newline but I do not see hello world being printed to stdout. Also the program doesn't terminate fully, I think one of the Copy function is blocked on Read.

Q1: The program just prints a newline but I do not see hello world being printed to stdout.

Args holds command line arguments, including the command as Args[0]. See here. Please modify as:
Args: []string{"echo", "hello world"},

Q2: Also the program doesn't terminate fully, I think one of the Copy function is blocked on Read.

Yes, you are right. So I use only one <-exit. And the blocked goroutine will be destroy when close pty and term.
Here is the revised complete code:
package main

import (
	"github.com/iyzyi/aiopty/pty"
	"github.com/iyzyi/aiopty/term"
	"github.com/iyzyi/aiopty/utils/log"
	"io"
	"os"
)

func main() {
	// open a pty with options
	opt := &pty.Options{
		Path: "echo",
		Args: []string{"echo", "hello world"},
		Dir:  "",
		Env:  nil,
		Size: &pty.WinSize{
			Cols: 120,
			Rows: 30,
		},
	}
	p, err := pty.OpenWithOptions(opt)

	// You can also open a pty simply like this:
	// p, err := pty.Open(path)

	if err != nil {
		log.Error("Failed to create pty: %v", err)
		return
	}
	defer p.Close()

	// When the terminal window size changes, synchronize the size of the pty
	onSizeChange := func(cols, rows uint16) {
		size := &pty.WinSize{
			Cols: cols,
			Rows: rows,
		}
		p.SetSize(size)
	}

	// enable terminal
	t, err := term.Open(os.Stdin, os.Stdout, onSizeChange)
	if err != nil {
		log.Error("Failed to enable terminal: %v", err)
		return
	}
	defer t.Close()

	// start data exchange between terminal and pty
	exit := make(chan struct{}, 2)
	go func() { io.Copy(p, t); exit <- struct{}{} }()
	go func() { io.Copy(t, p); exit <- struct{}{} }()
	<-exit
}

This works , would you be open to accosting PRs? I think this a great lib and I have some ideas to make it more standard lib compatible.

This works, thank you very much .Would you be open to accosting PRs? I think this a great lib and I have some ideas to make it more standard lib compatible.

Looking forward to your PRs. And please try not to break the existing interfaces unless absolutely necessary.


Digital Standards Playbook Project Page Updates

DS Project Page Update
Test Instructions

Go to Digital Standards Playbook page, /projects/digital-standards-playbook/
See that it renders properly on mobile and desktop and matches design

Check out a preview of your pull request here: https://app-servicecanadalabs-dev-pr-preview-3.azurewebsites.net

Check out a preview of your pull request here: https://app-servicecanadalabs-dev-pr-preview-3.azurewebsites.net


Add the temporal integration

What does this PR do?

Motivation

Additional Notes

Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] PR title must be written as a CHANGELOG entry (see why)
[ ] Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
[ ] PR must have changelog/ and integration/ labels attached
[ ] If the PR doesn't need to be tested during QA, please add a qa/skip-qa label.

Closed in favour of https://github.com/DataDog/integrations-core/pull/13807


Expose nebula-http-gateway in clients for docs

I just realized we didn't highlight its existence, which could enable flexible frontend-oriented Graph Viz App development
https://docs.nebula-graph.com.cn/3.5.0/14.client/1.nebula-client/

Currently nebula-http-gateway is only used as a component of dashboard community edition, not as one of the clients for the time being, you can suggest it to the product department.


Ceylon cannot check union of function types correctly.

Boolean(Integer)|Integer(Boolean) f = function(Integer n) => true;

This successfully compiles, but applying it fails:
print(f(1));

Error:
error: argument list type must be assignable to parameter list type:
       '[Integer]' is not assignable to 'Nothing'
    print(f(1));
           ^

Also Ceylon refuses to compile the following code,
saying "cases are not disjoint":
Boolean nb(Integer n) { return true; }
Integer bn(Boolean b) { return 0; }

Integer|Boolean dispatch(Boolean(Integer)|Integer(Boolean) f) {
    switch (f)
    case (is Boolean(Integer)) {
        return f(1);
    }
    case (is Integer(Boolean)) {
        return f(true);
    }
}

That's perfectly correct behavior. The following code should clearly be rejected:
Boolean(Integer)|Integer(Boolean) f = (Boolean b) => 1;
print(f(1));

All you know is that you have a function that either accepts an Integer, or accepts a Boolean. That doesn't prove it accepts an Integer.

Furthermore, your switch statement has non-disjoint branches because Anything(Nothing) is an instance of both types. If an if/else instead.

All you know is that you have a function that either accepts an Integer, or accepts a Boolean.
That doesn't prove it accepts an Integer.

Hmm, this makes sense. We need something which is both an Integer and Boolean, that is Nothing.


Support for hard-fork contract

There's a TUNAv2 contract with some changes. We want to be able to support both.
I've looked through the TUNA hard-fork contract, and I've noticed a few differences:

the TargetState object (used by the miner internally) has its fields reordered and a new miner bytestring field added
the miner bytestring is a blake2b hash of either a public key hash or a NFT (for mining pools, I think)
if using a PKH for the miner field, the contract will verify that it matches one of the signers on the transaction
the on-chain datum fields have been reordered to put extra (message) last instead of before the interlink

It's harder to determine what changes we'll need in the TX building (extra redeemers or whatever) from looking at the smart contract, so we may need to wait until Fortuna is updated to see what's needed

This was done partially in #98

This was finished in #192


Merge changes for asynchronous result API

This PR updates the MySQL plugin to conform to the new Swift Kuery API for fetching results.
It also update the plugin to behave in an asynchronous manner when fetching results.

@kilnerm looks good, make sure the Package.swift changes are reverted before merging


[Bug]: Image not inserted and alignment

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug

Image of thailand is not inserted.

and in above not proper alignment
@apu52 plzz assign me so i could correct it.well i have also contributed in this repo earlier also.I added those place u want to visit.so happy by again seeing this wonderfull project here.
Expected behavior
it should be crrcted
Add ScreenShots/Videos
No response
On which device are you experiencing this bug?
No response
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 Extended contributor
[X] I'm a Hacktoberfest'24  contributor
[X] I have starred the repository

please assign this to me under gssoc and hacktoberfest tag..so that i can contribute...

@TanviGandhotra please try to insert the image only under this issue
As like in the 2nd alignment issue u have proposed has already been assigned to another contributor in another issue


Looking forward for mailsac v2

Before setting up, I was expecting to get a mail server looks the same with mailsac.com. But....err. Only the name looks familiar :-(
So, would there be a schedule to release the new mailsac core? Looking forward to it :-P
Regards,

Hi, man I would love to say I can commit to a schedule. Not gonna happen at the moment. Right now there are two blockers. 1 - extracting some of the payment stuff. 2 - troubleshooting an issue with ECONNRESET or Error: socket hang up that should be fixed with node 0.12, however other things break. If i can extract the payment stuff i'll probably release it anyways though.

+1 on this

+1 on this

@ruffrey Sorry but what exactly is that "payment stuff"?

Hey, it pains me to see the last response I gave was Feb 25. damn sorry guys.
Payment stuff is that now you can pay for a little storage or private messages on mailsac.com, it almost keeps the lights on but still operates at a loss. i hacked it in there when rewriting the server and it needs to be modularized otherwise my paypal info is not just sitting in the open source project, heh. I have a branch with half-implemented plugin system IIRC


What's the root cause to wiki's recommendation: "Pause consumption when add a new column"?

In the ingestion transformation wiki page, it mentioned that

If a new column is added to table or schema configuration during ingestion, incorrect data may appear in the consuming segment.

To ensure accurate values are reloaded, do the following: Pause consumption (and wait for pause status success)

Is it only limited to table which has ingestion transform config? If not, it seems a breaking change since we need to pause consumption for every schema update.
Another question: In which scenario the data will incorrectly appears?

I think the doc might be outdated. When the table config or schema is updated (e.g. new column/index added), a reload is required for segments to pick up the new config and generate the new column/index as needed. For consuming segment, reload (with includingConsuming flag set) will try force committing the segment and start a new consuming segment so that the config change can be picked up.
Do you want to help try this out and revise the documentation?
cc @kelseiv

Thanks Jackie! I will revise the doc when I have bandwidth to do the test.

@Jackie-Jiang what's the best practice for the upsert tables.
Is it required to restart servers for both full upsert and partial upsert tables?
for partial upsert table i think it's required since we need to re-initialize the upsert data manager so newly added column can be appeared in the upsertHandler.

For upsert table, currently we have to restart the server in order to pick up the table config changes because it is associated with table data manager instead of segment data manager.


Getting current dropEffect to DropTarget's drop handler

Hey,
I'm probably missing something obvious here, but I can't seem to figure out how to get the current dropEffect on the DropTarget. Or to be more precise, the drop effect that was used on drop.
What I am trying to do is something like this:
  const boxTarget = {
    drop (props, monitor) {
      if (monitor.getDropEffect() === 'copy') {
        // do something when copying
      }
      else {
        // do something different
      }
    }
  }

I don't believe that getDropEffect is a method on monitor parameter. A complete list of available monitor methods can be found in the monitor docs.
What do you mean by 'drop effect' ?

That was an example of what I would expect from the monitor. Sorry I wasn't clearer.
When holding ALT and dragging, you are effectively in copy mode. So can I figure out inside of DropTarget what mode an item was dropped in with?

Any help with this? I'm simply trying to know what happened on drop. Was it a copy or a simple move? Basically whether ALT was held down during the drop.

In endDrag or drop:
const dropResult = monitor.getDropResult();
if (dropResult) {
  console.log(dropResult.dropEffect);
}

getDropResult always returns null for me in the target's drop function

Nevermind, it seems I can only get the dropResult in endDrag of the source.


add cassandra, gopher, jenkins and jira logos

Some new SVG logos !

Thanks!
Added in a6df8c5feb81efde698dc5f200c707c1527dd442


Snippet with TM_SELECTED_TEXT does not work well

This problem still occurs even if you disable all extensions. This problem was confirmed in Markdown file.
To reproduce

Create a snippet containing TM_SELECTED_TEXT
like below:
"TM_SELECTED_TEXT": {
  "prefix": "tm_selected_text",
  "body": "the original ${1:$TM_SELECTED_TEXT} is gone.",
  "description": "Snippet included TM_SELECTED_TEXT"
},

Use the snippet made in 1

Version

Windows 11

VS Code
Version  : 1.73.0 (user setup)
Commit   : 8fa188b2b301d36553cbc9ce1b0a146ccb93351f
Date     : 2022-11-01T15: 34: 06.111Z
Electron : 19.0.17
Chromium : 102.0.5005.167
Node.js  : 16.14.2
V8       : <IP_ADDRESS>-electron.0
OS       : Windows_NT x64 10.0.22000
Sandboxed: No

I also checked the Insider version. As in the Stable, TM_SELECTED_TEXT didn't work well.
In addition, the Insider version didn't show snippet suggestions. It was displayed by pressing Ctrl+Space.
Version  : 1.74.0-insider (user setup)
Commit   : 2282be522a8031209a739783baede67bfa9fc95a
Date     : 2022-11-08T05: 23: 21.062Z
Electron : 19.1.3
Chromium : 102.0.5005.167
Node.js  : 16.14.2
V8       : <IP_ADDRESS>-electron.0
OS       : Windows_NT x64 10.0.22000
Sandboxed: Yes

I would like to say, TM_SELECTED_TEXT was working the past week, so this issue is recent.

@jrieken

You need to use "F1 > Insert Snippet", "F1 > Surround With Snippet", or the surround with quick fix (from the light bulb) for this

I know the functions of Insert Snippet and Surround With Snippet. However, until now, simply selecting a snippet displayed in the suggestions would insert the selected text at the specified location by TM_SELECTED_TEXT.
Is there no longer a way to use snippets as before?

this is a regression from https://github.com/microsoft/vscode/commit/1e662b070fd9f847dfefbdf3f45d0fe538d7e974

Thank you for reopening this issue.  Sorry for the confusion caused by my poor explanation of this issue.

have the echo-snippet provided above
in the editor have ddd and select it all
type ech, see and accept the snippet suggestion
🐛 only echo is inserted, not echo ddd

The problem presented by this procedure is the very issue I have raised.

Glad this is opened. Following this one. Hopefully this is resolved quickly.

fixed via https://github.com/microsoft/vscode/pull/165871

fixed via #165871

Do I have to use the insiders version to test these changes?

Still, $TM_SELECTED_TEXT does not work!

It seems there is still something wrong with version 1.74.0.
Consider the following snippet and register it for plaintext, markdown and latex (without LaTeX-Workshop enabled)
	"poo": {
		"prefix": "pair",
		"body": "xx((${0:$TM_SELECTED_TEXT}))xx"
	}

Select some text and type pair with quick suggestions enabled. Observe that

in .txt files, some text is replaced by xx((some text))xx as expected
in .md or .tex files, some replaced is replaced by xx(())xx.

I can open a new issue if you prefer.

I've found patterns where $TM_SELECTED_TEXT doesn't work well. This problem still occurs even if you disable all extensions. And this was confirmed in markdown file.
However, the problem does not reproduce all the time, but occasionally. So sorry I couldn't create a GIF image because the problem doesn't reproduce all the time. However, it is quite irritating when encountered.

Create a snippet like this:{
  "fooBar": {
    "prefix": "foobar",
    "body": "foo${1:$TM_SELECTED_TEXT}bar",
    "description": "fooBar"
  }
}

Select multiple lines of text containing the string contained in "body". (It works fine if it's just single line)

Select whether the string inserted by the snippet adjacents the original string or has only one newline between them.

foo bar
foo bar
foo bar

Use the snippet. The following results are obtained.foobar

On the other hands, When using snippets for text such as the following, it was not possible to confirm the behavior that does not work well.
baz
foo bar
baz
foo bar
baz

my impressions of the problems
I feel that $TM_SELECTED_TEXT does not work well when memory is overloaded... I felt that it was reproduced more often when the extensions was enabled than when all the extensions were disabled.
Additionally, even if once this problem occurs, using Ctrl+Z to undo and then using the snippet again works fine.
Sorry for the unclear report.
Stable version
Version  : 1.74.0 (user setup)
Commit   : 5235c6bb189b60b01b1f49062f4ffa42384f8c91
Date     : 2022-12-05T16:38:16.075Z
Electron : 19.1.8
Chromium : 102.0.5005.167
Node.js  : 16.14.2
V8       : <IP_ADDRESS>-electron.0
OS       : Windows_NT x64 10.0.22621
Sandboxed: No

Now it works for me.
Version: 1.74.0 (user setup)
Commit: 5235c6bb189b60b01b1f49062f4ffa42384f8c91
Date: 2022-12-05T16:38:16.075Z
Electron: 19.1.8
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19044
Sandboxed: No

This is still not working in *.txt, *.md, *.js, *.ts, and *.json files. Works in *.html, *.xml, including *.config, files though.
Version: 1.74.1 (user setup)
Commit: 1ad8d514439d5077d2b0b7ee64d2ce82a9308e5a
Date: 2022-12-14T10:30:51.966Z
Electron: 19.1.8
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19042
Sandboxed: No


add loader dry run

Summary
Adds a dry run flag to the loader, enabling users to execute the loader without generating trace loads or running experiments
Implementation Notes :hammer_and_pick:

Add a dryRun flag to the loader which accepts a boolean value.
Modifies the execution flow to exit early by checking the dryRun flag just before calling RunExperiment via the experiment driver in the runTraceMode function.

External Dependencies :four_leaf_clover:

N/A

Breaking API Changes :warning:

N/A

Simply specify none (N/A) if not applicable.

I think you forgot the CPU limit check that we were talking about, please add it

I think you forgot the CPU limit check that we were talking about, please add it

Oh yes, thought I will be adding that in another PR, will add that in


com.mangopay.entities.UboDeclaration processedDate is an Integer but should be long

In the project, the SDK represent a Date value as long timestamp (in second) but in the new class :
com.mangopay.entities.UboDeclaration the "processedDate" is an int.
First, is there any reason behind this choice ?
Second, I think it provock an incoherent processedDate with real values. It would be more reasonable to keep Long as in com.mangopay.entities.KycDocument
Thanks

Fixed in latest release


Support passing CancellationToken to Send()

We would like to pass a CancellationToken through to ForwardContext.Send so we can cancel a request after a certain period.
The use case is replaying the same request to a canary release and we want to ensure that we don't add too much overhead to the overall request i.e. if the canary doesn't work as expected, we can cancel the request.
I'm going to raise a separate issue regarding the sequential processing of the canary request.

Done. It has additional use case for timing out a requests.


Alpine Expression Error: w.bind is not a function

Hi guys, I am running my function as in the async-alpine docs but I am encountering this error.
app-WoaEjcF-.js:2 Alpine Expression Error: w.bind is not a function
app-WoaEjcF-.js:6 Uncaught TypeError: w.bind is not a function

I am getting the same error in more than one function I use Async.

I am using it with Livewire and I have made the initializer arrangement as in "https://livewire.laravel.com/docs/installation#manually-bundling-livewire-and-alpine".
admin/app.js

Update:
When I run it with "npm run dev" it worked fine (vite)
When I build it with "npm run build" it doesn't work fine (vite build)

Hi @fahrim, this looks like it's specifically related to the build setup and how that Vite::asset function processes assets.
Could you please provide a reduced test case repo where I can replicate this?
I'm not very familiar with Livewire or Vite so will need that yo dig into it.
It may be worth considering trying Data components — that is how I've had the most success with build tools like Vite in the past.

@Accudio

Sorry I don't have enough experience to create a test. I'm working on the problem, as far as I understand there is no problem with this asyn-alpine work, I must be doing something wrong somewhere else. I focused on two issues; Maybe there is a problem with the JS Loading order or maybe I have a problem with the vite:asset loading as you mentioned.
https://filamentphp.com/docs/3.x/support/assets#asynchronous-alpinejs-components There is a similar usage here and it says that the js code should be built with esbuild before using it. I don't know why this is necessary, but I can try it in my own code as a final plan.

I found out exactly what was causing the problem. Vite (in vite.config.js) was not building the js assets I specified, that is, the "public/build/assets/example.js" (for Laravel-vite) content was empty after the build. I found out that the reason for this was that vite was processing es6 compatible codes.
Therefore, the codes are made to work by subjecting them to the esbuild process mentioned in the Filament documentation that I mentioned in the previous comment.
I read in a solution discussion that vite does not compile unused codes. Therefore, a solution could be to assign your unbuilt function to a global function with "window." and have vite compile your code. These methods were not very healthy for me, maybe my desk was too messy.
We are divided into two ways for the solution:
Either to overcome Vite's build problem, or to add our js code to a static location (cdn or /public). In terms of overcoming the build problem of Vite, using Async-alpine with Data Components as @Accudio mentioned gives excellent results. Because Vite already understands that your code is being used and returns the build result as positive and at the same time you do not have to specify it in the assets section on vite.config.js. Vite detects that you are using it and compiles it for you.
@Accudio, thank you for your help. I hope it helps those who are stuck on such a topic.

@fahrim Ah, good to know!
Yeah, when dealing with Vite and other build tools I've found that sticking with data components and dynamic imports is way easier.
I'm glad that's working out better for you and thanks for the context for others.


Add dockerfile for fenics+gmsh+h5py

Shouldnt we let this docker image be based on the fenics image we have created? Otherwise there will be a lot of duplicated instructions.

Shouldnt we let this docker image be based on the fenics image we have created? Otherwise there will be a lot of duplicated instructions.

Yes, that is probably a good idea.

@jorgensd I have now created Dockerfiles that are based on the current fenics image. I had to create two separate dockerfiles; one for x86_64 and one for aarch64, since they need different base images and the environment variable HDF5_DIR are different.
There are still a few issues at the moment

If we create a new release, then we will also create a new release of the fenics images. However, if we are going to do this in the same CI run then it would mean that we need to base the fenics-gmsh images on the previous release (or we need to find a way to run the fenics build first and then update the tag in the fenics-gmsh image to use the newest release). This is probably why I thought it would be easiest to just duplicate the Dockerfiles.
I must admit that this CI setup for building the images were rather complicated, and I could need some hints on how to add the build of fenics-gmsh. Perhaps @minrk have some ideas? We would probably need to setup a matrix, but I was note sure how implement this in the current setup.

one for x86_64 and one for aarch64, since they need different base images and the environment variable HDF5_DIR are different.

These can be handled via docker build arguments, if you don't want to duplicate too much. But they actually shouldn't need different base images after #5. We haven't triggered a fenics build since then, though, so I started one.
For the HDF5_DIR, this can be handled at runtime, with:
RUN export HDF5_DIR="/usr/lib/$(uname -m)-linux-gnu/" \
 && python3 -m pip install h5py...

But since build and push are triggered purely on manual trigger, I think we may want to also trigger on merge, depending on what directories were changed. I.e. another workflow that runs on merge that submits the build workflow run with the appropriate arguments.

one for x86_64 and one for aarch64, since they need different base images and the environment variable HDF5_DIR are different.

These can be handled via docker build arguments, if you don't want to duplicate too much. But they actually shouldn't need different base images after #5. We haven't triggered a fenics build since then, though, so I started one.
For the HDF5_DIR, this can be handled at runtime, with:
RUN export HDF5_DIR="/usr/lib/$(uname -m)-linux-gnu/" \
 && python3 -m pip install h5py...

But since build and push are triggered purely on manual trigger, I think we may want to also trigger on merge, depending on what directories were changed. I.e. another workflow that runs on merge that submits the build workflow run with the appropriate arguments.

We could also build HDF5 from source: https://github.com/FEniCS/dolfinx/blob/main/docker/Dockerfile#L211-L216
which would make it architecture independent.

multi-arch fenics:2023-02-17 is up.

building test from single dockerfile: https://github.com/scientificcomputing/packages/actions/runs/4203395037

successfully build multiarch image: https://github.com/scientificcomputing/packages/pkgs/container/fenics-gmsh/71341286?tag=2023-02-17a

Thanks a lot @minrk. This looks great! I guess we can merge this now.


feat: add 'removeMarkdownHeaders' option to remove headers if needed

Remove the headers for example used for Dinosaur like:
---
title: Title
---
Markdown content

If not removed it will processed as table.

Thanks for the PR, I'll check it later this week. Looks good at first glance.


Updating Windows Insider Preview Text

Updating Windows 10 to Windows 11

Learn Build status updates of commit d094dd6:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

hub/apps/design/widgets/index.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


Setting @ErrorHandler doesn't catch SocketTimeoutException

Tested by adding the @ErrorHandler annotation before my class:
@Retrofit.ErrorHandler(MYErrorHandler.class)
public abstract class TestCreationApi {

My handler looks like the example but it never gets called:
public class MyErrorHandler implements ErrorHandler {
    @Override
    public Throwable handleError(RetrofitError cause) {
        Response r = cause.getResponse();
        if (r != null && r.getStatus() == 401) {
            // clear authToken because we've been logged out by the server
            AuthApiManager.setAuthToken(null);

            Log.d("MyErrorHandler", "Logged out because of bad token");

            return new RuntimeException("401", cause);
        }
        return cause;
    }
}

Breakpoints don't get hit in the code and the print statement doesn't seem to work.  Generated code seem to set it though.
        if (this.errorHandler == null) {
            this.errorHandler = new com.serverapi.MYErrorHandler();
        }

It doesn't catch the 401 error or a sockettimeout.  You can reproduce the sockettimeout by disconnecting your internet network temporarily on the emulator (genymotion) so that's easier to test than the 401.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/28176969-setting-errorhandler-doesn-t-catch-sockettimeoutexception?utm_campaign=plugin&utm_content=tracker%2F8327951&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F8327951&utm_medium=issues&utm_source=github).

On Nov 12, 2015, at 4:46 PM, Andrew Chen<EMAIL_ADDRESS>wrote:
It should be fixed: b2b6607 https://github.com/yongjhih/NotRetrofit/commit/b2b66075fdc455c66d37c1191d39ee12e83f3b23
—
Reply to this email directly or view it on GitHub https://github.com/yongjhih/NotRetrofit/issues/6#issuecomment-156246092.

Forgot to mention I did add this to my gradle file:
compile 'com.github.yongjhih.NotRetrofit:retrofit:b2b6607'
apt 'com.github.yongjhih.NotRetrofit:retrofit-processor:b2b6607'

On Nov 12, 2015, at 4:46 PM, Andrew Chen<EMAIL_ADDRESS>wrote:
It should be fixed: b2b6607 https://github.com/yongjhih/NotRetrofit/commit/b2b66075fdc455c66d37c1191d39ee12e83f3b23
—
Reply to this email directly or view it on GitHub https://github.com/yongjhih/NotRetrofit/issues/6#issuecomment-156246092.

Hi Ken,
Sorry for replying late, I was on holiday. I think you were using blocking method. I couldn't reproduce with non-blocking method such as Observable<String> getXXX().
The blocking method must use try-catch avoid throw exception after ErrorHandler handled.
try {
 service.getXXX();
} catch (Throwable e) {}

However, after 9928af72559ec5374156dc3bf4a659f6726b3780, You can use ErrorHanlder to ignore error with blocking method by the following:
public class GitHubErrorHandler implements ErrorHandler {
    @Override public Throwable handleError(RetrofitError cause) {
        Response r = cause.getResponse();
        if (r != null && r.getStatus() == 401) {
            return new RuntimeException("401", cause);
        }
        return null; // ignore error
    }
}

Thanks.  Hope you had a good holiday :-)
FWIW, I tried pinning to the latest:
compile 'com.github.yongjhih.NotRetrofit:retrofit:06d6be7'
apt 'com.github.yongjhih.NotRetrofit:retrofit-processor:06d6be7'
but my code in my error handler never runs.  I think it’s because it gets no exception when the server sends a 401…it sends JSON back w/ an error msgs so it doesn’t cause any exceptions in your code so the error handler never gets calls.  The error handler does get called if it can’t reach the server when I shut off wifi though, so I know the error handler is hooked in right.
ken

On Nov 17, 2015, at 4:10 AM, Andrew Chen<EMAIL_ADDRESS>wrote:
Hi Ken,
Sorry for replying late, I was on holiday. I think you were using blocking method. I couldn't reproduce with non-blocking method such as Observable getXXX().
The blocking method must use try-catch avoid throw exception after ErrorHandler handled.
try {
service.getXXX();
} catch (Throwable e) {}
However, after 9928af7 https://github.com/yongjhih/NotRetrofit/commit/9928af72559ec5374156dc3bf4a659f6726b3780, You can use ErrorHanlder to ignore error with blocking method by the following:
public class GitHubErrorHandler implements ErrorHandler {
@Override public Throwable handleError(RetrofitError cause) {
Response r = cause.getResponse();
if (r != null && r.getStatus() == 401) {
return new RuntimeException("401", cause);
}
return null; // ignore error
}
}
—
Reply to this email directly or view it on GitHub https://github.com/yongjhih/NotRetrofit/issues/6#issuecomment-157314557.

You are right, I also found it. So i committed 6de8721: Support httpError exception throwing.

I’m not getting an error code.
I think I see a problem in your MainTest unit test.
You’re checking that the error handler gets called, in your mock service, but you’re not checking that it’s passing through the response code.  I see that when the error handler gets hit too…the response field is null.
With that checkin, I’m also finding that when I get an error response code, the JSON in the body of the response is no longer being parsed like it used to be.  I can work around this, but it’s a different behavior than before.  Unit tests are definitely a good idea :-)
Hope that helps.
ken

On Nov 17, 2015, at 2:01 PM, Andrew Chen<EMAIL_ADDRESS>wrote:
You are right, I also found it. So i committed 6de8721 https://github.com/yongjhih/NotRetrofit/commit/6de8721057e1c3b1dfd2fc995e083047ff570593: Support httpError exception throwing.
—
Reply to this email directly or view it on GitHub https://github.com/yongjhih/NotRetrofit/issues/6#issuecomment-157471415.

It's mistake that did not check response code in testHttpErrorHandler(). But I was checked the testing log that represented the right response code of 404 before, not null. I think you must clean build after updated version.
And welcome any patch for unit tests to show the problem up.

Cleaning build didn’t help.
I’ll try to write up a unit test for it and submit a PR :-)
thanks,
ken

On Nov 17, 2015, at 2:37 PM, Andrew Chen<EMAIL_ADDRESS>wrote:
It's mistake that did not check response code in testHttpErrorHandler(). But I was checked the testing log that represented the right response code of 404 before, not null. I think you must clean build after updated version.
And welcome any patch for unit tests to show the problem up.
—
Reply to this email directly or view it on GitHub https://github.com/yongjhih/NotRetrofit/issues/6#issuecomment-157482477.

Added unit tests and confirmed error handler gets called when using the MockWebServer.  The unit tests show the issue of not getting the response body if the error handler is called w/ an error response code.
Not sure why it’s not acting the same way on my system w/ a real web server but will investigate more.

On Nov 17, 2015, at 2:37 PM, Andrew Chen<EMAIL_ADDRESS>wrote:
It's mistake that did not check response code in testHttpErrorHandler(). But I was checked the testing log that represented the right response code of 404 before, not null. I think you must clean build after updated version.
And welcome any patch for unit tests to show the problem up.
—
Reply to this email directly or view it on GitHub https://github.com/yongjhih/NotRetrofit/issues/6#issuecomment-157482477.

Merged #8


[Request] Some additional stylings

Hi!
First of all, thank you for this library. It's really promising. This is what I currently use:
prog_name = 'substoforced'
prog_version = '1.0.1'

class RParse(argparse.ArgumentParser):
    def _print_message(self, message, file=None):
        if message:
            if message.startswith('usage'):
                message = f'[bold cyan]{prog_name}[/bold cyan] {prog_version}\n\n{message}'
                message = re.sub(r'(-[a-z]+\s*|\[)([A-Z]+)(?=]|,|\s\s|\s\.)', r'\1[{}]\2[/{}]'.format('bold color(231)', 'bold color(231)'), message)
                message = re.sub(r'((-|--)[a-z]+)', r'[{}]\1[/{}]'.format('green', 'green'), message)
                message = message.replace('usage', f'[yellow]USAGE[/yellow]')
                message = message.replace('options', f'[yellow]FLAGS[/yellow]', 1)
                message = message.replace(self.prog, f'[bold cyan]{self.prog}[/bold cyan]')
            message = f'[not bold white]{message.strip()}[/not bold white]'
            print(message)

class CustomHelpFormatter(argparse.RawTextHelpFormatter):
    def _format_action_invocation(self, action):
        if not action.option_strings or action.nargs == 0:
            return super()._format_action_invocation(action)
        default = self._get_default_metavar_for_optional(action)
        args_string = self._format_args(action, default)
        return ', '.join(action.option_strings) + ' ' + args_string

parser = RParse(
    add_help=False,
    formatter_class=lambda prog: CustomHelpFormatter(prog)
)
parser.add_argument('-h', '--help',
                    action='help',
                    default=argparse.SUPPRESS,
                    help='show this help message.')
parser.add_argument('-v', '--version',
                    action='version',
                    version=f'[bold cyan]{prog_name}[/bold cyan] [not bold white]{prog_version}[/not bold white]',
                    help='show version.')
parser.add_argument('-s', '--sub',
                    default=argparse.SUPPRESS,
                    help='specifies srt input')
parser.add_argument('-f', '--folder',
                    metavar='DIR',
                    default=None,
                    help='specifies a folder where [bold color(231)]SubsMask2Img[/bold color(231)] generated timecodes (optional)\nyou should remove the junk from there manually')
args = parser.parse_args()

With this I get a result like this:

I don't think all of these is currently possible with rich-argparse, but it would be great. These are the missing pieces compared to this (at least I think):

no styling for the comma between long and short option (I think this should be the default but an option would be nice)
missing colon after groups
separate styling for metavar
uppercase "usage" to match groups
same styling for usage options as in groups
rename default group (options)
option to print version before help or other additional texts (I guess this is possible but I don't know how)

Thanks for the feedback. I'll try to respond to all your comments:

missing colon after groups

This is not something that the formatter should do by default as the goal is formatting without altering text as much as possible. That said, you have the option to set RichHelpFormatter.group_name_formatter to do whatever you want (I just realized I forgot to mention it in the readme, sorry about that, I will write about it soon) as you can see the example below.

no styling for the comma between long and short option (I think this should be the default but an option would be nice)
separate styling for metavar

This is a fair request and I would look into it, but it will complicate the implementation so I don't think this will happen any time soon.

uppercase "usage" to match groups
same styling for usage options as in groups

The uppercase "usage" can be done now by overriding the RichHelpFormatter.add_usage method (see example below).
As for styling the usage text in general, I currently use the Syntax feature from rich for its simplicity. I intend to change that in the future for a custom regex highlighter where the options in the usage will have the same format as in the groups and the "usage" prefix will follow other groups formatting.

rename default group (options)

This is not related to the formatter. The formatter just uses whatever group names you pass to it. The solution IMO is to create your own group with the name you want (see example below). Another solution is to ask CPython to change the default in argparse (If I remember correctly, someone already did it when they change the default group name from optional arguments to options in a recent version of python)

option to print version before help or other additional texts (I guess this is possible but I don't know how)

This is considered altering the structure of the help text, so I can't have the formatter do that. The formatter is only meant to style the help text in a better way than the default formatters  provided by argparse without altering its structure. However, what you want is also available by overriding the RichHelpFormatter.add_usage method (or argparse.HelpFormatter.add_usage  in general) to add the text you want before the usage as shown in the example below.
import argparse

from rich_argparse import RichHelpFormatter

prog_name = 'substoforced'
prog_version = '1.0.1'
version_text = f'[bold cyan]{prog_name}[/bold cyan] [not bold white]{prog_version}[/not bold white]'

class CustomHelpFormatter(RichHelpFormatter):
    def add_usage(self, usage, actions, groups, prefix=None) -> None:
        self.add_text(version_text)  # <-- print version before help
        if prefix is None:
            prefix = 'USAGE: '  # <-- "usage" prefix with same formatting as the groups
        return super().add_usage(usage, actions, groups, prefix=prefix)

CustomHelpFormatter.group_name_formatter = lambda name: name.upper() + ':'  # <-- custom group name formatting
CustomHelpFormatter.styles['argparse.text'] = 'default'  # <-- if you don't want the default (italic)
parser = argparse.ArgumentParser(
    prog_name,
    add_help=False,
    formatter_class=CustomHelpFormatter
)
flags_group = parser.add_argument_group('flags')  # <-- custom group replacing "options"
flags_group.add_argument('-h', '--help',
                    action='help',
                    default=argparse.SUPPRESS,
                    help='show this help message.')
flags_group.add_argument('-v', '--version',
                    action='version',
                    version=version_text,
                    help='show version.')
flags_group.add_argument('-s', '--sub',
                    default=argparse.SUPPRESS,
                    help='specifies srt input')
flags_group.add_argument('-f', '--folder',
                    metavar='DIR',
                    default=None,
                    help='specifies a folder where [bold color(231)]SubsMask2Img[/bold color(231)] generated timecodes (optional)\nyou should remove the junk from there manually')
args = parser.parse_args()

I'm looking forward for the updates. Wouldn't it be possible that the library was a replacement for argparse instead of a replacement for it's helpformatter? It would give the users more control over what can be customized and probably easier syntaxes. For example you could do something like this:
parser = rparse.ArgumentParser(
    prog_name,
    add_help=False,
    syntax_color=xy,
    metavar_color=etc
    ...
)

I also have another feature idea. I know it's a little over the top but it would be visually pleasing. The idea is to allow the long options to align with other long options when you have short options with different length. For example you could align 1 extra space to be added.
This:
  -drc, --dynamic-range-compression
  -k, --keeptemp              keep temp files
  -mo, --measure-only         kills DEE when the dialnorm gets written to the progress bar
  -c, --config                show config and config location(s)
  -gc, --generate-config      generate a new config

would become this:
  -drc, --dynamic-range-compression
  -k,  --keeptemp              keep temp files
  -mo, --measure-only         kills DEE when the dialnorm gets written to the progress bar
  -c,  --config                show config and config location(s)
  -gc, --generate-config      generate a new config

Wouldn't it be possible that the library was a replacement for argparse instead of a replacement for it's helpformatter?

No, I wouldn't add yet another library for CLI.
Also, it would be impossible then to use it with any third party code that defines its custom parser class (say django).
Additionally, I would trust a third party help formatter more than I would trust a third party argument parser. Look at it as in the worst case scenario where the formatter has bugs, code depending on the parser should still work fine in production, it is only the --help that might be affected.

I also have another feature idea. I know it's a little over the top but it would be visually pleasing. ...

Sorry but I don't think I will add this. I would like to keep the implementation as simple as possible and this would complicate things. Also, using multiple chars in short options is against the posix conventions as it messes with stacking short options together (i.e. somthing like -ab in place of -a -b) so it is not recommended.
P.S. I might have some free time this weekend to look into your request regarding metavar formatting

I got the metavar coloring adjustable with RichHelpFormatter.styles['argparse.metavar'] in https://github.com/hamdanal/rich-argparse/tree/metavar-style. Please give it a try.
I'll be working on the "usage" coloring next.

i've been doing some kind of crazy stuff with Rich since i discovered it recently[^1] and was pretty stoked to find this new project.  Wanted to offer some feedback based on my obsessive tweaking of Rich configurations to try to maximize the usefulness of data being presented to the end user.
These are just opinions and they're mostly about the default config. I know i can adjust it but a) first impressions are important and b) most potential users will not adjust the defaults. Springboarding them with some good design would probably go a long way towards improving the usefulness of this project as well as its adoption.

i think it would be good design to have the default metavar styling be noticeably different from but still related to the color of the option itself.
i think it's a little odd to color the options in one color at the top line (red/pink) and another one in the detailed view.  coloring things that are the same with the same color helps human beings understand immediately that they are the same thing.
the orange for group headings is great; really makes the sections jump out. but IMHO either the group label or the group description should be italic, not both.  I'd vote for making the group name not italic.  In fact I'd vote for making only the longform description italic along with the option names in the detailed view
i would vote for having the long and short names of the option be very similar but noticeably different colors.  just as an example they could be bright_cyan for the short name and cyan for the long name
optional args should less brightly colored than required args

i'll try to come up with a screenshot of tweaks to the defaults along these lines; easier to discuss visual stuff w/visuals
[^1]: at least crazy enough that the author of Rich tweeted out some images i created along  with the word "intense" and an exclamation point

Thanks for the feedback @michelcrypt4d4mus. To answer your questions:

This is already in progress and will change in the next release.
As I said in https://github.com/hamdanal/rich-argparse/issues/4#issuecomment-1242694775, I am working on changing that. This is a tough change as it requires ditching a simple call to rich.syntax.Syntax for a complex implementation that a simple regex cannot cut. Hopefully I will get an implementation that cover at least the most common uses soon.
In an effort to make the default styling less "sparkly", in the upcoming release I am removing the italic style from all the defaults styles. Users who prefer to have it  can of course still set it in their code.
I prefer to not to add more knobs for now, it already became very complicated to get everything working properly and every time I change something it leads to more and more bugs
Same answer as the previous. Also positional args can be optional and --options can be required so coloring depending on whether the arg is required or not doesn't seem like a good idea

before i respond to any individual points i will throw out there that my python andRich skills/interest are probably at the top of their game right now and i'd be more than happy to poke around if you could put just a very short section in the README about the best way to get a dev env up and running.  i've made a fair amount of private python packages in my day but it's been a while and also i suspect there may some gotchas given the nature of the project.
beyond that just to offer some more feedback/ideas that you should feel free to ignore:

re: italics i actually think a few subtle italics are really great!  I would leave them on the optional switches (left col main table).  w/the caveat that i have not really thought this through, maybe a design rule could be that optional arguments (the actual arg, not the metavar) are lowercase italicized lowercase and anything required is lowercase not italicized?
re: the top vs. side coloring keep it fun.  btw i can't seem to upload screenshots but i also have a weird miscoloring - in [--max-decode-length MAX] only in the top panel... the length is a brighter shade of pink.
re: fixing one thing breaks another - i was going to say "maybe pytest can help and then i looked and saw the very impressive ratio of pytest code to project code already going on in her, so let me be the first to congratulate you for doing that.

Thanks, probably the easiest way to get a dev env is using tox --devenv venv. I'll add a "contributing" section to the readme in the future.

maybe in the future, for now they will be off by default.
this is a bug caused by the current use of Syntax with awk lexer . I'll open a ticket to point that out. It will be solved with the new usage parser I am working on.
yeah unfortunately even with 100% test coverage stuff beak

been messing around with some color themes/tweaks; feel free to take a look
occurred to me that offering a couple of default options as far as theming could be cool

@pcroland I have implemented a bunch of stuff that may interest you, It is still not published on PyPI yet but you can test it with pip install git+https://github.com/hamdanal/rich-argparse.git@main.
These are all now implemented (and the default)

no styling for the comma between long and short option (I think this should be the default but an option would be nice)
missing colon after groups
separate styling for metavar
uppercase "usage" to match groups
same styling for usage options as in groups

With this version, your example roughly becomes
import argparse

from rich_argparse import RichHelpFormatter

version = "1.0.1"
version_text = f"[bold cyan]%(prog)s[/] {version}"

class CustomHelpFormatter(RichHelpFormatter):
    def add_usage(self, usage, actions, groups, prefix=None) -> None:
        self.add_text(version_text)
        return super().add_usage(usage, actions, groups, prefix=prefix)

CustomHelpFormatter.styles["argparse.args"] = "green"
CustomHelpFormatter.styles["argparse.groups"] = "yellow"
CustomHelpFormatter.styles["argparse.metavar"] = "bold color(231)"
parser = argparse.ArgumentParser(
    "substoforced", add_help=False, formatter_class=CustomHelpFormatter
)
flags_group = parser.add_argument_group("flags")
flags_group.add_argument("-h", "--help", action="help", help="show this help message.")
flags_group.add_argument(
    "-v", "--version", action="version", version=version_text, help="show version."
)
flags_group.add_argument("-s", "--sub", default=argparse.SUPPRESS, help="specifies srt input")
flags_group.add_argument(
    "-f",
    "--folder",
    metavar="DIR",
    default=None,
    help=(
        "specifies a folder where [bold color(231)]SubsMask2Img[/] generated timecodes (optional)"
        "\nyou should remove the junk from there manually"
    ),
)
args = parser.parse_args()

I'll close the issue but feel free to reopen it if needed; Thanks for the feedback

Nice, thanks for the update, it works great. I'm going to update my stuff with it once it's on PyPI.


Voxel GC

It seems that much of the noisy voxels in the reconstruction which stem from the noise in the depth estimation can be singled out by looking at their weight. In the following image, red areas correspond to voxels with weight = 1, i.e., which were only "seen" once:

The blue voxels are those with the maximum allowed weight (in our case, just 10, since the camera is moving quite quickly; the original InfiniTAM had ~100).
A quick and dirty solution would be to simply not render low-weight voxels, but that would also require modifications to the mesh generation algorithm.
A better solution would be to (efficiently) trim all voxels with age > k but weight < l, effectively regularizing the resulting reconstruction. This is what I'll be implementing in the end.
Subtasks:

[x] Naive cleanup implementation: kernel to detect old, low-weight voxels, and clear them.
[x] Compute empty blocks
[x] Deallocate empty blocks
[x] Graph to visualize empty block count; hopefully it will no longer constantly go down if we do this!
[x] Clean up code (weird comments, TODOs, debug prints, etc.)
[x] Apply voxel decay to reconstructed object instances as well.
[x] Support "reap mode" which can operate on entire volumes.
[ ] Clean up after getting reap mode working.
[ ] Support deleting elements where collisions occur by looking up excess list entries correctly, and then
performing a proper list deletion in the hash table, in addition to deallocating the VBA element.
[ ] GUI button to "reap" currently active object.

May be worth implementing #19 first to simplify the bookkeeping.

I implemented #19 and, together with it, added full support for voxel gc, complete with correct block freeing and recycling. Everything is done, except the dedicated GUI button, which is low-priority and should be a different issue anyway.


Merge pull request #3353 from clearlydefined/clearlyoss_200114_194143.715

Type: Missing
Summary:
Tech Team - set the declared license
Details:
This component is licensed under the Boost Software License (BSL).
Resolution:
Set the declared license for this component.
Affected definitions:

boost_locale-vc141 1.67.0

@capfei - Hi, this looks like Apache-2.0 to me?