Issue when obfuscating a library which is referenced by a web api project

System in use

.net core 3.1
Release config
Project type: Class Library

Description
When obfuscating a project that is directly referenced by the main .net core project and running the main webapi project, we get this error
System.AggregateException: An error occurred while writing to logger(s). (Value does not fall within the expected range. (Parameter 'typeName@57'))  ---> System.ArgumentException: Value does not fall within the expected range. (Parameter 'typeName@57')    at System.Reflection.CustomAttribute._CreateCaObject(RuntimeModule pModule, RuntimeType type, IRuntimeMethodInfo pCtor, Byte** ppBlob, Byte* pEndBlob, Int32* pcNamedArgs)    at System.Reflection.CustomAttribute.CreateCaObject(RuntimeModule module, RuntimeType type, IRuntimeMethodInfo ctor, IntPtr& blob, IntPtr blobEnd, Int32& namedArgs)    at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder1 derivedAttributes)    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType)    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)    at System.Reflection.RuntimeMethodInfo.GetCustomAttributes(Type attributeType, Boolean inherit)    at System.Attribute.GetCustomAttributes(MemberInfo element, Type type, Boolean inherit)    at System.Reflection.CustomAttributeExtensions.GetCustomAttributes[T](MemberInfo element, Boolean inherit)    at System.Diagnostics.StackTrace.TryResolveStateMachineMethod(MethodBase& method, Type& declaringType)    at System.Diagnostics.StackTrace.ToString(TraceFormat traceFormat)    at System.Exception.get_StackTrace()    at System.Exception.ToString()    at Microsoft.Extensions.Logging.Console.ConsoleLogger.CreateDefaultLogMessage(StringBuilder logBuilder, LogLevel logLevel, String logName, Int32 eventId, String message, Exception exception)    at Microsoft.Extensions.Logging.Console.ConsoleLogger.WriteMessage(LogLevel logLevel, String logName, Int32 eventId, String message, Exception exception)    at Microsoft.Extensions.Logging.Console.ConsoleLogger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func3 formatter)
at Microsoft.Extensions.Logging.Logger.g__LoggerLog|12_0[TState](LogLevel logLevel, EventId eventId, ILogger logger, Exception exception, Func3 formatter, List1& exceptions, TState& state)
--- End of inner exception stack trace ---
at Microsoft.Extensions.Logging.Logger.ThrowLoggingError(List1 exceptions)    at Microsoft.Extensions.Logging.Logger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func3 formatter)
at Microsoft.Extensions.Logging.Logger1.Microsoft.Extensions.Logging.ILogger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func3 formatter)
at Microsoft.Extensions.Logging.LoggerMessage.<>c__DisplayClass4_0.b__0(ILogger logger, Exception exception)
at Microsoft.AspNetCore.Diagnostics.DiagnosticsLoggerExtensions.UnhandledException(ILogger logger, Exception exception)
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.HandleException(HttpContext context, ExceptionDispatchInfo edi)
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
HEADERS
=======
Connection: Keep-Alive
Content-Type: application/json; charset=UTF-8
Accept: /
Accept-Encoding: gzip, deflate, br
Accept-Language: en,en-US;q=0.9,hr;q=0.8,bs;q=0.7
Cookie: ***
Host: ***
Referer: ***
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36
Origin: ***
Content-Length: 4
access-control-allow-origin: *
administration: true
lcid: 2057
access-control-allow-methods: *
sec-fetch-site: same-origin
sec-fetch-mode: cors
sec-fetch-dest: empty
MS-ASPNETCORE-TOKEN: ***
X-Original-Proto: http
X-Original-For: <IP_ADDRESS>:45856`
This only happens when we set the "UseUnicodeNames" option in the obfuscar.xml config to true
This is our config
<?xml version='1.0'?>
<Obfuscator>
<AssemblySearchPath path="***" />
<AssemblySearchPath path="***" />
<Var name="InPath" value="." />
<Var name="OutPath" value=".\Obfuscated" />
<Var name="KeepPublicApi" value="true" />
<Var name="HidePrivateApi" value="true" />
<Var name="RenameProperties" value="true" />
<Var name="RenameEvents" value="true" />
<Var name="RenameFields" value="true" />
<Var name="UseUnicodeNames" value="false" />
<Var name="HideStrings" value="true" />
<Var name="OptimizeMethods" value="true" />
<Var name="SuppressIldasm" value="true" />
<Module file="$(InPath)***.dll" />
</Obfuscator>

Same issue. Any workaround or fix?

@pigiax yes, you have to play around with the config on the projects in question, it was trial and error for us. Not much more help than that.

Ok, for the moment it seems to be fixed with UseKoreanNames instead of UseUnicodeNames. Of course we need to continue to check if everything continues to work

@zlatkolakisic It would be super great if you could share what code you were having an issue with and what exactly fixed it.

@TechnikEmpire I cannot share the source as it is not open source, however turning off UseUnicodedNames solved our issue. Could you shed more light on where this could cause issues?

@zlatkolakisic Of course I understand not being able to share source. But you can still share metadata about that source, like what kind of types they were, where those types were being invoked etc.
Also, no, I can't shed more light on it because I'm as lost as you are lol, that's why I made my comment. I ended up working around this issue as far as I can tell by simply excluding the host class/type that the exception was coming from.

@TechnikEmpire The solution is huge with around 43,000 lines of code so far, I can generate a list of types that are used. Are there any that from your experience proved problematic?

@zlatkolakisic Just to be clear, I'm not a maintainer or contributor to the project. I'm a user who was just asking if you wouldn't mind sharing your experience with working around this issue because I too faced it. I have no clue what type caused the issue, I just excluded the class that the exception was originating from so it didn't get obfuscated. That's how I solved this problem.

@TechnikEmpire in my case I've started like you, excluding the class causing the issue. But i got same error in another class.
The main cause was that obfuscar renames varibles with a new name composed of two letter. The first letter indicates the class, the second it's a progressive letter. In my case i got a property "Name" obfuscated as "ID". Of course on my class i've already got a property named "Id" and this caused a collision and the issue. To solve I've used this code:
<?xml version="1.0" encoding="utf-8" ?> <Obfuscator>   <Var name="InPath" value="./bin/CI/netcoreapp3.1" />   <Var name="OutPath" value="$(InPath)/obfuscar_out" />   <Var name="KeepPublicApi" value="true" />   <Var name="HidePrivateApi" value="true" />   <Var name="RenameProperties" value="true" />   <Var name="RenameEvents" value="true" />   <Var name="RenameFields" value="true" />   <Var name="UseUnicodeNames" value="false" />   <Var name="UseKoreanNames" value="true" />   <Var name="HideStrings" value="true" />   <Var name="OptimizeMethods" value="true" />   <Var name="SuppressIldasm" value="true" />   <Var name="ReuseNames" value="false" />   <Module file="$(InPath)/MyOwn.dll" /> </Obfuscator>

@pigiax thanks for sharing.

针对.NET Core项目

需添加这个配置

@TechnikEmpire in my case I've started like you, excluding the class causing the issue. But i got same error in another class. The main cause was that obfuscar renames varibles with a new name composed of two letter. The first letter indicates the class, the second it's a progressive letter. In my case i got a property "Name" obfuscated as "ID". Of course on my class i've already got a property named "Id" and this caused a collision and the issue. To solve I've used this code:
<Obfuscator>
  <Var name="InPath" value="./bin/CI/netcoreapp3.1" />
  <Var name="OutPath" value="$(InPath)/obfuscar_out" />
  <Var name="KeepPublicApi" value="true" />
  <Var name="HidePrivateApi" value="true" />
  <Var name="RenameProperties" value="true" />
  <Var name="RenameEvents" value="true" />
  <Var name="RenameFields" value="true" />
  <Var name="UseUnicodeNames" value="false" />
  <Var name="UseKoreanNames" value="true" />
  <Var name="HideStrings" value="true" />
  <Var name="OptimizeMethods" value="true" />
  <Var name="SuppressIldasm" value="true" />
  <Var name="ReuseNames" value="false" />
  <Module file="$(InPath)/MyOwn.dll" />
</Obfuscator>```

.NET Core3.1 有效


Master issue for Custom Model Binding

This is the master issue for this doc. If you PR this doc and don't fix all the issues, open the issues you fix and close against that.

[ ] Possible error in route parameter name "authorId" #10341

[ ] Need more detail on using TypeConverters in ASP.NET Core #11820

[ ] Custom model binder for methods with more than one parameter are not documented well enough #13387

[x]  ~How do I specify a binder for a type outside of my project? #16314~

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cfb00531-4ac6-8bf6-9bd8-1eed6446397f
Version Independent ID: ed418d09-6c33-699a-0d48-9aabc99457be
Content: Custom Model Binding in ASP.NET Core
Content Source: aspnetcore/mvc/advanced/custom-model-binding.md
Product: aspnet-core
Technology: aspnetcore-mvc
GitHub Login: @ardalis
Microsoft Alias: riande

How do I specify a binder for a type outside of my project? #16314

This one is already covered in the section Implementing a ModelBinderProvider.

@Rick-Anderson The main sample for this topic is still on 1.0. Is it alright if I do a 3.1 update? If that's a "yes", do you want a 2.2 update first? i.e. Should we have 2.2 and 3.1 or just 3.1?

All we need to maintain is 2.1 and 3.1

Ok. I'll do a 2.1 update first and then add a 3.1 version, unless you say otherwise. Feel free to suggest something else if there's higher priority issues I can help with.


useSubdomain Hook not working correctly

It looks like this hook should be returning a subDomain based on the current network for creating a link to the proper etherscan site.
currently returning a HTML element instead of a string?

https://github.com/decent-dao/fractal-interface/blob/develop/src/hooks/useSubDomain.tsx

Agree, this line seems incorrect:
https://github.com/decent-dao/fractal-interface/blob/537259e53c018d5ce43d1418d3cbf504c659369a/src/hooks/useSubDomain.tsx#L20


User reports blank screen in his VDC

User reports a blank screen in his VDC (same user and same VDC as the one mentioned in #2952 ).
“When I log in to my account and I click "Go to My VDC" I then just get a blank page. I also cannot get into the Marketplace, clicking that button does nothing.”

IP reserved: <IP_ADDRESS>
3bot ID: sgherzi.3bot

please make sure to add all of the vdc relevant information

3Bot ID added, no more info received. What else is needed ?

@gmachtel e.g your 3bot Id linked to all of these VDCs

I can't guess around for every bug report, please make sure 3bot id and VDC name are always attached to any bug report


Improved corner radius logic

Clamps radius distance to half available length (from/to current corner) and also uses a better heuristic for guessing the distance of the control point from the through point instead of just blindly using the arc constant regardless of angle.

It's a draft as I'm waiting to see how designers feel about making this "the way". https://2dimensions.slack.com/archives/CHTMQJAJZ/p1655166724192089

Clamps radius distance to half available length

Just curious why we chose that limit, instead of, say, lowering the radii on both sides until they don't overlap?
I really like how predictable and consistent this rule is, but I'm just wondering if someone would ever want a valid rounded corner that takes up more that half of the available length (and whose neighbor then takes up less than half).


Include ToolsVersion="14.0" in all msbuild files

Didn't notice until I ran a detailed log. MSBuild produces warnings when it encounters our project files:
Project file contains ToolsVersion="". This toolset may be unknown or missing, in which case you may be able to resolve this by installing the appropriate version of MSBuild, or the build may have been forced to a particular ToolsVersion for policy reasons. Treating the project as if it had ToolsVersion="15.0". For more information, please see http://go.microsoft.com/fwlink/?LinkId=293424.

@pranavkm - did you end up doing this?


Insufficient alignment in u_ union in tensorflow/core/lib/gtl/inlined_vector.h

Environment info
Operating System: Linux
Installed version of CUDA and cuDNN: None
If installed from sources, provide the commit hash: aa2cacd6627ffb296bedc910c957a0fd4a2f957f
Steps to reproduce

Find an architecture with 32 bit pointers but strict alignment requirements for 64 bit
Compile and run "benchmark" from tensorflow/contrib/makefile
Get bus error from misaligned pointer

What have you tried?

Fixing the issue by increasing the u_ alignment.  This works.

In tensorflow/core/lib/gtl/inlined_vector.h, the data buf is aligned by adding a pointer.  However, it is cast to other types, including uint64_t.  On architectures where pointers are 32 bits, this cast causes errors.
The fix is to add another component to the union that will force alignment to the largest size, such as uint64_t.
For example:
diff --git a/tensorflow/core/lib/gtl/inlined_vector.h b/tensorflow/core/lib/gtl/inlined_vector.h
index e8fe66c..518b421 100644
--- a/tensorflow/core/lib/gtl/inlined_vector.h
+++ b/tensorflow/core/lib/gtl/inlined_vector.h
@@ -276,6 +276,8 @@ class InlinedVector {
unsigned char data[kSize];
// Force data to be aligned enough for a pointer.
T* unused_aligner;

// EJP: force 8 byte alignment, as pointers could be 4-byte but data elements 8-byte?
uint64_t unused_64;
} u_;

You won't see this issue on architectures where pointers are the size of the largest type (aarch64, x86-64) or where a 32-bit architecture loads 64-bit things but can do so at arbitrary 32-bit boundaries (x86, armv7 with misaligned pointers enabled).
But it might improve performance to have 64-bit values on 64-bit boundaries.

What you're saying makes sense. What are some example processor architectures with this property? It's unlikely we are going to support it, but we would be open to a patch that detects this property in a architecture and does this alignment only when necessary. You indicate it might improve performance to do this extra alignment but it actually also might hurt performance, and without testing we are hesitant to make such a change. Adding @vrv to comment.

Some ARM variants will give an exception on misaligned LDRD (but some do
not, especially if misaligned support is enabled in ARMv7).
You'd probably have a problem if someone used the buffer for a double
precision value on many targets that don't support misaligned loads and
stores (MIPS, ARM)
Sorry, my patch got formatted by github:
========
diff --git a/tensorflow/core/lib/gtl/inlined_vector.h
b/tensorflow/core/lib/gtl/inlined_vector.h
index e8fe66c..518b421 100644
--- a/tensorflow/core/lib/gtl/inlined_vector.h
+++ b/tensorflow/core/lib/gtl/inlined_vector.h
@@ -276,6 +276,8 @@ class InlinedVector {
unsigned char data[kSize];
// Force data to be aligned enough for a pointer.
T* unused_aligner;

// EJP: force 8 byte alignment, as pointers could be 4-byte but data
elements 8-byte
uint64_t unused_64;
} u_;

inline void InitRep() { u_.data[kSize - 1] = 0; }
Basically a one-liner: since a 64-bit value could go into the data (by
cast), you put a 64-bit value into the union to ensure alignment.
On Thu, Jul 7, 2016 at 11:42 AM, Andrew Selle<EMAIL_ADDRESS>wrote:

What you're saying makes sense. What are some example processor
architectures with this property? It's unlikely we are going to support it,
but we would be open to a patch that detects this property in a
architecture and does this alignment only when necessary. You indicate it
might improve performance to do this extra alignment but it actually also
might hurt performance, and without testing we are hesitant to make such a
change. Adding @vrv https://github.com/vrv to comment.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/3186#issuecomment-231136191,
or mute the thread
https://github.com/notifications/unsubscribe/ABkOanmmqCpZ4crPg83EvlyqGFA_-GfRks5qTSx9gaJpZM4JEg_y
.

Created pull request with fix


HelpCommand - recursive layout fix

old
 * Example #2. Fixing with configuration: ['functions' => array ( 'phpversion', )].
   ---------- begin diff ----------
   --- Original
   +++ New
   @@ @@
    <?php
   -echo phpversion();
   +echo PHP_VERSION;
    echo pi();
   ----------- end diff -----------

new
 * Example #2. Fixing with configuration: ['functions' => ['phpversion']].
   ---------- begin diff ----------
   --- Original
   +++ New
   @@ @@
    <?php
   -echo phpversion();
   +echo PHP_VERSION;
    echo pi();
   ----------- end diff -----------

Good catch!
Could you please update/add a test in DescribeCommandTest?

sure, lets do https://github.com/FriendsOfPHP/PHP-CS-Fixer/pull/2876 first than to prevent conflicts

:+1:

do we need to wait for #2876 to merge this PR ? those are different commands, and DescribeCommandTest shall be rather updated in mentioned PR, not here

The updated code is actually a helper method that is used in both HelpCommand and DescribeCommand thus I think we should update DescribeCommandTest here.

DescribeCommandTest updated to test the changes in this PR.

is there any WIP left?

this one is ready, but the tests has conflicts with https://github.com/FriendsOfPHP/PHP-CS-Fixer/pull/2876

please resolve conflicts at 2876


a non-coding gene (MALAT1) with missense and frameshift variants?

ENSG00000251562 is a non-coding gene, therefore it does not give rise to a protein.
However we list some somatic mutations as being either frameshift of missense (consequence terms that involve a change of the amino acid in a protein or change of the open reading frame of a protein).
https://www.targetvalidation.org/evidence/ENSG00000251562/EFO_0000365?view=sec:somatic_mutation
Is this really correct? This was picked up at the STORM Therapeutics workshop this afternoon.
A quick glance at the curated paper does not show this gene (aka MALAT1): https://europepmc.org/backend/ptpmcrender.fcgi?accid=PMC6428276&blobtype=pdf
Looking at COSMIC, they do have the variants indeed as missense:
https://cancer.sanger.ac.uk/cosmic/gene/analysis?ln=MALAT1&mut=substitution_missense#variants
but all their external references point to a non-coding RNA (lincRNA):
https://cancer.sanger.ac.uk/cosmic/gene/analysis?ln=MALAT1&mut=substitution_missense#links
There used to be a protein sequence for this (Q9UHZ2), now obsolete. Haven't the variant annotations been updated since?
COSMIC has a transcript named ENST00000332541 but this no longer exists in Ensembl.
My guess is that in the past the annotation of this gene was protein coding (where the variant annotations from COSMIC come from) but this is no longer the case.
Even in the previous assembly of the human genome, MALAT1 was annotated as non-coding RNA:
http://grch37.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000251562;r=11:65265233-65273940

Ensembl lists these variants as “non coding transcript exon variant” (see http://www.ensembl.org/Homo_sapiens/Location/Variant/Table?r=11:65497688-65506516;g=ENSG00000251562;db=core and then filter by consequences to get counts).

This doesn't look good. The latest COSMIC file we have is from March this year (cosmic-2019-03-22.json.gz) but you are right, this gene is not protein coding and it hasn't been for a long time:

Uniprot deleted Q9UHZ2 in Jan 2017: https://www.uniprot.org/uniprot/Q9UHZ2

According to Ensembl ENST00000332541 has not been protein coding since release 49 (GRCh36, ~2009??): http://www.ensembl.org/Homo_sapiens/Transcript/Idhistory?t=ENST00000332541

Unless @iandunham or @MichaelaEBI know more about what may be happening here, we will need to ask COSMIC about this.

I have checked how widespread this issue is in the 19.04 data and it seems that it only affects 3 non-noding genes (out of 723 in COSMIC) and 20 evidence strings (out of 59k):

Gene ID
Biotype
Mutation Types in COSMIC
Link

ENSG00000197880
lincRNA
Missense
http://cancer.sanger.ac.uk/cosmic/gene/analysis?ln=MDS2

ENSG00000251562
lincRNA
Missense, frameshift
http://cancer.sanger.ac.uk/cosmic/gene/analysis?ln=MALAT1

ENSG00000280757
unprocessed_pseudogene
Missense, framshift, stop gained, translational product function, gene fusion
http://cancer.sanger.ac.uk/cosmic/gene/analysis?ln=DUX4L1

@MichaelaEBI  to include in emails on files to COSMIC so can address this and introduce @AsierGonzalez  to team as Data SME.

This issues was discussed with Zbyslaw from COSMIC who said that it's related to non-coding genes involved in gene fusions and that the old COSMIC schema didn't deal well with those cases. The new COSMIC data schema should be able to deal with it correctly and treat these mutations as non-coding. COSMIC v90 was released the 5th Spetember 2019 and those links don't work any more but they still appear in the evidence strings with the same mutation types.

The evidence for these three non-coding RNAs have disappeared in the COSMIC evidence file submitted for 19.11. Changing status to fixed and awaiting for COSMIC to confirm that the issue has been fixed.


Definitions should be verified to have a valid dfn type

While reviewing the effects of switching Bikeshed's dfn harvester to Reffy, I noticed a new dfn cropped in with a type of "idl" https://github.com/w3c/webref/blob/master/tr/dfns/payment-request.json#L1992
This isn't a valid definition type. (It's a linking supertype that encompasses all the IDL-related types.)
I guess the type of a dfn should be verified to be valid, and the dfn omitted if it's not?
(I reported this on ReSpec as well, since it's not being reported as an error in the source document, but regardless of the inputs, the Reffy data should be clean.)

(All the currently-recognized dfn types are at https://github.com/tabatkins/bikeshed/blob/main/bikeshed/config/dfnTypes.py#L4, + "dfn".)

Looking at dfn types that are currently extracted, "idl" seems to be the only incorrect type in specs for the time being.
There are also "event" and "namespace" types that are not in the documentation but since they are in dfnTypes.py, this seems more a doc maintenance issue.


Allow configuring the truncating of the prompt.

I had the unpleasant situation of working more than three levels deep from my CWD, which prompted me to make this change locally. Perhaps you want it in the theme itself?

Personally, I prefer to have less than three levels — in that case path always fits on the screen. But that's a good point. It makes theme more flexible.

Thank you, merged!


Prompt that SUI version needs to be updated

Whenever the SUI version is updated, there are always some strange bugs reported. Such as
unknown variant~boo1~expected one of 'u16~,u32~~u64f64' at line 1 column 3447. So hopefully there will be a reminder that users need to upgrade the version, rather than reporting weird bugs

The code is correct, I can't use it until I update the version

@Sifotd thanks for reporting this. This was indeed fixed in v1.30.1, but unfortunately it did not make it into brew yet because of some CI error in that PR. If you can download the release from GitHub, it might be worth a few minutes to get you unblocked.
https://github.com/MystenLabs/sui/releases/tag/testnet-v1.30.1


"~/" is rejected as relative path

~/ is not in the POSIX standard. It's totally a shell thing. You can create a directory with the name ~ just fine:
# cd /tmp
# mkdir '~'
# ls -la | grep '~'
drwxr-xr-x  2 foo users   4096 Mar 29 16:41 ~
# touch '~/foo'
# ls -la '~'
drwxr-xr-x  2 foo users  4096 Mar 29 16:42 .
drwxrwxrwt 21 root root  20480 Mar 29 16:41 ..
-rw-r--r--  1 foo users     0 Mar 29 16:42 foo
# cd '~'
# pwd
/tmp/~

A path library probably shouldn't do any shell-like stuff. ~/ is a valid relative path. If the user expects shell-like expansion of paths, then that has to be handled on another level by the application.
Note that System.FilePath has correct behavior here:
Prelude System.FilePath> isRelative "~/"
True

I agree its a bit odd to be aware of the "~" expansion at this level. Another discrepancy is that if we are disallowing "~/" then why not "~user/".

I agree. Let's lift that restriction.


[WIP][CORE] Optimize shuffle partitioner algorithm to take advantage of  auto-vectorization in GCC

What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
(Fixes: #ISSUE-ID)
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

/Benchmark Velox

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_3066_time.csv
log/native_master_09_06_2023_fb7f707e7_time.csv
difference
percentage

q1
43.54
43.16
-0.372
99.15%

q2
24.34
24.64
0.293
101.20%

q3
37.31
37.33
0.021
100.06%

q4
42.26
41.42
-0.834
98.03%

q5
69.54
70.22
0.679
100.98%

q6
6.36
5.16
-1.200
81.13%

q7
87.86
84.66
-3.198
96.36%

q8
81.12
81.38
0.267
100.33%

q9
118.67
114.35
-4.315
96.36%

q10
47.53
48.41
0.885
101.86%

q11
19.37
19.58
0.206
101.06%

q12
24.48
24.98
0.503
102.06%

q13
50.90
49.53
-1.365
97.32%

q14
15.97
16.77
0.805
105.04%

q15
29.32
30.10
0.774
102.64%

q16
15.95
16.20
0.254
101.59%

q17
120.34
120.43
0.088
100.07%

q18
163.36
164.70
1.341
100.82%

q19
12.19
12.30
0.113
100.93%

q20
32.07
31.16
-0.912
97.16%

q21
244.62
238.90
-5.712
97.67%

q22
15.68
15.29
-0.399
97.46%

total
1302.77
1290.69
-12.079
99.07%

supply some information about why we are making this change,  We made vs. benchmark in https://quick-bench.com/, and the result shows the speed of the improved Partitioner::computer is 2x faster than the old implement.

/Benchmark Velox

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_3066_time.csv
log/native_master_09_07_2023_192f5b03e_time.csv
difference
percentage

q1
43.67
43.44
-0.226
99.48%

q2
24.96
24.73
-0.224
99.10%

q3
36.75
37.30
0.553
101.51%

q4
40.01
41.95
1.944
104.86%

q5
70.07
70.09
0.018
100.03%

q6
6.65
6.54
-0.108
98.38%

q7
85.53
84.15
-1.381
98.39%

q8
81.16
76.28
-4.881
93.99%

q9
114.37
116.72
2.355
102.06%

q10
48.89
42.30
-6.593
86.52%

q11
18.82
19.70
0.878
104.66%

q12
23.78
25.74
1.965
108.26%

q13
51.01
49.36
-1.652
96.76%

q14
14.60
16.10
1.502
110.29%

q15
28.83
31.19
2.355
108.17%

q16
16.49
15.34
-1.150
93.03%

q17
120.04
121.01
0.961
100.80%

q18
162.42
160.47
-1.952
98.80%

q19
12.39
11.95
-0.444
96.42%

q20
29.73
25.56
-4.176
85.95%

q21
236.72
233.87
-2.853
98.79%

q22
16.02
15.38
-0.644
95.98%

total
1282.92
1269.17
-13.751
98.93%

/Benchmark Velox

Just a small issue in merging code. Let's remove [WIP] in final commit message next time. Thanks!

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_3066_time.csv
log/native_master_09_07_2023_192f5b03e_time.csv
difference
percentage

q1
43.64
43.44
-0.192
99.56%

q2
23.25
24.73
1.486
106.39%

q3
36.90
37.30
0.400
101.08%

q4
41.76
41.95
0.190
100.46%

q5
67.26
70.09
2.828
104.20%

q6
6.41
6.54
0.131
102.04%

q7
84.44
84.15
-0.287
99.66%

q8
80.48
76.28
-4.209
94.77%

q9
115.23
116.72
1.491
101.29%

q10
48.11
42.30
-5.814
87.92%

q11
20.70
19.70
-0.998
95.18%

q12
24.25
25.74
1.498
106.18%

q13
48.99
49.36
0.373
100.76%

q14
18.98
16.10
-2.882
84.82%

q15
32.06
31.19
-0.869
97.29%

q16
15.66
15.34
-0.327
97.91%

q17
121.09
121.01
-0.082
99.93%

q18
160.81
160.47
-0.333
99.79%

q19
12.00
11.95
-0.053
99.56%

q20
30.54
25.56
-4.980
83.69%

q21
245.29
233.87
-11.419
95.34%

q22
15.49
15.38
-0.113
99.27%

total
1293.33
1269.17
-24.161
98.13%


Select end if root has no children in focus logic

It shouldn't matter if the editor is empty when called editor.focus. The cursor should still appear somewhere in the editor when the method is called.

I'm would dare to say this was intentional but I can't recall why..

I looked through the blame history but didn't see anything. I guess we'll see!

I'm would dare to say this was intentional but I can't recall why..

I looked through the blame history but didn't see anything. I guess we'll see!

I think this is why:
https://github.com/facebook/lexical/actions/runs/3322384417/jobs/5492978426
collab tests on webkit failing starting with this diff.


Add an additional conclusion about inference systems

FEATURE: https://github.com/Xalgorithms/interlibr/issues/47
In (), we make some conclusions about what Interlibr is and does. In the document conclusion, we talk about some of the categories, but we don't make any comments about inference systems. Since these are a big part of the landscape, there should be a paragraph in the conclusion about them.

And machine learning. Data aggregation.
William Olders
613-762-9383
On Thu, Oct 25, 2018, 9:39 AM Don Kelly<EMAIL_ADDRESS>wrote:

FEATURE: Xalgorithms/interlibr#47
https://github.com/Xalgorithms/interlibr/issues/47
In (), we make some conclusions about what Interlibr is and does. In
the document conclusion, we talk about some of the categories, but we don't
make any comments about inference systems. Since these are a big part of
the landscape, there should be a paragraph in the conclusion about them.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Xalgorithms/general-documentation/issues/45, or mute
the thread
https://github.com/notifications/unsubscribe-auth/Ag8dAVRW5Zqg-22kYJTHtURBmFep5x6Iks5uob71gaJpZM4X6TOG
.


Core dumped while running apt-cache inside a container

BUG REPORT INFORMATION
Output of docker version:
Client:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:40:35 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:40:35 2016
 OS/Arch:      linux/amd64

Output of docker info:
Containers: 3
 Running: 1
 Paused: 0command
 Stopped: 2
Images: 58
Server Version: 1.10.2
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 64
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 4.2.0-22-generic
Operating System: Ubuntu 15.10
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.679 GiB
Name: jcaden
ID: MQV6:XORS:4QH3:7ICR:67OH:64HF:CA3O:OZJH:XRGL:PJ7N:JNDY:BAI3
WARNING: No swap limit support

Provide additional environment details (AWS, VirtualBox, physical, etc.):
I'm running docker on a laptop, but I think that this problem is also present when running in openstack with a different version of host OS.
List the steps to reproduce the issue:

docker run -t -i ubuntu:trusty /bin/bash
2.apt-cache --no-generate show a

Describe the results you received:
Segmentation fault (core dumped)
Describe the results you expected:
N: Unable to locate package a
E: No packages found
Provide additional info you think is important:
This command is running on a ubuntu trusty computer (physical or virtual)
This is the backtrace got from coredump:
Core was generated by `apt-cache --no-generate show a'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f79f98bd190 in pkgCache::FindGrp(std::string const&) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
(gdb) bt
#0  0x00007f79f98bd190 in pkgCache::FindGrp(std::string const&) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
#1  0x00007f79f99422ba in APT::PackageContainerInterface::FromGroup(APT::PackageContainerInterface*, pkgCacheFile&, std::string, APT::CacheSetHelper&) ()
   from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
#2  0x00007f79f99435b8 in APT::PackageContainerInterface::FromString(APT::PackageContainerInterface*, pkgCacheFile&, std::string const&, APT::CacheSetHelper&) ()
   from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
#3  0x00007f79f9944691 in APT::VersionContainerInterface::FromString(APT::VersionContainerInterface*, pkgCacheFile&, std::string, APT::VersionContainerInterface::Version const&, APT::CacheSetHelper&, bool) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
#4  0x00007f79f9945c4e in APT::VersionContainerInterface::FromCommandLine(APT::VersionContainerInterface*, pkgCacheFile&, char const**, APT::VersionContainerInterface::Version const&, APT::CacheSetHelper&) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
#5  0x000000000040a81d in ?? ()
#6  0x00007f79f98aa072 in CommandLine::DispatchArg(CommandLine::Dispatch*, bool) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
#7  0x0000000000407be3 in ?? ()
#8  0x00007f79f8d6fec5 in __libc_start_main (main=0x407a40, argc=4, argv=0x7fff6e6a01e8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff6e6a01d8)
    at libc-start.c:287
#9  0x0000000000407d8f in ?? ()

Sorry I sent it twice


Game crash on apply setting.

Description
The game would crash when try to apply the setting changes.
How To Reproduce
Steps to reproduce the behaviour:

Go to "Setting"
Click on 'Apply"
See error

Expected Behaviour
The game would crash upon clicking Apply regardless of changing the settings or not.
Screenshots

Additional Context
None
Temporary Fix
None

Are you still getting this? If so, can you share the full stack trace

Ah, I found the culprit. Window blocked write access to that specific folder due to my specific security setup.

Ah, I found the culprit. Window blocked write access to that specific folder due to my specific security setup.

Is this just something you had on your computer or could this same thing happen on other Windows computers


imagine process doesn't stop even after process is completed on discord

the code that reproduces this issue or a replay of the bug
the issue is client.imagine keeps on running even after the img has been developed on discord because of that i can't get the image url
Describe the bug
Describe the bug
A clear and concise description of what the bug is.
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
error log
PRESENCE_UPDATE 2023-09-24T09:02:39.218Z
heartbeat 1 2023-09-24T09:02:48.652Z
heartbeat 2 2023-09-24T09:03:28.654Z
heartbeat 3 2023-09-24T09:04:08.655Z
event PRESENCE_UPDATE 2023-09-24T09:04:39.585Z
heartbeat 4 2023-09-24T09:04:48.655Z
event PASSIVE_UPDATE_V1 2023-09-24T09:05:25.652Z
heartbeat 5 2023-09-24T09:05:28.659Z
heartbeat 6 2023-09-24T09:06:08.661Z
event PRESENCE_UPDATE 2023-09-24T09:06:38.174Z
heartbeat 7 2023-09-24T09:06:48.664Z

Yeah i have the same error, before it was working well...


HealthMate

there is a registe page after putting data into it we come to home page. home page have 8 health parameters at right side. a ai healthmate chatbot at left. at  botton there is home, analytics, profile, the health parameters have add new data button which can take new input and create a graph at analytics page for each of the data updated. the graph shoukld be time series data. the profile should be a basic obe with all data and few health motivation lines

check it out


Support Exact Match with Static Directive

I use Caddy as a static file server in front of a few different backend servers including a php-fpm, django, and wordpress server.
the status directive has syntax status code path, where the docs say

path is the base request path to match.

I am running Caddy containers in Google Kubernetes Engine, and the default health check for web services is that a request to / returns a 200-399 response.  I have some containers that only handle traffic at /some-path/*.  status 200 / returns an empty-response 200 for all paths including /some-path/foo.  If there was a way to specify that this is an exact match, the status directive would nicely answer health checks.

The status package uses a httpserver.ConfigSelector, so this should be doable by giving Rule a custom Match function, right?
Incidentally, a workaround in the meantime - since rewrite executes before status - is to rewrite under the exact circumstances you want, to some unlikely URI (e.g. /rewrite-status-200) and then use status 200 /rewrite-status-200. Rewriting on an exact path can be done with if {path} is /.

Thanks for the workaround suggestion.  I solved it by just dropping an empty index.html file in my root.

I am looking into writing an ExactMatcher (to put next to the existing PathMatcher?), and then using that instead of the PathMatcher whenever an exact match is required.
How do we want to signify the need for an exact match? I was thinking status 404 /secrets exact or status 404 { /secrets exact }?

I don't have a strong opinion on the syntax right now, but does Caddy allow for relative paths in the status directive?  Hopefully not, but if so, I'm not sure how your syntax would work.  It seems it would cause a collision with someone wanting to literally match the path exact.
If there is any such collision, I think requiring an absolute path and using your syntax is clean.  I can't imagine a real-world use case for the problematic relative paths I'm describing, but I know removing a feature is not something we could take lightly.

It wouldn't collide. The word exact would need to be the last argument on each line and would require each previous argument to be present, e.g. status 404 exact exact. Although, that would never work, as r.URL.Path should never equal exact, only perhaps /exact.

Alternately, https://github.com/mholt/caddy/pull/1948 would provide the required capabilities, and even uses status as an example.

Your syntax is definitely cleaner, but I agree #1948 solves this issue as well.
As I said before, I'm not too picky about the syntax as long as the feature is supported one way or another.  If #1948 gets merged in, I think this issue could be closed.

This should be a non-issue in Caddy 2.


getting error:Target of URI doesn't exist: 'package:intl/intl.dart', though I have added   intl: ^0.15.8 in pubspec.yaml

@shweta30sharma
Did you run flutter pub get after adding that dependency ?
Also please ask support questions on stackoverflow. Closing from here as this seems to be  an implementation issue.


Reading files from code

If I mount a file system directory, open a notebook in that context, and then try to read a file in that context, e.g. using pandas I get a file not found error.
Is there a way of setting a path either to a file in the file system directory, or a path to a file in the default file system (the browser file directory context, not the shared file-system-access context) when working on a notebook in the shared file-system context?

Thanks for opening an issue.
Are you talking about a JupyterLab context or a JupyterLite context?

In the case of jupyterlite, you will be able to read and write into the "mounted" drive, because we also mount the drive in the Emscripten context (at least concerning jupyterlite-xeus's python kernel or jupyterlite-pyodide).
In the case of jupyterlab, it will not work. The Python kernel executes its code on the server or an another external machine and it does not know anything about what happens in the front-end. jupyterlab-filesystem-access being a pure front-end package it does not communicate anything to the Python kernel about the mounted directory.
I don't know if we can easily workaround this, but it's surely a difficult problem: jupyterlab is multi-client, what do you do if two users mount a local directory? Should we communicate their files to the server for the Python kernel to read and write? I don't think there is a good (and safe) solution to this problem.

In Jupyterlite with Pyodide kernel, I tried to access the mounted files.
def show_file_contents(file_path):
    try:
        with open(file_path, 'r', encoding='utf-8') as file:
            contents = file.read()
            print(contents)
    except FileNotFoundError:
        print(f"File not found: {file_path}")
    except Exception as e:
        print(f"An error occurred: {e}")

file_path = "FileSystemAccess:client-patch.py"

But an issue happened that "File not found: FileSystemAccess:client-patch.py". Actually I copied the path from the menu item "Copy Path" in the Jupyterlite file browser.
Maybe there are some conventions to refer the local files.
Please clarify.
Thanks,

Have you tried simply:
def show_file_contents(file_path):
    try:
        with open(file_path, 'r', encoding='utf-8') as file:
            contents = file.read()
            print(contents)
    except FileNotFoundError:
        print(f"File not found: {file_path}")
    except Exception as e:
        print(f"An error occurred: {e}")

file_path = "client-patch.py"

I tried before and I failed also.
Because there is another built-in filesystem, which is based on IndexedDB, so there must be some conventions for the code to know it is a local file instead of an IndexedDB file.

Because there is another built-in filesystem, which is based on IndexedDB, so there must be some conventions for the code to know it is a local file instead of an IndexedDB file.

Similar to classic usage, it depends on where your Notebook lives. If your notebook lives in the directory exposed by jupyterlab-filesystem-access, it has access to local files there. If it lives in the IndexedDB file system, it has access to files on the indexedDB

I expected a local file system can be accessed from any code after an approval after I read https://pyodide.org/en/stable/usage/file-system.html#experimental-using-the-native-file-system-in-the-browser.
Now I understand the underlying file system is the same as that for the notebook and I tested successfully.
Thanks,

I tried the JupyerLite demo of this repo in Chrome and allowed access to a local folder. While I was able to see all files in the sidebar, opening them from a pyodide kernel notebook didn’t work. Where is the local file system mounted?

opening them from a pyodide kernel notebook didn’t work

Where was the notebook? If it's not in the local folder it does not have access to the local folder

I tried it with a notebook from the local folder and that didn't work either, the current working dir was still the general JupyterLite drive. When I tried to open a notebook in a subdirectory of the local folder, launching the kernel failed since that folder was not available in the filesystem.
Interestingly, the behaviour you describe does work with the jupyterlab-github extension, where opening a notebook within the readonly repo allows that notebook access to the repo folder (though seemingly in a sub-filesystem in the middle of nowhere since its current path is just "." and listing the contents of ".." lists nothing and listing the "/drive" also lists nothing even though the main JupyterLite drive does contain files).
I think I had expected that the JupyterLite drive would be mounted at "/drive" and work regardless and that other drives like the file system access one would also be mounted at public paths within the kernel FS, e.g. "/drive-local"


Broken index after (or during) the deletion of another index

My Environment

ArangoDB Version:        3.7.10
Storage Engine:             RocksDB
Deployment Mode:       Single Server
Deployment Strategy:   Manual Start
Configuration:
Infrastructure:               own
Operating System:        Ubuntu 18.04
Total RAM in your machine:        128Gb
Disks in use:        HDD
Used Package:              Ubuntu .deb

Component, Query & Data
Affected feature:
Query using index vs. query not using index
AQL query (if applicable):
Query 1 (with index)
for x in z_search
	filter x.clean == 'hedera helix'
	limit 100
	return x

Get 1 result
Query 2 (no index)
for x in z_search
    filter contains(x.clean, 'hedera helix', true) == 0
    filter length(x.clean) == 12
    limit 100
    return x

Get 38 results
AQL explain and/or profile (if applicable):
Query 1
Execution plan:
 Id   NodeType        Est.   Comment
  1   SingletonNode      1   * ROOT
  7   IndexNode          1     - FOR x IN z_search   /* skiplist index scan */
  5   LimitNode          1       - LIMIT 0, 100
  6   ReturnNode         1       - RETURN x

Indexes used:
 By   Name              Type       Collection             Unique   Sparse   Selectivity   Fields        Ranges
  7   idx_11016979962   skiplist   z_search   false    false        74.11 %   [ `clean` ]   (x.`clean` == "hedera helix")

Optimization rules applied:
 Id   RuleName
  1   use-indexes
  2   remove-filter-covered-by-index
  3   remove-unnecessary-calculations-2

Query 2
Execution plan:
 Id   NodeType                     Est.   Comment
  1   SingletonNode                   1   * ROOT
  2   EnumerateCollectionNode   4547685     - FOR x IN z_search   /* full collection scan */   FILTER ((CONTAINS(x.`clean`, "hedera helix", true) == 0) && (LENGTH(x.`clean`) == 12))   /* early pruning */
  7   LimitNode                     100       - LIMIT 0, 100
  8   ReturnNode                    100       - RETURN x

Indexes used:
 none

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   move-calculations-up-2
  4   move-filters-up-2
  5   fuse-filters
  6   move-filters-into-enumerate

Dataset:
Collection with 4.5 millions documents.
Exemples:
{
	"_key": "10963007718",
	"_id": "z_search/10963007718",
	"_rev": "_YAqOWJK--_",
	"id": "Hedera helix L.",
	"referential": "usa48_sync_v1",
	"type": "sp_woa",
	"illustrated": true,
	"original": "Hedera helix",
	"clean": "hedera helix",
	"genus_without_author": "Hedera",
	"family_without_author": "Araliaceae",
	"updated": true
},
{
	"_key": "10960998656",
	"_id": "z_search/10960998656",
	"_rev": "_YAqsvqW--_",
	"id": "Hedera helix L.",
	"referential": "canada_sync_v1",
	"type": "sp_woa",
	"illustrated": true,
	"original": "Hedera helix",
	"clean": "hedera helix",
	"genus_without_author": "Hedera",
	"family_without_author": "Araliaceae",
	"updated": true
}

Steps to reproduce

Drop an index in another collection (millions of documents)
Run query1 and query 2

Note: I don't know if the deletion of the index is still in progress.
Problem:
Queries using index or not don't return coherent results
Expected result:
Get the same results using index or not

Note: it's the same once the index is deleted and also after restarting ArangoDB.
Best

Hello ArangoDB team,
I confirm that by removing the index I can find all the expected results by running query1.

Hi,
Any guess about what could have happened ?
I can confirm that an index in a given collection got corrupted while another index was removed in another collection. Not sure there is a connection, but that would be a weird coincidence.
This makes us a little uncomfortable with the idea of removing indexes on a production server, although it is something we should do one of these days.
Thank you for any suggestion,
All the best

Hi,
It just happened again after deleting a collection (which had indexes).
An index (maybe several indexes, not tested yet) does not return all the data.
Query using the index:
FOR o IN z_obs
	FILTER o.project_id == 'the-plant-list'
	SORT o.date_created desc
	collect with count into c return c

=> 1
Query String (118 chars, cacheable: true):
 FOR o IN z_obs
 	FILTER o.project_id == 'the-plant-list'
 	SORT o.date_created desc
 	collect with count into c return c

Execution plan:
 Id   NodeType          Est.   Comment
  1   SingletonNode        1   * ROOT
  9   IndexNode       152497     - FOR o IN z_obs   /* reverse persistent index scan, scan only */
  7   CollectNode          1       - COLLECT  WITH COUNT INTO c   /* count */
  8   ReturnNode           1       - RETURN c

Indexes used:
 By   Name              Type         Collection   Unique   Sparse   Selectivity   Fields                             Ranges
  9   idx_11016981662   persistent   z_obs        false    false         9.14 %   [ `project_id`, `date_created` ]   (o.`project_id` == "the-plant-list")

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   move-calculations-up-2
  4   move-filters-up-2
  5   use-indexes
  6   remove-filter-covered-by-index
  7   use-index-for-sort
  8   remove-unnecessary-calculations-2

Query NOT using the index:
FOR o IN z_obs
	FILTER o.project_id == 'the-plant-list'
	//SORT o.date_created desc
	collect with count into c return c

=> 11913
Query String (120 chars, cacheable: true):
 FOR o IN z_obs
 	FILTER o.project_id == 'the-plant-list'
 	//SORT o.date_created desc
 	collect with count into c return c

Execution plan:
 Id   NodeType         Est.   Comment
  1   SingletonNode       1   * ROOT
  7   IndexNode       92422     - FOR o IN z_obs   /* hash index scan, scan only */
  5   CollectNode         1       - COLLECT  WITH COUNT INTO c   /* count */
  6   ReturnNode          1       - RETURN c

Indexes used:
 By   Name              Type   Collection   Unique   Sparse   Selectivity   Fields             Ranges
  7   idx_11016980876   hash   z_obs        false    false         0.00 %   [ `project_id` ]   (o.`project_id` == "the-plant-list")

Optimization rules applied:
 Id   RuleName
  1   use-indexes
  2   remove-filter-covered-by-index
  3   remove-unnecessary-calculations-2

@AntoineAA @matcho: no guess why this should happen.
If it happens again and queries don't return the expected results from the index, can you run the commands db.<collection>.properties(), db..count(), db.<collection>.indexes() and db.<collection>.figures(true) on the affected collection and share the results here?
That will show how many entries will be in the collection and the indexes for it as far as the RocksDB storage engine is concerned.
I can't say yet if that is enough to figure out the reason, but it could at least help narrow it down.
Thanks!

Hi @jsteemann , thank you for your answer.
Nothing in the log messages, just slow queries warnings, although log level is set to DEBUG.
If it happens again we'll run the commands you suggest.

Hi @jsteemann,
The second time, I did not recreate the index to keep it in its inconsistent state.  The figures reported clearly show that there is an issue.
The following indexes seem to have an off-count value:
  {
    "deduplicate" : false,
    "fields" : [
      "project_id",
      "date_created"
    ],
    "id" : "z_obs/11016981662",
    "name" : "idx_11016981662",
    "selectivityEstimate" : 0.09192590003495281,
    "sparse" : false,
    "type" : "persistent",
    "unique" : false
  },
...
  {
    "deduplicate" : false,
    "fields" : [
      "author.name"
    ],
    "id" : "z_obs/11016982469",
    "name" : "idx_11016982469",
    "selectivityEstimate" : 0.11957830519358896,
    "sparse" : false,
    "type" : "hash",
    "unique" : false
  },

The reported count for index<PHONE_NUMBER>2 is just 316, but it should be 3049944.
The reported count for index<PHONE_NUMBER>9 is just 1361793, but it should be 3049944.
I can't tell if any sparse indexes or array indexes are affected, because they may contain less (sparse) or more (array indexes) index entries than documents.

No, there were no index additions or deletions in progress, nor collection being deleted.
We have not worked on this database since my message 6 days ago to keep it in an incoherent state to be able to give you the necessary information.

Quick update: problem did not occur for me yet.
Now trying some more variations to see if the problem is tied to a very specific condition.

Tried with v3.7.9 and more than 200 concurrent runs of creating and dropping random indexes on 2 different collections, and after each run checking the consistency across all indexes.
This did not uncover any issue yet. So I am wondering what special case I may be missing.
Is this problem easily reproducible for you in a development environment?

Thank you for testing.
We don't know if we can reproduce it because we notice it a few days later.
We have deleted object.field and _indexname name pattern indexes and on each concerned collection (4) we have this issue.
Examples: missing.email, missing, _couchid.
Some indexes had the same name (different collections).

Ok, thanks, if you could provide me the exact index information (i.e. type, attribute names, uniqueness, sparseness, ideally some example attribute data) for at least one of the affected indexes that may help. Plus, if you are dropping it by name or id.

We still have the indexes on the production database (we did not dare to delete them).

Here are examples data within the collection.
[
  {
    "_key": "10009394129",
    "_id": "votes/10009394129",
    "_rev": "_X2mI2N---_",
    "date":<PHONE_NUMBER>873,
    "generated": true,
    "image_id": null,
    "observation_id": "1000723727",
    "type": "determination",
    "user_id": "100468472",
    "value": "Rhododendron maximum L."
  },
  {
    "_key": "10009391098",
    "_id": "votes/10009391098",
    "_rev": "_X2l4iN---_",
    "date":<PHONE_NUMBER>141,
    "generated": true,
    "image_id": "fa565a4126414adecad460f873d6188c1aebcb32",
    "observation_id": "1000425168",
    "type": "quality",
    "user_id": "100068404",
    "value": true
  },
  {
    "_key": "10009361886",
    "_id": "votes/10009361886",
    "_rev": "_X2lIiN---_",
    "date":<PHONE_NUMBER>181,
    "generated": true,
    "image_id": null,
    "observation_id": "1002406962",
    "type": "determination",
    "user_id": "100119726",
    "value": "Juncus effusus L."
  },
  {
    "_key": "10009311613",
    "_id": "votes/10009311613",
    "_rev": "_X2kyaN---_",
    "date":<PHONE_NUMBER>479,
    "generated": true,
    "image_id": "e218edf989f098788300f5097faf29fbbf811d2e",
    "observation_id": "1002701563",
    "type": "quality",
    "user_id": "100091529",
    "value": true
  },
  {
    "_key": "10009349058",
    "_id": "votes/10009349058",
    "_rev": "_X2lCON---_",
    "date":<PHONE_NUMBER>134,
    "generated": true,
    "image_id": "e4969f7cbe318132411e7d86287dc2677371ca33",
    "observation_id": "1000980833",
    "type": "organ",
    "user_id": "100225703",
    "value": "flower"
  },
  {
    "_key": "10009372269",
    "_id": "votes/10009372269",
    "_rev": "_X2lOaN---_",
    "date":<PHONE_NUMBER>171,
    "generated": true,
    "image_id": "346a9122e232ae524a1d30c935b7c2307e1c60b2",
    "observation_id": "1000425298",
    "type": "quality",
    "user_id": "100095211",
    "value": true
  },
  {
    "_key": "10009371349",
    "_id": "votes/10009371349",
    "_rev": "_X2lO-N---_",
    "date":<PHONE_NUMBER>400,
    "generated": true,
    "image_id": "4b00498eb589801f55199239ec8f292654fa31bc",
    "observation_id": "1001123720",
    "type": "quality",
    "user_id": "100127292",
    "value": true
  },
  {
    "_key": "10009324139",
    "_id": "votes/10009324139",
    "_rev": "_X2k3CN---_",
    "date":<PHONE_NUMBER>646,
    "generated": true,
    "image_id": "73961e744f591d15fb664b87a40349a920d2c6e1",
    "observation_id": "1002846884",
    "type": "quality",
    "user_id": "100002869",
    "value": true
  },
  {
    "_key": "10009365070",
    "_id": "votes/10009365070",
    "_rev": "_X2lJeM---_",
    "date":<PHONE_NUMBER>661,
    "generated": true,
    "image_id": null,
    "observation_id": "1002073031",
    "type": "determination",
    "user_id": "100079009",
    "value": "Muscari comosum (L.) Mill."

  },
  {
    "_key": "10009347784",
    "_id": "votes/10009347784",
    "_rev": "_X2lAyM---_",
    "date":<PHONE_NUMBER>055,
    "generated": true,
    "image_id": "e51b0f5e98ca9f6123476bb44232baf750416015",
    "observation_id": "1002226548",
    "type": "organ",
    "user_id": "100599128",
    "value": "flower"
  },
  {
    "_key": "10009347370",
    "_id": "votes/10009347370",
    "_rev": "_X2lAqM---_",
    "date":<PHONE_NUMBER>006,
    "generated": true,
    "image_id": "4e7cc7de0ed2a22dfcebe0b022b038972452b227",
    "observation_id": "1002318395",
    "type": "quality",
    "user_id": "100599128",
    "value": true
  },
  {
    "_key": "10009340176",
    "_id": "votes/10009340176",
    "_rev": "_X2k82N---_",
    "date":<PHONE_NUMBER>812,
    "generated": true,
    "image_id": "8a236175e50b2e20115540fbaf36f60a561ab459",
    "observation_id": "1002663242",
    "type": "quality",
    "user_id": "100002869",
    "value": true
  },
  {
    "_key": "10009373058",
    "_id": "votes/10009373058",
    "_rev": "_X2lQOM---_",
    "date":<PHONE_NUMBER>555,
    "generated": true,
    "image_id": "cc4180197e97e494c2f9c40acb4324002259aa8a",
    "observation_id": "1001414038",
    "type": "quality",
    "user_id": "100468552",
    "value": true
  },
  {
    "_key": "10009392447",
    "_id": "votes/10009392447",
    "_rev": "_X2m_eM---_",
    "date":<PHONE_NUMBER>630,
    "generated": true,
    "image_id": "c673a9232e5bdf343e47ce41bc63a3f66c7328f7",
    "observation_id": "1001765135",
    "type": "organ",
    "user_id": "100548432",
    "value": "leaf"
  },
  {
    "_key": "10009382781",
    "_id": "votes/10009382781",
    "_rev": "_X2lfGM---_",
    "date":<PHONE_NUMBER>471,
    "generated": true,
    "image_id": "cc3be8c51ea6c8b784218a69d47ed61b34dfc37a",
    "observation_id": "1001820447",
    "type": "quality",
    "user_id": "100474696",
    "value": true
  },
  {
    "_key": "10009372268",
    "_id": "votes/10009372268",
    "_rev": "_X2lOaM---_",
    "date":<PHONE_NUMBER>171,
    "generated": true,
    "image_id": null,
    "observation_id": "1000425298",
    "type": "determination",
    "user_id": "100095211",
    "value": "Crepis aurea (L.) Tausch"
  },
  {
    "_key": "10009395204",
    "_id": "votes/10009395204",
    "_rev": "_X2mOSM---_",
    "date":<PHONE_NUMBER>465,
    "generated": true,
    "image_id": "0bd5a4113b20818b8a063fd272d2165642b65793",
    "observation_id": "1001386836",
    "type": "quality",
    "user_id": "100164123",
    "value": true
  },
  {
    "_key": "10009341988",
    "_id": "votes/10009341988",
    "_rev": "_X2k9mL---_",
    "date":<PHONE_NUMBER>990,
    "generated": true,
    "image_id": "e2acbee93cb5cdda0d0c2fa700d4adf6ebedd817",
    "observation_id": "1001421095",
    "type": "quality",
    "user_id": "100123516",
    "value": true
  },
  {
    "_key": "10009367837",
    "_id": "votes/10009367837",
    "_rev": "_X2lLeL---_",
    "date":<PHONE_NUMBER>318,
    "generated": true,
    "image_id": "f2139afab45accc8a7e340a8b7d0f42f3313da35",
    "observation_id": "1001424460",
    "type": "quality",
    "user_id": "100486762",
    "value": true
  }
]

The index that is no longer consistent is the one based on observation_id, user_id, image_id, type fields.

The index that is no longer consistent is the one based on date_created fields.

Hi,
Any news about this issue?
We would like to update some indexes on production database, but we don't know if it's safe.

@AntoineAA : no update here. I tried reproducing the issue locally for a few days, but without any luck.
What I can say for sure is that we did not make any change in recent 3.7 versions w.r.t. this.
So if this affects one of your 3.7.10 instances, I cannot be sure if won't affect another.
I am wondering what would be a good way of reproducing this issue locally and what ingredient could be missing in the script I tried (https://github.com/arangodb/arangodb/issues/13974#issuecomment-830301715). Do you have any idea what could be missing from my setup?

I believe we have upgraded from 3.4 to 3.7, and we use keys that start with _ inside documents.

I experienced a similar issue in production today, which I verified with the following query:
LET doc = DOCUMENT(@document_id)

LET matches = (
    FOR edge IN contains
        FILTER edge._to == doc._to
        RETURN edge
)

RETURN {document: doc, matches: matches}

(contains is the name of an edge collection that has, in addition to the builtin indexes, a unique index on _from and _to.)
This query returned a document, **but an empty array for ** matches. This signifies that, for some reason, the index was corrupted.
I went AFK for a while, and when I came back, the query now returned the expected result (a non-empty array in matches). In the server log, I saw the following entries logged in the time I was away:
2024-05-02T02:07:44Z [1] INFO [9bf16] {engines} auto flushing RocksDB wal and column families because number of live WAL files is 10
2024-05-02T02:09:20Z [1] WARNING [9123d] {engines} rocksdb has stopped incoming writes to column family 'Documents' to let background writes catch up


chore: release notes for v0.18.x

Also update INSTALL_GOOGLE_CLOUD_CPP_COMMON_FROM_SOURCE rules, and
google_cloud_cpp_common and google_cloud_cpp_grpc_utils ABI dumps.

This change is

CHANGELOG.md, line 12 at r1 (raw file):

* fix: make test not flaky (#143)
* test: implement unit tests for async RPCs (#142)
* chore: remove `CompletionQueue` parameter to `AsyncGrpcOperation::Notify` (#136)

this one is also a BREAKING CHANGE that I neglected to tag as such (and perhaps chore wasn't right either, although I don't know what to call it).  I agree with Carlos' comment below about putting these in their own section (there should be examples here or in other repos)


Use palette in default values.

I would like to make it so you can use the palette for default values.

The rest of this issue assumes you are okay with that...

To avoid a circular import, I would need to move all option related values to a function… extend_palette makes the most sense, they will all be in the same place to which I like.
The only part that's not straight forward is the reduced blue option.
I don't like how it works currently, (you can't change it back once it's set to reduced).
To fix that, I would need to save a reduced and normal option, I can't think of a clean way of doing that.

My best idea so far:
extend_colors could take the options as an argument, and it could then be call in config, there by setting the defaults.
I could then have something like palette.white_normal and palette.white_reduced_blue in colors/nordic.
However palette.white0 would not have a value until config is run (seems like a bad idea), unless I added something like palette.white0 = palette.white_normal which is setting two defaults (also feels like a bad idea).

Does anyone have any ideas or does that work?

There is no problem implementing this after #107.
Anything else on this point will be put in #111


Fix HttpWebRequest and HttpWebResponse

Description

HttpWebResponse was missing  the Close() method.
HttpWebRequest was initializing the input network stream wrongly for HTTPS connections.
Minor fixes in comments.

Motivation and Context

Fixes nanoFramework/Home#616.

How Has This Been Tested?

HTTP WebRequest sample.

Screenshots
Types of changes

[ ] Improvement (non-breaking change that improves a feature, code or algorithm)
[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Config and build (change in the configuration and build system, has no impact on code or features)
[ ] Dependencies (update dependencies and changes associated, has no impact on code or features)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Ignoring SonarCloud report on increase of vulnerabilities and code smells.
(to be fixed on a future PR)


Package Manager Support

Are you interested in adding support for package managers to this project?
Cocoapods/Carthage/Swift Package Manager
Even if you have no interest of making it work yourself, I will probably create a Podspec or similar for my own purposes. Would you be interested in merging this then?

Heya!
Yes, I would be interested; my only concern is historically when dealing with Pods for clients, is that over time a lot of dependencies do not keep up with Apple's deprecations (or if they were relying on bugs in the Apple SDKs, start acting erratically). Fortunately, I have no dependencies...
Maintaining a package in these systems adds overhead to maintaining the library in general. A Podspec I could probably deal with, but any future issues people have with it, I may punt to you. :)

I understand and agree completely. Let's see what I come up with :)


Fix \frefpart

The hyperref generated from \frefpart was empty because of a typo. I don't know it really deserves a PR, but I did it for my project so why not.

Thanks!


feat: support configurable chunk size

Adds an environment variable to configure chunk size when pushing monitors to Kibana.
Testing

Create a test project
Check out this PR. Run npm install then npm run build.
Run npm link
Check out the directory for the test project
run npm link @elastic/synthetics
Run CHUNK_SIZE=1 SYNTHETICS_API_KEY=MXhmQURwRUI1UGtRakt5TW5zWk46VWtFTWJrbE9RMjZTdFBrRVpFd190Zw== npm run push
Watch as only 1 monitor is pushed at a time

I think its because I pushed a change.

@dominiqueclarke If you can squash my commit along with yours and remove mine using rebase, We can get this merged faster.

@vigneshshanmugam are you the only one who can approve PRs for merge in this repo?


Can WSL2 use Windows network proxy, or access Windows services easily?

Due to the restricted network environment, I need to use a proxy service. If a static IP address is used in Windows, WSL2 can easily access services on Windows; but if a dynamic IP address is used in Windows, WSL2 needs to change the configuration every time.
I wish WSL2 could simply access services in Windows. In addition, I also hope that the IP of WSL2 can be static. Is there any good way?
Thanks!

Should probably be a discussion item rather than an Issue ;-).

If a static IP address is used in Windows, WSL2 can easily access services on Windows; but if a dynamic IP address is used in Windows, WSL2 needs to change the configuration every time.
I wish WSL2 could simply access services in Windows

WSL provides an mDNS name for the Windows host.  See this Stack Overflow answer for details, but to summarize, the Windows host name, with .local appended should give you a "fixed" name that you can use to address services in Windows, even when the dynamic IP changes.
But do take note of the other information in my Stack Overflow answer about the service binding rules as well as possible firewall issues that need to be addressed.

I also hope that the IP of WSL2 can be static

I still need to try this at some point myself, but see this blog post (not mine) for a possibility.  I'm not sure if that will work for providing a static address or not.
It should also provide localhost access to the Windows side of things from inside WSL2, meaning an mDNS name wouldn't be needed if so.

@NotTheDr01ds
The methods described in this blog post may become invalid with updates. I think this need is pretty common, but there is no official way to provide a method, which confuses me a lot.


WIP: Update gnunet, multiuser nixos module

Changelog: https://git.gnunet.org/gnunet.git/tree/ChangeLog?h=v0.19.4## Description of changes
I'm opening this PR as a draft so it will be easier to find if useful for others.
GNUnet multiuser does not support the fs subsystem, which makes it not very reasonable to use as default.
This isn't looking for review or feedback. and should not be merged

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

https://bugs.gnunet.org/view.php?id=7355 is the issue for multiuser gnunet-fs


Implement np.take and ndarray.take

This implements np.take and ndarray.take for basic integer based
indexing using ndarrays as the index, it also supports scalar and
single indexed lists as indexes.
There is a slight inefficiency in having to copy the index array
if it is F ordered to ensure C ordering for walking with nditer,
this is due to nditer not supporting flags at present.
Closes #2466.

Codecov Report

Merging #2490 into master will decrease coverage by 0.02%.
The diff coverage is 76.19%.

@@            Coverage Diff             @@
##           master    #2490      +/-   ##
==========================================
- Coverage   86.56%   86.54%   -0.03%
==========================================
  Files         316      316
  Lines       61312    61458     +146
  Branches     6751     6772      +21
==========================================
+ Hits        53077    53188     +111
- Misses       7167     7198      +31
- Partials     1068     1072       +4

Looks great!  Thanks!


anchor-lang dependency issue preventing compile

I am running into an issue compiling this and I am not the only one, as I have found it elsewhere recently. IT has to do with a jacked version 24, which is preventing it from using 0.24.2 as a suitable dependency. I believe that this is a huge issue...
Updating crates.io index
Installing metaboss v0.6.1
error: failed to compile metaboss v0.6.1, intermediate artifacts can be found at C:\Users\softw\AppData\Local\Temp\cargo-installJNQlbh
Caused by:
failed to select a version for the requirement anchor-lang = "^0.19.0"
candidate versions found which didn't match: 0.24.2
location searched: crates.io index
required by package anchor-client v0.19.0
... which satisfies dependency anchor-client = "^0.19.0" of package metaboss v0.6.1

I am running into an issue compiling this and I am not the only one, as I have found it elsewhere recently. IT has to do with a jacked version 24, which is preventing it from using 0.24.2 as a suitable dependency. I believe that this is a huge issue...
Updating crates.io index Installing metaboss v0.6.1 error: failed to compile metaboss v0.6.1, intermediate artifacts can be found at C:\Users\softw\AppData\Local\Temp\cargo-installJNQlbh
Caused by: failed to select a version for the requirement anchor-lang = "^0.19.0" candidate versions found which didn't match: 0.24.2 location searched: crates.io index required by package anchor-client v0.19.0 ... which satisfies dependency anchor-client = "^0.19.0" of package metaboss v0.6.1

Run it with the --locked option:
cargo install --locked metaboss
cargo build --locked --release
etc.
I've removed Anchor deps in v0.7.0 so we won't have this problem going forward.

Run it with the --locked option:
cargo install --locked metaboss cargo build --locked --release

Just want to report that this solved the issue for me. Thanks!


Initial commit

Sample codec with decode and encode blocks.

@untergeek big :+1:  for the new examples, I have just a strange feeling about the codec one, because it is in fact a wrapper around the line codec (missing the gem dependency btw ;P) might be confusing why you need another codec to build your own codec. Isn't plain codec already the simplest example possible, or am I missing the point ?

Ah, the hazards of having the line codec already in place.  Thanks for catching that.  Patch coming soon.
As far as the line codec goes, it's actually a little more confusing.  Yes, wrapping is done here, but that also demonstrates that you can use other codecs within a codec.  By not having to reinvent the wheel for the line-by-line ingestion, I can do a simple transform of a line.
Really, these examples are for people to get the directory structure, Gemfile, Rakefile and other necessary bits in place so it's easy to start from there.

@untergeek what is the status on this?


Implement postcondition evaluation against state sets

This is needed for postcondition-based checking, which is (I think) the expected form of checking in the current critical path.

Closed, as we actually just rely on litmus doing this for us.


Add iterative interpolation for splines in N-dimensions

Hi @f0uriest - this package does a lot of things that I need, thank you! Is this issue still in the to-do queue? I could really use a multilinear ND interpolator.
Thanks!

It's still on the list, but likely won't have time to do much on it in the near-medium time frame (weeks ~ months). If anyone else wants to take a stab I'd welcome a PR.

I hope you're doing well! I'm interested in using Interpax for my research project, and it has been incredibly helpful so far. I was wondering if there are any plans to include an ND interpolator in the next release. It would be a great addition to my work.


Don't throw error when max model len > max seq len

Override max model len when it would fail due to being larger than max seq len
The context here is that we upgraded our serving runtime to use 0.2.7 for the performance upgrades, but models that could previously be served by 0.2.5 are now failing due to this error. We had to downgrade to 0.2.6 (which does not include the error, introduced in https://github.com/vllm-project/vllm/commit/8041b7305e93a8626d85cb23b3fcb995882867c1) and are missing out on the performance upgrades.

Hi, could you provide some context for this PR? We prefer to explicitly raise errors to ensure that the user is aware of the issue, instead of silently/magically fixing it for them (even if we log - people miss those!), as that may lead to behavior unexpected by the user.

Hi, that makes sense. What if we introduce an argument like override_max_model_len that allows this behavious?
The context here is that we upgraded our serving runtime to use 0.2.7 for the performance upgrades, but models that could previously be served by 0.2.5 are now failing due to this error. We had to downgrade to 0.2.6 (which does not include the error, introduced in https://github.com/vllm-project/vllm/commit/8041b7305e93a8626d85cb23b3fcb995882867c1) and are missing out on the performance upgrades.

Hi, could you provide some context for this PR? We prefer to explicitly raise errors to ensure that the user is aware of the issue, instead of silently/magically fixing it for them (even if we log - people miss those!), as that may lead to behavior unexpected by the user.

@Yard1 yeah that's very fair. I have a slightly different opinion: I think that as context lengths of models scale up, it starts to make less sense to blanket reject launching the model on hardware that cannot support the full context length. For example, let's say we had a model with 200K context length, would we expect vllm's default behavior to be to throw unless we can allocate 200K tokens worth of KV cache on the node?
Imo, before 0.2.7, we were already doing something magical which is that we were inferring the max model length from the Huggingface config (https://github.com/vllm-project/vllm/blob/3dad94448583a835230ae68c726d4b67e46845f2/vllm/engine/arg_utils.py#L134-L138). To that end, I think modifying the behavior to throw by default if this is not achievable is kind of a breaking change - engines that launched in 0.2.5 cannot launch if upgraded to 0.2.7 (though I guess one could argue it's a bug fix since the model would have OOMed at high enough context length).
For us, in most cases, we aren't even configuring this value in the HuggingFace config themselves, and they may not care to use the full context length. What do you think about the following:

If the user explicitly sets --max-context-length, then we should respect that and throw if the hardware cannot support it
If the user does not explicitly set the value and it is inferred, then we log a warning that the full context length will not be supported (the server will also return an error if the context passed in is too long) and allow the launch?

Also, I will be at the meetup tomorrow - happy to discuss with you in person!


Support Shared vCPU (Arm64) HCloud server types

Goals
Support Shared vCPU (Arm64) Hcloud server types.
User Story
As a user I would like to provision the new Hetzner CAX machine types CAX11, CAX21, CAX31, and CAX41
Detailed Description
I'd like to be able to set the spec.template.spec.type of HCloudMachineTemplate to one of the values mentioned above.
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCloudMachineTemplate
metadata:
  name: my-cluster-control-plane
  namespace: default
spec:
  template:
    spec:
      imageName: ubuntu-22.04
      placementGroupName: control-plane
      type: cx31

Currently, doing so results the following error message
Error from server (Invalid): error when creating "my-clusteryaml": HCloudMachineTemplate.infrastructure.cluster.x-k8s.io "my-cluster-control-plane" is invalid: spec.template.spec.type: Unsupported value: "CAX11": supported values: "cpx11", "cx21", "cpx21", "cx31", "cpx31", "cx41", "cpx41", "cx51", "cpx51", "ccx11", "ccx12", "ccx21", "ccx22", "ccx31", "ccx32", "ccx41", "ccx42", "ccx51", "ccx52", "ccx62", "cax11", "cax21", "cax31", "cax41"

/kind proposal

okay, I see the issue has been about the case. See CAX11 vs cax11


Add contact to contact list

Is there a way to programmatically store a contact to whatsapp ?
i can check for contacts works fine but how do i added?
Thanks,
Victor

There is none. This package can do everything that the web.whatsapp.com webapp does. There is no method for web.whatsapp.com to add contacts.

Thanks.


[BP-3.2][FLINK-25828] Let build-stateful-functions.sh ignore javadoc and sources jar

This is a backport of #285 onto release-3.2.

Merging this PR since the original one has been reviewed.

Manually merged via 0d957c4113ad8b7fc2e4fafd08b90d93c8fe1566


Fix all eslint and Flow warnings, no behavior changes or new features

Automated message from Dropbox CLA bot
@borisyankov, it looks like you've already signed the Dropbox CLA. Thanks!


Firestore's get(/databases/) call on a sub-collection #firestore assignee: MilaGGL

testdevelop_security_rules.txt
Hi @MilaGGL
I am finding an unusual result when calling Firestore's get(/databases/) function for data that is two sub-collections deep.  For some reason, the data that is meant to be read isn't allowed, with Firestore confirming that the caller does not have permission to perform the operation.
To replicate the situation, please arrange Firestore collections with the following arrangement:
/users/
/userA/                                                                                /userB/
/signInData/   /otherDetails/   /exclusiveA/                                 /signInData/   /otherDetails/   /exclusiveB/
Please note: collections 'users', 'userA' and 'userB' are empty.
The contents of collections 'signInData', 'otherDetails', 'exclusiveA' and 'exclusiveB' are confirmed in the attached images, and, the 'userA' collection does not have any user-data but collection 'userB' does.
So, the get(/databases/) call is meant to permit the reading of documents in collection 'exclusiveA' if the boolean field-value of 'creditCard' in collection 'exclusiveB' is set to false.  This is where the failure is occurring.
The code querying my security rules is within file: stream_results.txt
The security that have been set are within file: testdevelop_security_rules.txt
Please share your feedback on this situation and whether there is a way that this problem can be worked around.  The main thing is the conditional reading of collection 'exclusiveA'.
Thank you for your time and for your assistance.
The Flutter code querying my Firestore security rules is:

testdevelop_security_rules.txt
stream_results.txt

@mathems-m
Can you provide flutter doctor -v, any logs that are generated including the permission error you mentioned ?

Hi
Thank you for your response.
The flutter doctor -v response is:
@.*** developer_testing % flutter doctor -v
[✓] Flutter (Channel stable, 3.0.5, on macOS 12.3.1 21E258 darwin-x64,
locale en-GB)
• Flutter version 3.0.5 at /Users/m.mathems/Developer/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision f1875d570e (10 months ago), 2022-07-13 11:24:16
-0700
• Engine revision e85ea0e79c
• Dart version 2.17.6
• DevTools version 2.12.2
[✓] Android toolchain - develop for Android devices (Android SDK version
33.0.0)
• Android SDK at /Users/m.mathems/Library/Android/sdk
• Platform android-33, build-tools 33.0.0
• ANDROID_HOME = /Users/m.mathems/Library/Android/sdk/
• Java binary at: /Users/m.mathems/Library/Application
Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/212.5712.43.2112.8609683/Android
Studio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build
11.0.12+0-b1504.28-7817840)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• CocoaPods version 1.12.0
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build
11.0.12+0-b1504.28-7817840)
[✓] Android Studio (version 2021.2)
• Android Studio at /Users/m.mathems/Library/Application
Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/212.5712.43.2112.8609683/Android
Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build
11.0.12+0-b1504.28-7817840)
[✓] IntelliJ IDEA Ultimate Edition (version 2022.2 EAP)
• IntelliJ at /Users/m.mathems/Applications/JetBrains Toolbox/IntelliJ
IDEA Ultimate.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] IntelliJ IDEA Ultimate Edition (version EAP IU-222.3153.4)
• IntelliJ at /Users/m.mathems/Library/Application
Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/222.3153.4/IntelliJ IDEA 2022.2
EAP.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] IntelliJ IDEA Ultimate Edition (version 2022.1.3)
• IntelliJ at /Users/m.mathems/Library/Application
Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/221.5921.17/IntelliJ IDEA.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] Connected device (7 available)
• sdk gphone64 x86 64 (mobile)        • emulator-5554
• android-x64    • Android 12 (API 32) (emulator)
• iPhone 12 Pro Max (mobile)          •
4BB077E1-56E6-468B-A35A-A8F533ACD849 • ios            •
com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• iPhone SE (3rd generation) (mobile) •
E0095356-C638-46CD-9601-C43A14EE4641 • ios            •
com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• iPad Pro (10.5-inch) (mobile)       •
67B76729-B225-4F02-86EE-380A7F0724FC • ios            •
com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• iPad mini (5th generation) (mobile) •
2975B3C2-C864-481A-8BBC-6C4B9752441F • ios            •
com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• macOS (desktop)                     • macos
• darwin-x64     • macOS 12.3.1 21E258 darwin-x64
• Chrome (web)                        • chrome
• web-javascript • Google Chrome 112.0.5615.137
[✓] HTTP Host Availability
• All required HTTP hosts are available
• No issues found!
The error message when I am using the iOS emulator is: flutter:
[cloud_firestore/permission-denied]
The caller does not have permission to execute the specified operation.
The error message when I am using the Android emulator is:  W/Firestore(24205):
(24.4.3) [Firestore]: Listen for
Query(target=Query(users/l9NzQFjN4iB0JlJaY3AI/userA/2VzSHur3RllcF5PojT61/exclusiveA
order by name);limitType=LIMIT_TO_FIRST) failed:
Status{code=PERMISSION_DENIED, description=Missing or insufficient
permissions., cause=null} D/EGL_emulation(24205): app_time_stats:
avg=15753.46ms min=383.96ms max=31122.96ms count=2
On Tue, 9 May 2023 at 10:43, darshankawar @.***> wrote:

@mathems-m https://github.com/mathems-m
Can you provide flutter doctor -v, any logs that are generated including
the permission error you mentioned ?
—
Reply to this email directly, view it on GitHub
https://github.com/firebase/flutterfire/issues/10926#issuecomment-1539743243,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AZJHPOLDR57OR4J7QRODF6DXFIGUTANCNFSM6AAAAAAX2KDPW4
.
You are receiving this because you were mentioned.Message ID:
@.***>

The flutter doctor -v response is:
m.mathems@Ms-MacBook-Pro developer_testing % flutter doctor -v
[✓] Flutter (Channel stable, 3.0.5, on macOS 12.3.1 21E258 darwin-x64, locale en-GB)
• Flutter version 3.0.5 at /Users/m.mathems/Developer/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision f1875d570e (10 months ago), 2022-07-13 11:24:16 -0700
• Engine revision e85ea0e79c
• Dart version 2.17.6
• DevTools version 2.12.2
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
• Android SDK at /Users/m.mathems/Library/Android/sdk
• Platform android-33, build-tools 33.0.0
• ANDROID_HOME = /Users/m.mathems/Library/Android/sdk/
• Java binary at: /Users/m.mathems/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/212.5712.43.2112.8609683/Android Studio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• CocoaPods version 1.12.0
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
[✓] Android Studio (version 2021.2)
• Android Studio at /Users/m.mathems/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/212.5712.43.2112.8609683/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
[✓] IntelliJ IDEA Ultimate Edition (version 2022.2 EAP)
• IntelliJ at /Users/m.mathems/Applications/JetBrains Toolbox/IntelliJ IDEA Ultimate.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] IntelliJ IDEA Ultimate Edition (version EAP IU-222.3153.4)
• IntelliJ at /Users/m.mathems/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/222.3153.4/IntelliJ IDEA 2022.2 EAP.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] IntelliJ IDEA Ultimate Edition (version 2022.1.3)
• IntelliJ at /Users/m.mathems/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/221.5921.17/IntelliJ IDEA.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] Connected device (7 available)
• sdk gphone64 x86 64 (mobile)        • emulator-5554                        • android-x64    • Android 12 (API 32) (emulator)
• iPhone 12 Pro Max (mobile)          • 4BB077E1-56E6-468B-A35A-A8F533ACD849 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• iPhone SE (3rd generation) (mobile) • E0095356-C638-46CD-9601-C43A14EE4641 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• iPad Pro (10.5-inch) (mobile)       • 67B76729-B225-4F02-86EE-380A7F0724FC • ios            • com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• iPad mini (5th generation) (mobile) • 2975B3C2-C864-481A-8BBC-6C4B9752441F • ios            • com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• macOS (desktop)                     • macos                                • darwin-x64     • macOS 12.3.1 21E258 darwin-x64
• Chrome (web)                        • chrome                               • web-javascript • Google Chrome 112.0.5615.137
[✓] HTTP Host Availability
• All required HTTP hosts are available
• No issues found!
The error message when I am using the iOS emulator is: flutter: [cloud_firestore/permission-denied] The caller does not have permission to execute the specified operation.
The error message when I am using the Android emulator is:  W/Firestore(24205): (24.4.3) [Firestore]: Listen for Query(target=Query(users/l9NzQFjN4iB0JlJaY3AI/userA/2VzSHur3RllcF5PojT61/exclusiveA order by name);limitType=LIMIT_TO_FIRST) failed: Status{code=PERMISSION_DENIED, description=Missing or insufficient permissions., cause=null} D/EGL_emulation(24205): app_time_stats: avg=15753.46ms min=383.96ms max=31122.96ms count=2

Thanks for the update and feedback. I am going ahead and keeping this issue open for team's input on reported behavior.

Hi Darshan
Thank you for your response and for handling my enquiry.
Could you tell me about what you have found/noticed so far please?
With regards to what I am trying to achieve (pending data in an external
collection), is there an explanation for why it is not working?
With regards to what I am trying to achieve (pending data in an external
collection), is there an alternative way of achieving the same goal please?
And lastly, with this get(/databases/) call being problematic and causing
delay to progress, would Firebase invent other ways (new ways) for
customers to achieve the same outcome (as it is in Flutter), so that if one
technique isn't working then another technique can be tried instead, so
that it will not matter so much if customers cannot get the
get(/databases/) function call to work?
Many thanks for your time and for your patience and for your assistance.
Kind regards
M Mathems
On Thu, 11 May 2023 at 11:59, darshankawar @.***> wrote:

Thanks for the update and feedback. I am going ahead and keeping this
issue open for team's input on reported behavior.
—
Reply to this email directly, view it on GitHub
https://github.com/firebase/flutterfire/issues/10926#issuecomment-1543782343,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AZJHPOLSXTCEGJF7AMVVBHTXFTBBPANCNFSM6AAAAAAX2KDPW4
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi Darshan
I am still looking into this, but I think I have found two alternative
explanations for why I have been receiving the permission-denied message
from Firestore.
I hope to know more with a little more testing, but I will e-mail what I
find if it sheds light.
With thanks.
Kind regards
M Mathems
On Thu, 11 May 2023 at 11:59, darshankawar @.***> wrote:

Thanks for the update and feedback. I am going ahead and keeping this
issue open for team's input on reported behavior.
—
Reply to this email directly, view it on GitHub
https://github.com/firebase/flutterfire/issues/10926#issuecomment-1543782343,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AZJHPOLSXTCEGJF7AMVVBHTXFTBBPANCNFSM6AAAAAAX2KDPW4
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi
Thank you for getting back to me. I have provided a detailed response to your enquiry.
Based on the error messages received thru Flutter, the source of the problem is the current version of Firestore, so that this situation should be possible to replicate where an application relies on Firestore.
As you are aware, I have completed my testing using Flutter and smartphone emulators, and the results have been the same for both emulators, however, for some reason, the iOS emulator responds to Firestore's security rules, so that a denial by Firestore prevents the read from occurring on the iOS emulator, whereas the Android emulator does not respond to Firestore's security rules, and grants the read, but later discovers the Firestore rule which creates the denial, and then confirms that the relevant permission has been denied.
The Android emulator's exact error feedback is, "W/veloper_testin( 987): Accessing hidden field Ljava/net/Socket;->impl:Ljava/net/SocketImpl; (unsupported, reflection, allowed) W/veloper_testin( 987): Accessing hidden method Ljava/security/spec/ECParameterSpec;->setCurveName(Ljava/lang/String;)V (unsupported, reflection, allowed) W/Firestore( 987): (24.4.3) [Firestore]: Listen for Query(target=Query(users/l9NzQFjN4iB0JlJaY3AI/userA/2VzSHur3RllcF5PojT61/exclusiveA order by name);limitType=LIMIT_TO_FIRST) failed: Status{code=PERMISSION_DENIED, description=Missing or insufficient permissions., cause=null}"
And the iOS emulator's exact feedback is, "flutter: Login successful. flutter: [cloud_firestore/permission-denied] The caller does not have permission to execute the specified operation."
I believe that the situation I have presented can be replicated perfectly by recreating the Firestore collection-formation illustrated in my primary text, and with reliance on the files attached here.
Please note: the 'exclusiveA' sub-collection contains the documents relevant to the get(/databases/) call at the heart of this enquiry.
The Attached Files
My written Firestore security rules My main.dart file My UI's login screen My UI's stream query My flutter -v response (confirms SDK version, Language Platform + Development Environment) Screenshots (confirms Firestore collection-formation + field-value titles for documents)
A link to the attached files is here. [Please confirm when you finish with these files so that I can hide them from the public]
This Firestore project features the following field-values holding the following types of data:
docRefID - string horsePower - number preferredCurrency - string topSpeed - number vehicleBrand - string vehicleName - string eAddress - string eName2 - string id - string sex - string eID - string eName1 - string loginEmail - string amount - number creditCard - boolean currency - string uid - string wantsVehicle - boolean
Further Background
I believe that the above answers your requests for further information.
For further clarity of the situation, I was trying to find the way to offer access to the collection on the basis of a field in another collection, so that in a real application, a user could be 'subscribed' or 'admin' or some other important status and have access granted/denied on that basis.
My original enquiry was to code so that only the documents in exclusiveA holding a particular string could be read, but it was later explained to me that Firestore would deny access entirely if a user would not be permitted to read every document within the collection where a rule grants permission to read. Hearing this, I thought that that was the cause of the permission-denied message I was noticing, so then I changed things so that a rule to read would be so that every document within the collection could be read by the user, and then I found that the same error message was still being received. What I learnt around that time, was that a query featuring 'where' for filtering, should be drafted at the UI code instead of at Firestore.
Lastly, I found a similar case example in the Firestore forum that was responded to by Todd Kerpelman, where he explained that Firestore tends not to accept a get(/databases/) call unless it has an example/instance of the relevant data first. This might help. You can notice the case here.
I believe you now have all of the information that you require and I hope that there is some mistake on my part that has caused this error to arise, so that with some embarrassment I can remember and learn from this. Please confirm what you find, and any other advice for guidance would also be greatly appreciated.
In the case that you can find no solution here, could I request a replicable working example of the get(/databases/) call instead please? I would then assemble the example at a Firestore near me.
Many thanks for your time and for your assistance.
Kind regards
M Mathems

Reading the logs received by both iOS and android, it appears permission is being denied on both platforms. It would be ideal if you gave concrete steps on a reproduction (i.e. no screenshots, just code to run including data to fill the firestore database). that way we can quickly reproduce your issue. I would also add that FlutterFire is a thin wrapper around the native SDKs, any discrepancy with the rules is highly likely to be an issue with the native SDK, not FlutterFire.


Design panel not updating when switching between designer tables

Describe the bug
The design panel does not update correctly when switching between tables. Alternatively, switching could be blocked until active changes are reverted or saved.
Expected behavior
You can either switch tables correctly, or the user is prompted to save/revert

I am experiencing this issue and it make the Designer feature unusable. If there is any priority system for bugs I would greatly appreciate it being escalated.

Thank you @macjuul! I'll try out the new build.