Sync eng/common directory with azure-sdk-tools for PR 1580

Sync eng/common directory with azure-sdk-tools for PR https://github.com/Azure/azure-sdk-tools/pull/1580  See eng/common workflow

JS core timeouts unrelated to this change.


Unprocessable entity error handling

:tada: This PR is included in version 2.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Way to have collapsed as the default?

*I closed this because i realized this was already asked
@nagarajanchinnasamy  I would like to thank you for subotal.js.  I do have a requirement where I need to either collapse multiple rows or just have them all collapsed by default.  Currently, I am able to collapse one row, but when we expand that row, the rows inside are automatically expanded as well.  I was hoping there was a way to have the user expand it manually instead of it defaulting to expanded.
Thank you,
Derrick

i want to make collapse at intial refresh


Missing OriginPath in cloudfront.py

Origin object is missing OriginPath attribute, as described here: http://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/DistributionConfigDatatype.html

Thank you for the fix. I'm wondering if this change should be made as well to ensure that list are only class Origin:
diff --git a/troposphere/cloudfront.py b/troposphere/cloudfront.py
index 2b6a93b..35e0932 100644
--- a/troposphere/cloudfront.py
+++ b/troposphere/cloudfront.py
@@ -120,7 +120,7 @@ class DistributionConfig(AWSProperty):
         'DefaultRootObject': (basestring, False),
         'Enabled': (boolean, True),
         'Logging': (Logging, False),
-        'Origins': (list, True),
+        'Origins': ([Origin], True),
         'PriceClass': (basestring, False),
         'Restrictions': (Restrictions, False),
         'ViewerCertificate': (ViewerCertificate, False),

@markpeek i think that you are right.


Remove OMP backend

Overall this looks ok to me.
@alkino : you have kept this in draft for any specific reason?


Incorrect Rendering results on Apple Vision Pro

I viewed GaussianSplats3D in WebVR mode on AppleVisionPro.
It generally works for the areas where splats are present, but did not work well for the areas where splats are not present.
Video of the areas where splats are present.
https://twitter.com/kotauchisunsun/status/1758524300318191932
Video of the areas where splats are not present.
https://twitter.com/kotauchisunsun/status/1759187480174305319
Here is the source code.
<html lang="en">

<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="x-ua-compatible" content="ie=edge">
  <title>3D Gaussian Splat Demo - VR Garden</title>
  <script type="text/javascript" src="js/util.js"></script>
  <script type="importmap">
    {
        "imports": {
            "three": "./lib/three.module.js",
            "@mkkellogg/gaussian-splats-3d": "./lib/gaussian-splats-3d.module.js"
        }
    }
  </script>
  <style>

    body {
      background-color: #000000;
      height: 100vh;
      margin: 0px;
    }

  </style>

</head>

<body>
  <script type="module">
    import * as GaussianSplats3D from '@mkkellogg/gaussian-splats-3d';
    import * as THREE from 'three';
    const viewer = new GaussianSplats3D.Viewer({
      'initialCameraPosition': [-0.77493, -0.54359, 1.63199],
      'initialCameraLookAt': [0.20786, -0.68154, -0.27311],
      'webXRMode': GaussianSplats3D.WebXRMode.VR,
      'gpuAcceleratesSort': false,
      'sharedMemoryForWebWorkers': false
    });
    let path = 'assets/data/bonsai/bonsai.ksplat';
    viewer.addSplatScene(path, {
        'rotation': new THREE.Quaternion().setFromUnitVectors(new THREE.Vector3(0.01933, -0.75830, -0.65161).normalize(), new THREE.Vector3(0, 1, 0)).toArray(),
        'scale': [0.25, 0.25, 0.25],
        'position': [0, 1, 0]
    })
    .then(() => {
        viewer.start();
    });
  </script>
</body>

</html>

Thanks for the nice library!

Unfortunately I can't really debug this issue because I don't have an Apple Vision Pro, and as of right now I don't plan on buying one until the price comes down :) If anyone else has some insights here, it would be great to get some help!

The same issue can be reproduced in the visionOS simulator.

The same issue can be reproduced in the visionOS simulator. There is no error message in the console. I would love to provide more information if needed.
output.mp4

Why do I show "AR NOT SUPPORTED"

Can you help me?

The same issue can be reproduced in the visionOS simulator. There is no error message in the console. I would love to provide more information if needed.
output.mp4

Have you find a solution?
I'm also troubled by this issue.
or Do you have any alternative methods?
thanks

Have you find a solution? I'm also troubled by this issue. or Do you have any alternative methods? thanks

No, I have not. I tried a few things and didn't have enough time to get it right. Let's hope the author would allocate some time in the future :)

I discussed this in #127 by accident. The old drop in viewer method was working before on AVP.

@mkkellogg so it's definitely the dropin viewer vs non-dropin viewer. I tested on the latest commit.

@dlazares Good to know, I'll try to find some time this weekend to dig into this issue.

@dlazares Would you be able to share the code you used to reproduce the issue, as well as an example of where it works properly?

I know this is a bit late, bit I have WIP branch that I think addresses this issue: orthographic_projection -- If you want to try it and see if it fixes the issue. I'll merge it into main and push out a release soon.

I can confirm that the rendering issue has been fixed in the latest main branch. Aside from the incorrect color gamma (which is a known issue for WebXR mode on visionOS: ref), the alpha falloff on the splat edges seems weird. I noticed the same problem in a Metal implementation. Any ideas why this could go wrong? Thanks.
https://github.com/mkkellogg/GaussianSplats3D/assets/8616975/28d67709-f91d-4817-ba46-3522cb6853af

@mkkellogg I noticed a new problem after this bug was fixed on Vision Pro. Splat scales (or alphas?) are wrong only in stereo rendering mode on Quest headsets. This can be verified easily through the WebXR Emulator provided by Meta. I've also checked that this bug can be reproduced on the headset (but no screen recordings though since it's so laggy).
https://github.com/mkkellogg/GaussianSplats3D/assets/8616975/176d93b6-f36e-4833-8be5-a220269c3c6c
BTW, any comments on the previous questions would also be nice! I tried a few things and couldn't get it right. It seems this repo does not have the hard alpha falloff problem.

I will take a look at the stereo rendering bug, hopefully it's an easy fix. As for the hard alpha falloff, I'm guessing it might have something to do with the fact that in my viewer the splats are rendered as quads that are aligned to the splat's 2D screen-space axes that are clipped at sqrt(8) standard deviations. I try to document that in the shader code. The value sqrt(8) may seem somewhat arbitrary; I'm mainly using it because it is (less obviously) the same cutoff that is used in other online viewers like Antimatter's splat viewer and the  Unity Gaussian splat viewer You could try increasing that value here: https://github.com/mkkellogg/GaussianSplats3D/blob/0f5b839b4f6d30d5c65b487b20844f4b2cad61e2/src/SplatMesh.js#L157 and then its corresponding squared value here: https://github.com/mkkellogg/GaussianSplats3D/blob/0f5b839b4f6d30d5c65b487b20844f4b2cad61e2/src/SplatMesh.js#L357
(I probably need to have better variable names :) )
In the metal-splats repo they use the original approach to rendering, which is calculate a bounding box that is aligned to the screen and has a side length that is the maximum of the 2 screen-space eigen values scaled by 3.0 standard deviations: https://github.com/laanlabs/metal-splats/blob/4a1c3f05c06a3ddf5f7358d6aaadd889f3e47d64/MetalSplat/SplatShaders.metal#L285. This means their bounding box is much less compact than the one I calculate in my viewer, and maybe that's why they don't have a hard alpha fall-off issue.
I would also guess that the gamma issue you mentioned might be exacerbating the issue in my viewer. However this is all just a guess! I will investigate further and get back to you once I figure something out.

@mkkellogg I noticed a new problem after this bug was fixed on Vision Pro. Splat scales (or alphas?) are wrong only in stereo rendering mode on Quest headsets. This can be verified easily through the WebXR Emulator provided by Meta. I've also checked that this bug can be reproduced on the headset (but no screen recordings though since it's so laggy).
resized.mp4
BTW, any comments on the previous questions would also be nice! I tried a few things and couldn't get it right. It seems this repo does not have the hard alpha falloff problem.

I'm stucked with the same bug on Quest 3, any updates? Thanks!!

So I think the issue is that three.js (and my viewer) can't tell when WebXR is rendering in stereo mode, and therefore don't know that the aspect ratio needs to be adjusted in that case. Basically when rendering in stereo mode, the viewport width value that is sent to the shader for the splat renderer needs to be halved. I'll keep digging!

So I have a WIP branch that I think addresses the issue (at least in the WebXR emulator extension): https://github.com/mkkellogg/GaussianSplats3D/tree/fix_webxr_stereo. Can you try that out and let me know if it works for you?

So I have a WIP branch that I think addresses the issue (at least in the WebXR emulator extension): https://github.com/mkkellogg/GaussianSplats3D/tree/fix_webxr_stereo. Can you try that out and let me know if it works for you?

It works perfectly on a Quest 3!

Great! I'll hopefully get a release out with this fix soon :)

The latest release v0.3.9 now has this fix in it :)

I'll close this for now, let me know if you run into any other problems!


Development : Update JPlag to current Version

Checklist
General

[ ] I tested all changes and their related features with all corresponding user types on a test server.
[ ] This is a small issue that I tested locally and was confirmed by another developer on a test server.

Server

[x] I followed the coding and design guidelines.

Changes affecting Programming Exercises

[ ] I tested all changes and their related features with all corresponding user types on Test Server 1 (Atlassian Suite).
[ ] I tested all changes and their related features with all corresponding user types on Test Server 2 (Jenkins and Gitlab).

Motivation and Context

I've updated JPlag to it's current version

https://github.com/jplag/JPlag/actions/runs/1548612101
https://github.com/jplag/JPlag/commit/e46aa72cbcc98c4ef70f85e22cf9f619bdab6076

Steps for Testing

Prerequisites:

1 Instructor
2..n Students

Create some programming submissions (create some similar submissions (they have to be different to the base code / template))
As instructor check for plagiarism
See whether everything works as expected

Review Progress

Code Review

[ ] Review 1
[ ] Review 2

Manual Tests

[ ] Test 1
[ ] Test 2

Thanks for this PR. I would suggest, we wait for the official release and add the dependency to build.gradle to avoid the workaround of storing the dependency as jar file in the Github repo

You are right that in the future this lib should be handled as simple dependency.
I would also see the PR more as a temporary solution that might be needed until JPlag gets officially released in version 3.0, because if you look at the history, some important improvements have been made to JPlag (since the version Artemis is currently using). At the moment we mostly download all repos from Artemis and run JPlag externally, for example because of issues like #4448.

Ok, then it's fine from my side. If we can fix a couple of issues with the newer "unofficial" 3.0 version, let's test that the chances work correctly and then merge this PR 👍

Thanks for testing! Might be related to the issues in #3603 (not the issue itself, but the list of possible issues that are not handled)

Tested on t3 in a testing session with @VinnavanSembian, @mallgaier, @lBruchmann , @Strohgelaender , @cremertim. As @Strohgelaender said the issue #4448  is still present.

We should probably filter empty submissions on Artemis the Artemis server before sending to Jplag

Also the BaseCode should not be submitted if no enough tokens are contained

Also see https://github.com/jplag/JPlag/pull/253
JPlag provides several different exceptions now, they should be handled by artemis later on :)

I added filtering for empty submissions before sending them to JPlag, please test again if this works better now.
We can then merge this PR and fix other issues in follow-up PRs.

I've added a workaround for bad base code submissions.


Fake pull to reduce code coverage - ignore this

Codecov Report

Merging #34 (fecb3d9) into master (20571df) will decrease coverage by 13.99%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@             Coverage Diff             @@
##           master      #34       +/-   ##
===========================================
- Coverage   73.10%   59.11%   -13.99%
===========================================
  Files          15       12        -3
  Lines        3588     2830      -758
  Branches      377      358       -19
===========================================
- Hits         2623     1673      -950
- Misses        926     1143      +217
+ Partials       39       14       -25

see 1 file with indirect coverage changes


Cosmetic changes to CTS Login Page

The Single Sign On feature is still referred to as new on the CTS login page.

@CalumTowers can you ensure this gets moved across to RTS BitBucket please?
@agentsquash - this was my fault, I thought you were talking about the holding page for Core before it redirects to SSO.


Component not rendering when using get() in react 19

When using enableReactTracking and get(), component does not render when using react 19.
It works fine in react 18 and with react 19 when wrapped with observer()
https://codesandbox.io/p/sandbox/nywrhq?file=%2Fsrc%2FApp.tsx%3A20%2C14

enableReactTracking does not work in React 19. The documentation for it warns about that: https://www.legendapp.com/open-source/state/v3/usage/configuring/#enablereacttracking
We're actually about to fully deprecate it and encourage using useSelector instead of this or observer, to work better with React 19 and the Compiler.
Are there still parts of the docs that encourage using enableReactTracking? I am doing a big pass across the docs right now to fix any mistakes and change the guidance to encourage useSelector.


fix: Mark fields in user_option and user_surround as optional

Type linter complains that missing options should be valid in configuration, when in fact they're optional. This marks the fields as optional.

Just out of curiosity, what linter are you using? I can't seem to be able to replicate this using LuaLS and selene. I'm not opposed to merging this in, but I'm also curious about how the placement of ? matters, i.e. ---@field var? type vs. ---@field var type?. I suspect the former means field not required and the latter means field required, but may be nil, but I would like clarification on that.

Hi.
I'm using LuaLS + neodev, and I had warnings in my nvim-surround config that motivated me to try to find a way to make the fields optional.
I'm not a Lua expert, so I'm not sure I'm doing the right thing here. I tried adding the field? question mark, and it got rid of the warnings in the way I was expecting, and it still provides completion, so it at least it works.
I tried to look for official documentation, but I couldn't find any info about optional fields, only optional params, and optional type?. I'm wondering whether field? could be undocumented behavior, of it might even be the same thing as type? considering table.field = nil is how you delete a field in Lua.
Also, would you mind also testing this branch to make sure it also works in another setup?

Hmm strange, I've also been using neodev. I believe this should be a LuaLS issue, but I've been unable to reproduce it for myself, despite best efforts. I tried adding the ? to keymaps, and see this in my autocomplete:

However, I don't get any warnings/errors about missing fields etc. Have you updated nvim-surround/LuaLS to their latest respective versions?

The question mark behavior is documented for @param, and it’s also present in the syntax for @field.


Multiple issues when you have separated classes in different assemblies

Describe the bug
I have a solution which uses GremlinQ. Since I am using a Microservice architecture, I need some common logic between the MS that access the graph, which I put in a shared custom nuget package.
After I did this 2 things started to happen
Only on AddV (not on update), i get an Argument Exception saying my object is not part of the model.
The Path() response, changed, and instead of having the actual objects in the Objects field of the Path object, i has some Newtonsoft Dynamic representation (I dont have an example now, but if you need it I could try to recreate this).
I later found an answer to this, and it was adding
.ConfigureEnvironment(env=> env.UseModel(GraphModel.FromBaseTypes<Vertex, Edge>().AddAssemblies(typeof(Node).Assembly)));
Where Node is a vertex class that lives in the running assembly, and Vertex, Edge run in another assembly.
But that rises a 3rd issue, that is that the Path() query built is not correct, because it assuming the wrong label from the configured query.
Expected behavior
I expect the library to behave consistently whether we have the classes among one or multiple assemblies.
Version information
12.7.0
Minimal working example
https://github.com/sebastianamaro/Gremlinqissue
Additional context
I reported a bit more here: https://github.com/Gremlinq/ExRam.Gremlinq/issues/1627
But since I was not sure if I should open a new issue or not, I made it this way. Feel free to close either one of this.

Cont'd in #1627.


Default value

What do you think about having a default value in case there is an error while accessing a property.
Essentially,  the function signature might be like:
function idx<Ti, Tv, Td>(input: Ti, accessor: (input: Ti) => Tv, defaultValue: ?Td): ?Tv
And then we could use it as:
const DEFAULT_FRIENDS = [{ name: "Pavlos" }]
idx(props, _ => _.user.friends[0].friends, DEFAULT_FRIENDS)

Which can be transformed to
props.user == null
   ? props.user
   : props.user.friends == null
         ? props.user.friends
         : props.user.friends[0] == null
                ? DEFAULT_FRIENDS
                : props.user.friends[0].friends

I actually originally supported an optional third argument, defaultValue. However, I abandoned it because the expected behavior of such an argument turned out to be slightly more complicated than it seems, and I thought it might lead to surprises.
Here are the different potential behaviors for defaultValue (each with different nuances, none of which are obviously correct):

Return defaultValue only if the final member expression is null or undefined. (This is what you have proposed.)
Return defaultValue only if we attempt to access a property on null or undefined.
Return defaultValue if any of the member expressions are null or undefined.

Then, there's the question of whether or not we should return defaultValue if the final value is null or undefined.
The difference between these are pretty subtle and easy to get mixed up. In order to avoid confusion, it would be better for callers to explicitly expand the defaulting logic inline.
Also, we should think of idx as a temporary workaround until we have the real existential operator. A real existential operator will not support default values, so we should not either.
Happy to discuss further, but closing this for now. Thanks for the proposal, though!

In @ppoliani 's example:
const DEFAULT_FRIENDS = [{ name: "Pavlos" }]
idx(props, _ => _.user.friends[0].friends, DEFAULT_FRIENDS)

There are five chances to get undefined:
props == undefined
props.user == undefined
props.user.friends == undefined
props.user.friends[0] == undefined
props.user.friends[0].friends == undefined

We can pass the fourth argument about how to handle undefined. There are 3 values for strategies:

'LOOKUP'
'EXACTLY'
'ANY'

LOOKUP
We can define default values for levels:
const DEFAULT_FRIENDS = [{ name: "Pavlos" }]
const defaultValues = {
  0: [],    // for level 0 to level 3
  4: DEFAULT_FRIENDS  // for level 4
}
const result = idx(props, _ => _.user.friends[0].friends, defaultValues, 'LOOKUP')

So we can get different default values for different undefined levels:
props == undefined                           // get level 0 default value: []
props.user == undefined                      // get level 1 default value: []
props.user.friends == undefined              // get level 2 default value: []
props.user.friends[0] == undefined           // get level 3 default value: []
props.user.friends[0].friends == undefined   // get level 4 default value: [{ name: "Pavlos" }]

The 'LOOKUP' strategy uses inheritance, which means that if you did not provide the default value for that level, it will look up the previous level iteratively until finding a default value. If no default value provided in the inheritance, it will use undefined.
// now we only provide level 4 default value
const defaultValues = {
  4: DEFAULT_FRIENDS
}
const result = idx(props, _ => _.user.friends[0].friends, defaultValues, 'LOOKUP')

// results
props == undefined                           // get level 0 default value: undefined
props.user == undefined                      // get level 1 default value: undefined
props.user.friends == undefined              // get level 2 default value: undefined
props.user.friends[0] == undefined           // get level 3 default value: undefined
props.user.friends[0].friends == undefined   // get level 4 default value: [{ name: "Pavlos" }]

EXACTLY
When we use 'EXACTLY', it will only use the default value in the exact level:
const DEFAULT_FRIENDS = [{ name: "Pavlos" }]
const defaultValues = {
  0: [],    // only for level 0
  4: DEFAULT_FRIENDS  // only for level 4
}
const result = idx(props, _ => _.user.friends[0].friends, defaultValues, 'EXACTLY')

props == undefined                           // get level 0 default value: []
props.user == undefined                      // get level 1 default value: undefined
props.user.friends == undefined              // get level 2 default value: undefined
props.user.friends[0] == undefined           // get level 3 default value: undefined
props.user.friends[0].friends == undefined   // get level 4 default value: [{ name: "Pavlos" }]

ANY
We only provide a default value for all undefined levels:
const DEFAULT_FRIENDS = [{ name: "Pavlos" }]
const result = idx(props, _ => _.user.friends[0].friends, DEFAULT_FRIENDS, 'ANY')

props == undefined                           // get level 0 default value:  [{ name: "Pavlos" }]
props.user == undefined                      // get level 1 default value:  [{ name: "Pavlos" }]
props.user.friends == undefined              // get level 2 default value:  [{ name: "Pavlos" }]
props.user.friends[0] == undefined           // get level 3 default value:  [{ name: "Pavlos" }]
props.user.friends[0].friends == undefined   // get level 4 default value: [{ name: "Pavlos" }]

If you don't pass the fourth argument to idx(), the default strategy would be 'ANY;, which means that we'll get DEFAULT_FRIENDS for any undefined value.
const DEFAULT_FRIENDS = [{ name: "Pavlos" }]
const result = idx(props, _ => _.user.friends[0].friends, DEFAULT_FRIENDS) // Use 'ANY' strategy

How about this proposal?

@xareelee Thanks for the detailed proposal here and in claudepache/es-optional-chaining#16. However, I personally think it overcomplicates the problem that the optional chaining / existential operator seeks to solve.
Since idx is meant to be a polyfill for the ES proposal, I'll defer discussion there. If it gets incorporated into the proposal and makes reasonable progress in review, we can bring it into idx.


Graph restructure

The graph algorithms are solid, but the surrounding infrastructure is weak in terms of how scripting is going to work. Some restructuring is necessary to make scripting as easy and clean as possible.
Some changes to be made

Graph -> MainGraph
new Graph interface
new NestedGraph implementation
BuildIterator split out into its own class
GraphBase -> implementation/SnapGraph
new GraphImplementation interface
get rid of Graph::dumpToStdout() - useless in most situations. Going to keep the drawToFile().

This is going to be a single commit, and a large one at that, that deals with the following issues:

[x] #11
[x] #7
[x] #9
[x] #8

And requires the following checklist:

[x] Check that any re-ordering of methods is reflected in the adjacent .cc file (very pedantic about this)
[x] Check that includes aren't referring to specific implementations of interfaces (i.e. including main-registry.h when it should only include registry.h)
[x] Reduce headers to those directly needed by the unit
[x] Bin.lua reflects the changes
[x] All header include guards follow the format BIN_<FILE_NAME>_H__
[x] Make sure all test files are being linked into bin-test

All finished up. Looks good!


Twitter tip button uses wrong URL

Describe the bug
When using the tip button for a tweet the button uses an URL for one of the links that are included at the bottom of the tweet, e.g. /likes or /analytics. Instead it should use the URL of the tweet. Not the likes, analytics or whatever else.
In my case I didn't check the URL and was logged in. I tipped 1 AE to an URL that isn't publicly visible and therefore not claimable.
This is the tip:

https://superhero.com/tip/1534

Thanks for noticing and reporting this!


Android - cannot connect to SQL Server Express 2019 using any casual SqlClient version (v2.1.4, v4.1.0, v5Preview)

Describe the bug
Edit: please use the test application version added in the latest comment in this issue (more functions, less hardcoded)
Introduction:

there is no issue on Windows
on Windows, you should be seeing this with the sample app:

run as

This is a continuation of this issue:
https://github.com/dotnet/SqlClient/issues/1656
There, only Android 9 and higher can communicate with a SQL Server 2012 with latest patch and a valid certificate from DigiCert.
The message in SqlException is always the same for both of these issues:
(A connection was successfully established with the server, but then an error occurred during the pre-login handshake.)

In this issue, I have set up a local SQL Server Express 2019 instance and opened it to the local network (port 1433).
TCP/IP connection is enabled and a SQL account for authentication is set.
These versions of SQL Server API from C# fail to communicate to the instance from Android (.NET MAUI):

Microsoft.Data.SqlClient 2.1.4 (EF Core 6.0.6)
Microsoft.Data.SqlClient 5.0.0-preview2.22096.2 (EF Core 7.0.0-preview.5.22302.2)
Microsoft.Data.SqlClient 4.1.0
Microsoft.Data.SqlClient 2.0.0
Microsoft.Data.SqlClient 5.0.0-preview3.22168.1
System.Data.SqlClient 4.8.3

Result of the test:
No Android verison can communicate with the SQL Server Express 2019 instance properly after connection

all SqlClient versions have the same result on all Android versions when trying to communicate with the SQL Server 2019 instance -> "...pre-login handshake failed..." SqlException of Class 20 (A connection was successfully established with the server, but then an error occurred during the pre-login handshake.)
The Android devices do in fact connect to the server, because if they did not, I would have received a different SqlException.
using TrustServerCertificate=true; and/or Encrypt=false; does not help solving the error

Test project:

https://github.com/janseris/MAUI_SQLServer_Test/commit/8ee2e00dd7f78fe5d45b8939882af1b17b516299

About the test project:

note: watch the Debug Output window or place a breakpoint or execute step-by-step with debugger to observe the results (Label UI controls (which are supposed to show the text result) in MAUI are bugged and do not show on Android)
it executes a DB call using the list of SqlClients listed above

Here a detailed look into a separate test of Microsoft.Data.SqlClient 2.1.4 (EF Core 6.0.6 = current) alone:
The following are results (debug output from Visual Studio) with internal errors on Android versions 5,6,8,9,11,12 for attempted communication with SQL Server Express 2019 (all fail with the same error "pre-login handshake...") for EF COre 6.0.6 (using SqlClient 2.1.4 internally)

Android 5 emulator:

no internal exception shown

Android 6 emulator:

no internal exception shown

Android 8 emulator:

this happens for SQL Server 2012 as well

  [System.err] 	at com.android.org.conscrypt.OpenSSLEngineImpl.beginHandshakeInternal(OpenSSLEngineImpl.java:335)
  [System.err] 	at com.android.org.conscrypt.OpenSSLEngineImpl.beginHandshake(OpenSSLEngineImpl.java:325)
  [System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.n_onClick(Native Method)
  [System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.onClick(ButtonHandler_ButtonClickListener.java:30)
  [System.err] 	at android.view.View.performClick(View.java:6256)
  [System.err] 	at com.google.android.material.button.MaterialButton.performClick(MaterialButton.java:1194)
  [System.err] 	at android.view.View$PerformClick.run(View.java:24701)
  [System.err] 	at android.os.Handler.handleCallback(Handler.java:789)
  [System.err] 	at android.os.Handler.dispatchMessage(Handler.java:98)
  [System.err] 	at android.os.Looper.loop(Looper.java:164)
  [System.err] 	at android.app.ActivityThread.main(ActivityThread.java:6541)
  [System.err] 	at java.lang.reflect.Method.invoke(Native Method)
  [System.err] 	at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240)
  [System.err] 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767)```

Android 9 emulator:

no internal exception shown

Android 11 emulator:

this does not happen for SQL Server 2012
this happens with Encrypt=false and TrustServerCertificate=true

[System.err] 	at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:362)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1134)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1089)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:876)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:747)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:712)
[System.err] 	at com.android.org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:237)
[System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.n_onClick(Native Method)
[System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.onClick(ButtonHandler_ButtonClickListener.java:30)
[System.err] 	at android.view.View.performClick(View.java:7448)
[System.err] 	at com.google.android.material.button.MaterialButton.performClick(MaterialButton.java:1194)
[System.err] 	at android.view.View.performClickInternal(View.java:7425)
[System.err] 	at android.view.View.access$3600(View.java:810)
[System.err] 	at android.view.View$PerformClick.run(View.java:28305)
[System.err] 	at android.os.Handler.handleCallback(Handler.java:938)
[System.err] 	at android.os.Handler.dispatchMessage(Handler.java:99)
[System.err] 	at android.os.Looper.loop(Looper.java:223)
[System.err] 	at android.app.ActivityThread.main(ActivityThread.java:7656)
[System.err] 	at java.lang.reflect.Method.invoke(Native Method)
[System.err] 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
[System.err] 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
[System.err] Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
[System.err] 	at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:677)
[System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:554)
[System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:510)
[System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:428)
[System.err] 	at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:371)
[System.err] 	at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:102)
[System.err] 	at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:106)
[System.err] 	at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:250)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.verifyCertificateChain(ConscryptEngine.java:1644)
[System.err] 	at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method)
[System.err] 	at com.android.org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:568)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1095)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1115)
[System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1087)
[System.err] 	... 18 more
[System.err] Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
[System.err] 	... 32 more```

Android 12 emulator:

    [System.err] 	at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:363)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1134)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1089)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:876)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:747)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:712)
    [System.err] 	at com.android.org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:237)
    [System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.n_onClick(Native Method)
    [System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.onClick(ButtonHandler_ButtonClickListener.java:30)
    [System.err] 	at android.view.View.performClick(View.java:7441)
    [System.err] 	at com.google.android.material.button.MaterialButton.performClick(MaterialButton.java:1194)
    [System.err] 	at android.view.View.performClickInternal(View.java:7418)
    [System.err] 	at android.view.View.access$3700(View.java:835)
    [System.err] 	at android.view.View$PerformClick.run(View.java:28676)
    [System.err] 	at android.os.Handler.handleCallback(Handler.java:938)
    [System.err] 	at android.os.Handler.dispatchMessage(Handler.java:99)
    [System.err] 	at android.os.Looper.loopOnce(Looper.java:201)
    [System.err] 	at android.os.Looper.loop(Looper.java:288)
    [System.err] 	at android.app.ActivityThread.main(ActivityThread.java:7839)
    [System.err] 	at java.lang.reflect.Method.invoke(Native Method)
    [System.err] 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
    [System.err] 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1003)
    [System.err] Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
    [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:672)
    [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:549)
    [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:505)
    [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:425)
    [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:368)
    [System.err] 	at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:102)
    [System.err] 	at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:106)
    [System.err] 	at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:255)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.verifyCertificateChain(ConscryptEngine.java:1638)
    [System.err] 	at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method)
    [System.err] 	at com.android.org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:569)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1095)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1115)
    [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1087)
    [System.err] 	... 19 more
    [System.err] Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
    [System.err] 	... 33 more```

Android 12 device
- [System.err] javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.       [System.err] 	at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:363)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1134)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1089)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:876)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:747)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:712)       [System.err] 	at com.android.org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:237)       [System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.n_onClick(Native Method)       [System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.onClick(ButtonHandler_ButtonClickListener.java:30)       [System.err] 	at android.view.View.performClick(View.java:7451)       [System.err] 	at com.google.android.material.button.MaterialButton.performClick(MaterialButton.java:1194)       [System.err] 	at android.view.View.performClickInternal(View.java:7425)       [System.err] 	at android.view.View.access$3700(View.java:842)       [System.err] 	at android.view.View$PerformClick.run(View.java:28690)       [System.err] 	at android.os.Handler.handleCallback(Handler.java:938)       [System.err] 	at android.os.Handler.dispatchMessage(Handler.java:99)       [System.err] 	at android.os.Looper.loopOnce(Looper.java:346)       [System.err] 	at android.os.Looper.loop(Looper.java:475)       [System.err] 	at android.app.ActivityThread.main(ActivityThread.java:7889)       [System.err] 	at java.lang.reflect.Method.invoke(Native Method)       [System.err] 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)       [System.err] 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1009)       [System.err] Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.       [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:672)       [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:549)       [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:505)       [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:425)       [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:368)       [System.err] 	at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:102)       [System.err] 	at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:106)       [System.err] 	at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:255)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.verifyCertificateChain(ConscryptEngine.java:1638)       [System.err] 	at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method)       [System.err] 	at com.android.org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:569)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1095)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1115)       [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1087)       [System.err] 	... 19 more       [System.err] Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.       [System.err] 	... 33 more

To reproduce

have a working SQL Server Express 2019 instance exposed to the network (this tutorial will do: https://www.youtube.com/watch?v=xNmIdFjXzl4)

set up a SQL Server Express 2019
open port 1433 on firewall
configure the SQL Server Express instance to allow SQL authentication
enable an account on the instance (e.g. sa)
configure the instance to allow TCP/IP connection on port 1433

be able to run the MAUI application

download latest VS 2022 Preview (because MAUI can only be used in VS 2022 Preview)
add the MAUI part in Visual Studio Installer

install some Android emulators (default settings are OK) via Visual Studio ->

or use physical device - after connecting via USB and enabling USB debugging, you should be able to see and use it as follows:

Note:

the connection strings which are hardcoded into the sample app, have the following properties:

Database Ordinace devRemin
SQL account name sa
SQL account password sa
a table called USER in the default dbo schema

the EF test projects require that it has some columns but you can skip using the EF Core projects because they use SqlClient anyways

the other projects simply select number of rows from the USER table (could have selected something from a dummy table or be doing SELECT 'Hello World') the current version simply executes SELECT LEN('Hello World')

Note #2:

the Constants.LocalNetworkConnectionString string has a hardcoded private IP of <IP_ADDRESS>

this IP will probably be different in every local network
the correct IP can be obtained via running the command line command ipconfig on the host machine running the SQL Express (this example requires that the Android device and the machine hosting the SQL Server are in the same local network so that private IP can be used (or you can share your SQL Server Express to the internet and use a public IP address in fact but I did not do that)

look for IPv4 Address entry in the output of the ipconfig command, that's the private/local IP address

Expected behavior
Android can communicate with the SQL Server Express.
Further technical details
.NET target: .NET 6.0 (dotnet --version: 6.0.400-preview.22301.10)
SQL Server version: SQL Server Express 2019
Operating system: Windows 10 21H1

Day 2:

server configuration was not changed
tested older 6.0.1 project on Windows which still connects without TrustServerCertificate=true and without Encrypt=false and now the test application does not require TrustServerCertificate=true or Encrypt=false (which it did yesterday, because without it, it would keep saying that certificate validation failed on Windows)

The app has been updated for faster testing and examination.
The latest version can be seen here:
It now allows selecting connection string, entering a custom connection string, adding suffixes to the connection string, disable connection pooling (that did not help getting different results on different versions of SqlClient on Windows machine concerning the certificate though) etc.
The results for me are (for all versions of SqlClient):

SQL Server Express 2019 local instance: no Android device can communicate and Windows devices can communicate only when ignoring the certificate
SQL Server 2012 latest patch: Android 9+ devices can communicate and all Windows devices can communicate

What is the expected result:

the SQL Server Express 2019 should be fully accessible by any Android version and on Windows the certificate should not be checked when older SqlClient than 4.0.0 is used (because then Encryption is not enabled)
the SQL Server 2012 should be fully accessible by any Android version

Hi @janseris is this a duplication of #1656?

Hi @janseris is this a duplication of #1656?

Hi, no it is not.

one other step to check before going deeper into the issue. Can you look into Sql Server Configuration Manager , expand Sql Server Network Configuration right click on Protocols for <your instance name> and select property. You will see a window similar to below:

and make sure the Force Encryption is set to No. Also when you look into Protocols for <your instance name>,  on the right hand side column, can you let me know what protocol names are enabled?

Hi @janseris,
we spend a LOT of hours to get this working.
You have to use certificates, without them there is no connection possible.
This is our way to achieve the successful connection:
Setup the Device and Server

create a selfsigned Cert, consider this:

the Flag "CA:TRUE" must be set
set the commonName to the ServerName, ..local
Build a CRT-File and a PFX

install the CRT-File on the device and the PFX on the Server
(we have installed the cert in the Store "Computer/own Cert"  and make a copy to "trustworthy root... " (in german it was "Vertrauenswürdige Stammzertifizierungsstellen")
Set Permission -> "Manage Private Keys" see Link below (2)
you can now select the cert in the certicate-Tab in SQL Configuration Mgr
(SQL-Network config -> Right click on Protocols)
restart SqlService

Setup the MAUI-App (You have to explicitly allow User Certificates)

AndroidManifest (in application-tag): android:networkSecurityConfig="@xml/network_security_config"
include "network_security_config.xml" in project (attached to this comment as zip)
ConnectionString like this:
Data Source=..local; ... ;Trust Server Certificate=True

Give a Try.
Links:
(1) https://codekabinett.com/rdumps.php?Lang=2&targetDoc=create-install-ssl-tls-certificate-sql-server
(2) https://stackoverflow.com/questions/36830411/how-can-i-give-sql-server-permission-to-read-my-ssl-key
network_security_config.zip

@janseris were you able to make a connection to server with the provided information above?

@janseris were you able to make a connection to server with the provided information above?

Hi,  I had days off last week. I hope to get to this later this week. I haven't tried yet but saw your tips. Thank you.

Hello, I've been dealing with this error for 2-3 days, SQL SERVER 2019 Express is installed and all settings are done, even if I install the certificate, it is not accepted, I am trying to connect with DbCOntext via EF CORE.
Error detail;
 [System.err] javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. [System.err] 	at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:363) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1134) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1089) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:876) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:747) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:712) [System.err] 	at com.android.org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:237) [System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.n_onClick(Native Method) [System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.onClick(ButtonHandler_ButtonClickListener.java:30) [System.err] 	at android.view.View.performClick(View.java:7509) [System.err] 	at com.google.android.material.button.MaterialButton.performClick(MaterialButton.java:1194) [System.err] 	at android.view.View.performClickInternal(View.java:7486) [System.err] 	at android.view.View.-$$Nest$mperformClickInternal(Unknown Source:0) [System.err] 	at android.view.View$PerformClick.run(View.java:29358) [System.err] 	at android.os.Handler.handleCallback(Handler.java:942) [System.err] 	at android.os.Handler.dispatchMessage(Handler.java:99) [System.err] 	at android.os.Looper.loopOnce(Looper.java:201) [System.err] 	at android.os.Looper.loop(Looper.java:288) [System.err] 	at android.app.ActivityThread.main(ActivityThread.java:7892) [System.err] 	at java.lang.reflect.Method.invoke(Native Method) [System.err] 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548) [System.err] 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936) [System.err] Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:670) [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:547) [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:503) [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:423) [System.err] 	at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:366) [System.err] 	at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:102) [System.err] 	at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:106) [System.err] 	at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:255) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.verifyCertificateChain(ConscryptEngine.java:1638) [System.err] 	at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method) [System.err] 	at com.android.org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:569) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1095) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1115) [System.err] 	at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1087) [System.err] 	... 19 more [System.err] Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. [System.err] 	... 33 more **Microsoft.Data.SqlClient.SqlException:** 'A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: TCP Provider, error: 35 - An internal exception was caught)'

@janseris
Hi thank you,
Let's face it, we will now provide a solution with the RESTFULL API.

@FikretAkin thank you for tip but currently I am not planning this solution because this is a very expensive workaround.

i know the security of "direct" communicate with SQL Server Database. but we just "connect" locally, and no security issues here with small team like me. and too expensive with Restfull API, because we are not big team or corporate.
so maybe someone can fixed this bug? please

Hi, any update, please? There are multiple people in MAUI repo reporting this.

Thanks @janseris for your thorough analysis.  I'm currently investigating #1412 and it seems like a similar issue. I'm able to reproduce the connection failure issue using Sql Server 2019 express edition, and my .NET6 MAUI app running with MDS v5.0 on the Android Emulator running Android 12.1. Also did some digging and saw the issue also mentioned here https://github.com/dotnet/maui/issues/3522 that the project needs to set it's SSL/TLS Implementation to Native TLS 1.2+ ; however, in .NET6's MAUI project, the option no longer exists and I read there was a mention that TLS 1.2 is enabled by default https://techcommunity.microsoft.com/t5/app-development/enable-tls-1-2-on-net-maui-app-on-android/m-p/3267534 .  In the same issue someone also mentioned that it worked on AzureSqlServer, but only local/on-prem instances of SqlServer seems to run into the issue which makes me suspect, it might be a configuration issue, but I'll have to dig deeper to figure the root cause.

@lcheunglci for me it's SQL Server 2012 with latest patch working for Android 9+ and SQL Server Express 2019 working on none.
Also, if I remember correctly, SQL Server Express 2019 cannot be connected to from MAUI Windows
With WinForms, ASP.NET or Console, I never had any issue with any SQL Server version.

Thanks for the insight @janseris, I'll keep that in mind when I'm doing my testing.

btw. there are very similar issues with SMTP, WebSocket and gRPC to this one also for MAUI. Mainly affecting older Android versions, sometimes precisely matching the Android versions in this issue -> 8.1 and lower vs 9.0 and higher

Hi folks,
sorry for replying so late. Here are some information about creating a valid cert.
The guy who created the cert for me wrote one cnf-File and used openssl- tool to create the cert.
Maybe there are some unnecessary entries, but at the end this works for me.
this is the first call to openssl-Tool:
openssl req -config openssl.cnf -new -x509 -days 825 -out mycert27.crt
Content of Openssl.cnf (of course, you have to replace every "[PCName].[Domain].local"):
[ req ]
default_bits        = 4096
default_keyfile     = mykey31.key
default_md          = sha256
default_days        = 825
encrypt_key         = no
distinguished_name  = subject
req_extensions      = req_ext
x509_extensions     = x509_ext
string_mask         = utf8only
prompt              = no
The Subject DN can be formed using X501 or RFC 4514 (see RFC 4519 for a description).
Its sort of a mashup. For example, RFC 4514 does not provide emailAddress.
[ subject ]
countryName                 = DE
#stateOrProvinceName         = TH
#localityName                = X
organizationName            = X
#OU                          = X
Use a friendly name here because it's presented to the user. The server's DNS
names are placed in Subject Alternate Names. Plus, DNS names here is deprecated
by both IETF and CA/Browser Forums. If you place a DNS name here, then you
must include the DNS name in the SAN too (otherwise, Chrome and others that
strictly follow the CA/Browser Baseline Requirements will fail).
commonName              = [PCName].[Domain].local
#emailAddress            = .
Section x509_ext is used when generating a self-signed certificate. I.e., openssl req -x509 ...
[ x509_ext ]
#subjectKeyIdentifier      = hash
#authorityKeyIdentifier    = always,issuer
#authorityKeyIdentifier    = keyid:always,issuer
You only need digitalSignature below. If you don't allow
RSA Key transport (i.e., you use ephemeral cipher suites), then
omit keyEncipherment because that's key transport.
basicConstraints        = CA:TRUE
#basicConstraints        = critical, CA:TRUE
#keyUsage            = critical, digitalSignature, keyEncipherment, cRLSign, keyCertSign
#subjectAltName          = [PCName].[Domain].local
extendedKeyUsage = serverAuth
RFC 5280, Section <IP_ADDRESS> makes EKU optional
CA/Browser Baseline Requirements, Appendix (B)(3)(G) makes me confused
In either case, you probably only need serverAuth.
#extendedKeyUsage    = TLS Web Server Authentication
Section req_ext is used when generating a certificate signing request. I.e., openssl req ...
[ req_ext ]
subjectKeyIdentifier        = hash
basicConstraints        = CA:FALSE
#keyUsage            = digitalSignature, keyEncipherment
#subjectAltName          = [PCName].[Domain].local
#nsComment           = "OpenSSL Generated Certificate"
RFC 5280, Section <IP_ADDRESS> makes EKU optional
CA/Browser Baseline Requirements, Appendix (B)(3)(G) makes me confused
In either case, you probably only need serverAuth.
extendedKeyUsage    = serverAuth, clientAuth
[ alternate_names ]
DNS.1       = example.com
DNS.2       = www.example.com
DNS.3       = mail.example.com
DNS.4       = ftp.example.com
Add these if you need them. But usually you don't want them or
need them in production. You may need them for development.
DNS.5       = localhost
DNS.6       = localhost.localdomain
DNS.7       = <IP_ADDRESS>
IPv6 localhost
DNS.8     = ::1
Second call to openssl to create PFX from CRT and KEY:
openssl pkcs12 -export -in mycert27.crt -inkey mykey27.key -out mycert27.pfx
After creating these to files, you can install them on server and device as described in my previous post.
Good luck.

@stegl83
Thanks, now please why is this even required on Android when it is not on Windows?
Did it solve the connection problem?

Hi, I managed to get it working on the android emulator running Android 21 with Sql Server 2019 and M.D.SqlClient 5.0 GA.
After I generated the self signed certs and installing them on both the sql server and android emulator by following the instructions from @stegl83 on creating the self signed certificate here and installing certs i.e. installing pfx file on the my local sql server and the crt file onto the android device by copying it using adb push <path_to_self_signed_crt_not_pfx> /sdcard/download/ directory and on the device, going into settings and installing the certificate.
Go into Settings > Security > Encryptions & credentials > Install a certificate > CA Certificate

Select Install Anyway and locate your certificate in your Downloads folder

Once installed, the notification will give you warning that there's a CA installed from an unknown third party, which means your self-signed cert was installed successfully

Since my android emulator and the sql server were both on the same machine.  I found this article on the MAUI documentation and use the <IP_ADDRESS> ip address instead of my machine's hostname and adding the following to the network_security_config.xml in addition to the previous instructions on the MAUI side.
    <domain-config cleartextTrafficPermitted="true">
        <domain includeSubdomains="true"><IP_ADDRESS></domain>
    </domain-config>

Now boot up your app, and connect to <IP_ADDRESS>,1433 or whichever port you chose for your Sql Server and it should get passed the connection.Open()
and the connection string I used is:
Data Source=tcp:<IP_ADDRESS>,1433;Database=Northwind;uid=sa;pwd=***;encrypt=false;Connection Timeout = 30;TrustServerCertificate=true;
note: don't use trusted_connection=true; in the connection string as it'll override the uid and pwd as I tested it and resulted in a System.Data.Security.Native unable to load dll error with a stack trace identical to this.
As soon as I removed the self-signed cert from either end i.e. android or sql server, the error 35 returns.  I recall reading that the Azure Sql Server doesn't run into this issue, which I'm going to guess is because it's the url i.e. *.database.windows.net might already have proper ssl certificate issued by an official CA installed on the server and the client automatically trusts the certificate, so it doesn't this setup.
Let me know if this works for you.

@lcheunglci Perfect.
Do you have any clue why a certificate is required on Android while it is not on Windows and especially why it cannot be ignored by TrustServerCertificate true which should bypass the certificate validity check?

From a brief look at how the TrustServerCertificate connection parameter is used, it seems like it's only referenced when encrypt parameter is set to true or mandatory during the enabling of ssl, and will require more investigation in the MDS.SNI to figure out how it works.

I'm glad that I could help you.
It's annoying that we have so much effort for securing a connection in private network. Not only to spend a lot of hours to get this running, as far as there is no easier solution, we had additional effort for maintaining certificates on server and devices in each production environment we set up.
The next step is to find if its possible to rollout a certificate by MDM (Android Enterprise).

Managed to solve it with cert and IP address.
How?

Create a cert with powershell for your IP address: New-SelfSignedCertificate -certstorelocation cert:\localmachine\my -dnsname '<IP_ADDRESS>','localhost' -KeySpec KeyExchange -FriendlyName '<IP_ADDRESS>' -NotAfter (Get-Date).AddMonths(240)
(Will work with any IP address of your sql server later).

Set your cert in SQL Server Configuration manager.

Export that certificate as Base-64 encoded X.509 (.CER) file.

Import that cer in Platforms\Android\Resources\raw folder (if it does not exist create it.

In Platforms\Android\xml folder create nsc.xml (for me was not working if I name it network_security_config.xml)
with content: (change the data ofc IP and domain names and @raw/certname mine was called razvoj1.cer in \Platforms\Android\Resources\raw folder and my SQL server IP was <IP_ADDRESS> )

localhost
<IP_ADDRESS>,1433
<IP_ADDRESS>
RAZVOJ1
RAZVOJ1.localhost

In AndroidManifest add android:usesCleartextTraffic="true" and android:networkSecurityConfig = "@xml/nsc" to application tag :

Use this connection string:
string connectionString = @"Server=<IP_ADDRESS>,1433;Database=YOURDB;User Id=sa;Password=YOURPASS;Persist Security Info=True;Encrypt=True;TrustServerCertificate=True";

You should be able to open SQLConnection with Microsoft.Data.SQLClient 5.0.0 and .net 7 MAUI

Thanks @thevirtualdj for providing the steps in concise and organized manner.  It'll definitely make it easier for others to follow.  As for the TrustServerCertificate, I found that once the self-signed cert is installed on the Android device/emulator, it actually doesn't matter if it's set to true or false, which is a bug that I'll further investigate.

From a brief look at how the TrustServerCertificate connection parameter is used, it seems like it's only referenced when encrypt parameter is set to true or mandatory during the enabling of ssl, and will require more investigation in the MDS.SNI to figure out how it works.

The SSL/TLS moment is here during SQL Server login process. This part of the connection is always encrypted even when Encrypt=false. Otherwise, plaintext login and password would be sent over the network because SQL Server communication is not encrypted by default. AFAIK what we are talking all the time about here is that the SSL problem is in the login process because everything fails at this moment (the Encrypt true /false option isn't used at all because the actual data transfers aren't even initiated when login fails because of Android errors)

From a brief look at how the TrustServerCertificate connection parameter is used, it seems like it's only referenced when encrypt parameter is set to true or mandatory during the enabling of ssl, and will require more investigation in the MDS.SNI to figure out how it works.

The SSL/TLS moment is here during SQL Server login process. This part of the connection is always encrypted even when Encrypt=false. Otherwise, plaintext login and password would be sent over the network because SQL Server communication is not encrypted by default. AFAIK what we are talking all the time about here is that the SSL problem is in the login process because everything fails at this moment (the Encrypt true /false option isn't used at all because the actual data transfers aren't even initiated when login fails because of Android errors)

Glad you managed to get it rolling :)

From a brief look at how the TrustServerCertificate connection parameter is used, it seems like it's only referenced when encrypt parameter is set to true or mandatory during the enabling of ssl, and will require more investigation in the MDS.SNI to figure out how it works.

The SSL/TLS moment is here during SQL Server login process. This part of the connection is always encrypted even when Encrypt=false. Otherwise, plaintext login and password would be sent over the network because SQL Server communication is not encrypted by default. AFAIK what we are talking all the time about here is that the SSL problem is in the login process because everything fails at this moment (the Encrypt true /false option isn't used at all because the actual data transfers aren't even initiated when login fails because of Android errors)

Glad you managed to get it rolling :) Hope that .NET team will manage to solve usesCleartextTraffic case. Because in most cases apps are hidden behind VPN or some private net and does not need for cert encryption on this level.

Clear text is quite dangerous. Small businesses which use direct communication to db to save api development aren't probably the case you describe. Going plaintext is not a solution

From a brief look at how the TrustServerCertificate connection parameter is used, it seems like it's only referenced when encrypt parameter is set to true or mandatory during the enabling of ssl, and will require more investigation in the MDS.SNI to figure out how it works.

The SSL/TLS moment is here during SQL Server login process. This part of the connection is always encrypted even when Encrypt=false. Otherwise, plaintext login and password would be sent over the network because SQL Server communication is not encrypted by default. AFAIK what we are talking all the time about here is that the SSL problem is in the login process because everything fails at this moment (the Encrypt true /false option isn't used at all because the actual data transfers aren't even initiated when login fails because of Android errors)

Glad you managed to get it rolling :) Hope that .NET team will manage to solve usesCleartextTraffic case. Because in most cases apps are hidden behind VPN or some private net and does not need for cert encryption on this level.

Clear text is quite dangerous. Small businesses which use direct communication to db to save api development aren't probably the case you describe. Going plaintext on sending password to the DB over the network is the worst solution.

We understand that, but we are having a lot of troubles with certificates.
Example.. mine example is working fine with android 10,11,12 but on 7.1 we have error like:
[System.err] java.lang.IllegalStateException: Handshake has already been started
[System.err] 	at com.android.org.conscrypt.OpenSSLEngineImpl.beginHandshake(OpenSSLEngineImpl.java:139)
[System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.n_onClick(Native Method)
[System.err] 	at crc64fcf28c0e24b4cc31.ButtonHandler_ButtonClickListener.onClick(ButtonHandler_ButtonClickListener.java:31)
[System.err] 	at android.view.View.performClick(View.java:5637)
[System.err] 	at com.google.android.material.button.MaterialButton.performClick(MaterialButton.java:1194)
[System.err] 	at android.view.View$PerformClick.run(View.java:22433)
[System.err] 	at android.os.Handler.handleCallback(Handler.java:751)
[System.err] 	at android.os.Handler.dispatchMessage(Handler.java:95)
[System.err] 	at android.os.Looper.loop(Looper.java:154)
[System.err] 	at android.app.ActivityThread.main(ActivityThread.java:6121)
[System.err] 	at java.lang.reflect.Method.invoke(Native Method)
[System.err] 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:889)
[System.err] 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:779)

@thevirtualdj
The issue as you can see, is not with certificate, but is an issue with SSL handshake in implementation used by Android 8.1 and lower in C#.

We still have no solution to make it works without a cert right now, right?

2. SQL Server Configuration manager

I followed your steps but still getting the same error with SQL connection, Microsoft.Data.SQLClient 5.0.1 on SQL server 2019 but still get the error 31 T_T btw i tested on Android 9 and Android 11 real devices.

There are good news about the issue:
https://github.com/dotnet/runtime/pull/77386 has been merged.
Maybe we have a solution in .NET 8.

There are good news about the issue: dotnet/runtime#77386 has been merged.
Maybe we have a solution in .NET 8.

I followed that post while ago, the bad thing is that we have to wait 10 months, Right? xD

There are good news about the issue: dotnet/runtime#77386 has been merged.
Maybe we have a solution in .NET 8.

I followed that post while ago, the bad thing is that we have to wait 10 months, Right? xD

Yeah, right.
But better than never get it because its moved to the "backlog" AKA recycle bin.
I'm waiting for the first preview of .NET 8 for MAUI ...


unable to compile with current asmack master branch

I have built asmack-jse.jar from latest guardianproject asmack master branch (06a2c8363e), but the jar does not work with current Gibberbot.
Here is the compile error:
The method setCustomSSLContext(SSLContext) is undefined for the type ConnectionConfiguration	XmppConnection.java	/Gibberbot/src/info/guardianproject/otr/app/im/plugin/xmpp	line 680	Java Problem

it seems that the function "ConnectionConfiguration.setCustomSSLContext" is missing.

Try building asmack android. There are build targets
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
luzi82<EMAIL_ADDRESS>wrote:
I have built asmack-jse.jar from latest guardianproject asmack master branch (06a2c8363e), but the jar does not work with current Gibberbot.
Here is the compile error:
The method setCustomSSLContext(SSLContext) is undefined for the type ConnectionConfiguration	XmppConnection.java	/Gibberbot/src/info/guardianproject/otr/app/im/plugin/xmpp	line 680	Java Problem
it seems that the function "ConnectionConfiguration.setCustomSSLContext" is missing.
--
Reply to this email directly or view it on GitHub:
https://github.com/guardianproject/Gibberbot/issues/81

Thanks to your help. I have compiled asmack-android-4.jar and put to Gibberbot, but still does not help.
Moveover, there is no "setCustomSSLContext" in any file in asmack.

Which version of Android sdk are you compiling against?
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
luzi82<EMAIL_ADDRESS>wrote:
Thanks to your help. I have compiled asmack-android-4.jar and put to Gibberbot, but still does not help.
Moveover, there is no "setCustomSSLContext" in any file in asmack.
--
Reply to this email directly or view it on GitHub:
https://github.com/guardianproject/Gibberbot/issues/81#issuecomment-2506200

I have updated to the latest. And use sdk 4, just as the target
"compile-a4".
Would you please check if the github contains the latest src?  Thanks.
On Oct 25, 2011 1:11 AM, "n8fr8" <
<EMAIL_ADDRESS>wrote:

Which version of Android sdk are you compiling against?
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
luzi82<EMAIL_ADDRESS>wrote:
Thanks to your help. I have compiled asmack-android-4.jar and put to
Gibberbot, but still does not help.
Moveover, there is no "setCustomSSLContext" in any file in asmack.
--
Reply to this email directly or view it on GitHub:
https://github.com/guardianproject/Gibberbot/issues/81#issuecomment-2506200
--
Reply to this email directly or view it on GitHub:
https://github.com/guardianproject/Gibberbot/issues/81#issuecomment-2506274

I mean the latest src of asmack.

I can check tonight
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
luzi82<EMAIL_ADDRESS>wrote:
I have updated to the latest. And use sdk 4, just as the target
"compile-a4".
Would you please check if the github contains the latest src? Thanks.
On Oct 25, 2011 1:11 AM, "n8fr8" <
<EMAIL_ADDRESS>wrote:

Which version of Android sdk are you compiling against?
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
luzi82<EMAIL_ADDRESS>wrote:
Thanks to your help. I have compiled asmack-android-4.jar and put to
Gibberbot, but still does not help.
Moveover, there is no "setCustomSSLContext" in any file in asmack.
--
Reply to this email directly or view it on GitHub:
https://github.com/guardianproject/Gibberbot/issues/81#issuecomment-2506200
--
Reply to this email directly or view it on GitHub:
https://github.com/guardianproject/Gibberbot/issues/81#issuecomment-2506274

--
Reply to this email directly or view it on GitHub:
https://github.com/guardianproject/Gibberbot/issues/81#issuecomment-2506362

Thank you.  I will wait for your reply.  I need to recompile asmack to check the issue #75 or others.

How I compile asmack:

Fork the guardianproject/asmack project to luzi82/asmack.  The master (06a2c8363e) are the same.
Obtain asmack from git@github.com:luzi82/asmack.git
Create local.properties from "local.properties.example".  Variables are configurated.
Modify "patch/buddycloud/20-PubSubNode-getAllSubscriptions.patch" a bit, otherwise it will be compile error.

This step may not be necessary because the buddycloud builds will not be used, and the android builds will be still created.

Run "build.bash".  The following files are created:

build/asmack-android-4.jar
build/asmack-android-6.jar
build/asmack-android-7.jar
build/asmack-jse.jar

Copy asmack-android-4.jar to Gibberbot.
Compile Gibberbot project with eclipse.  Error occur.

After more digging, it looks like the setCustomSSLContext feature comes from here:
https://github.com/rtreffer/smack/commit/19438d9dce9ac1ae466973487a65732c575b63f6
which means the version of Smack we are using is actually:
https://github.com/rtreffer/smack
though in our version of build.bash that line is commented out:
fetchall() {
gitfetch "git://github.com/rtreffer/smack.git" "smack"
fetch "http://svn.igniterealtime.org/svn/repos/smack/trunk" "smack"
fetch "http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/management/common/src/main/" "qpid"
fetch "http://svn.apache.org/repos/asf/harmony/enhanced/java/trunk/classlib/modules/auth/src/main/java/common/" "harmony"
fetch "https://dnsjava.svn.sourceforge.net/svnroot/dnsjava/trunk" "dnsjava"
fetch "https://kenai.com/svn/jbosh~main/trunk/jbosh/src/main/java" "jbosh"
}
Hmm.

Thank you very much.  I will try that later.
(The change even does not show in http://www.igniterealtime.org/builds/smack/docs/latest/javadoc/org/jivesoftware/smack/ConnectionConfiguration.html , well. )

Please review this pull:
https://github.com/guardianproject/asmack/pull/2

The latest asmack is building just fine:
https://github.com/guardianproject/asmack


 The working directory needs to be in build/bin.
Try
cd build/bin
./blocks

I'll have to hunt for what's causing the crash. Probably just missing a null check in the draw

Try cd build/bin; ./blocks

That worked! Might be worth it to hack in absolute paths instead of relative paths in that case...

Can't reproduce the crash. It's crashing in SDL vulkan defragment and I saw some SDL issues about bad defragging. Probably a bug in SDL, closing.


Add setting to stop auto-opening compose box.

When you click on a topic inside a stream, the compose box automatically gets opened.  This is really annoying when you're on an iPhone using the browser app just to skim zulips.

Here are reasons I would rather have me open the compose box than have the software open it up for me:

I'm fine to open it myself, since click and enter work fine for replies.
I would get screen real estate back, especially on mobile.
The keyboard-only flow would be nicer when skimming messages.
It would be consistent with clicking on a stream, where we don't auto-open the compose box.

We could either have a setting to change this behavior, or we could try to use a flag to disable the auto-open on chat.zulip.org for a few days to see if we get any negative feedback, and then eventually remove the code.

Closing as dup.


UEFI installation always fail

I have already worked through the instructions on 3 computers, point by point and always the encrypted version.
None of the installations worked.
At the point refind-install (I had chosen the pre-build zfsbootmenu variant) I got efi variable error messages. I then ran efivar-tester which claimed that the device was not UEFI capable.
The first two laptops have a CSM option in the bios, so I assumed that they are UEFI capable. And the third laptop (Chromebook Acer C720 with Coreboot) there is even a UEFI shell.
In this respect, I must have made a mistake when installing according to instructions. What could I have done wrong.

If the laptop is booted in UEFI mode, there should be some files in this directory.
/sys/firmware/efi/efivars/

Again, this is a topic for the discussions. Please open a topic there if you like. If you identify a specific defect with the instructions or the ZBM logic, open an issue here.

I'm sorry, wasn't aware of 'discussions'. You can delete this Issue. I moved it.


expose election data to ocmw as they can only pick people who have been elected

Description
expose election data to ocmw as they can only pick people who have been elected
How to test
using https://github.com/lblod/frontend-lokaal-mandatenbeheer/pull/402, go to an ocmw of the new period and add mandataris instances after closing the IV prep. You should see person instances to select

Added these in the gemeente IV
And got these amounts in the OCMW

Mandatarissen

I assume that all of these should have been added to the ocmw when the IV is being terminated


include license files when publishing test-log-macros

Both the MIT and Apache-2.0 licenses require that redistributed sources contain a copy of the original license text.
Adding symbolic links in the macros directory should be enough to have cargo publish include the files (unless you're running "cargo publish" on an OS that does not support symbolic links).

Thank you!


DataClassContainsFunctions - allow boolean returning ExpressionBody functions

Expected Behavior of the rule
Disable check for alike functions:
fun isGrouped() = groupedAvps.isNotEmpty()
...
fun isEnum() = "Enumerated".equalsIgnoreCase(typeName)

Context
Allow boolean-returning ExpressionBody functions in data classes.
I feel like it doesn't have any negative impact on the data class, only makes it better by encapsulating boilerplate code (same checks in different places), and I reckon it's what most data classes have anyway, same as conversion functions

Thanks for the suggestion.
We won't introduce that for the following reasons. DataClassContainsFunction has a well defined behavior. If we introduce your feature, next week a another user will want to ignore functions ignoring functions returning, another one functions with one parameter of type Int and so on.
I recommend to use a custom rule, if you really want this feature.
You can copy the code of this rule and introduce this check.
Reference here


Unable to run patched rpcdaemon with docker-compose

Getting this error when trying to setup otterscan with my Erigon node: [EROR] [01-10|03:03:13.346] Could not connect to DB.  error="incompatible DB Schema versions: reader 3.0.0, database 5.0.0"

My Erigon is on v2021.12.03, running it with the given docker-compose file.
Used the tag v2021.12.02-otterscan from the patched repo.
Built the patched image using docker build -t otterscan -f Dockerfile .
Updated the docker-compose file to use the newly built otterscan image for rpcdaemon. This is the full compose file:

version: '2.2'

services:

  erigon:
    image: thorax/erigon:latest
    build: .
    command: erigon --metrics --metrics.addr=<IP_ADDRESS> --metrics.port=6060 --private.api.addr=<IP_ADDRESS>:9090 --pprof --pprof.addr=<IP_ADDRESS> --pprof.port=6061
    volumes:
      - ${XDG_DATA_HOME:-~/.local/share}/erigon:/home/erigon/.local/share/erigon
    ports:
      - "30303:30303/tcp"
      - "30303:30303/udp"
      - "30304:30304/tcp"
      - "30304:30304/udp"
    restart: unless-stopped

  prometheus:
    image: prom/prometheus:v2.30.2
    user: 1000:1000 # Uses erigon user from Dockerfile
    command: --log.level=warn --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus --web.console.libraries=/usr/share/prometheus/console_libraries --web.console.templates=/usr/share/prometheus/consoles
    ports:
      - "9090:9090"
    volumes:
      - ${ERIGON_PROMETHEUS_CONFIG:-./cmd/prometheus/prometheus.yml}:/etc/prometheus/prometheus.yml
      - ${XDG_DATA_HOME:-~/.local/share}/erigon-prometheus:/prometheus
    restart: unless-stopped

  grafana:
    image: grafana/grafana:8.2.2
    user: 1000:1000 # Uses erigon user from Dockerfile
    ports:
      - "3000:3000"
    volumes:
      - ${ERIGON_GRAFANA_CONFIG:-./cmd/prometheus/grafana.ini}:/etc/grafana/grafana.ini
      - ./cmd/prometheus/datasources:/etc/grafana/provisioning/datasources
      - ./cmd/prometheus/dashboards:/etc/grafana/provisioning/dashboards
      - ${XDG_DATA_HOME:-~/.local/share}/erigon-grafana:/var/lib/grafana
    restart: unless-stopped

  rpcdaemon:
    image: otterscan:latest
    command: rpcdaemon --datadir=/home/erigon/.local/share/erigon --private.api.addr=erigon:9090 --txpool.api.addr=erigon:9090 --http.addr=<IP_ADDRESS> --http.vhosts=* --http.corsdomain=* --http.api=eth,erigon,ots,web3,debug,net,trace,txpool --ws
    pid: service:erigon # Use erigon's PID namespace. It's required to open Erigon's DB from another process (RPCDaemon local-mode)
    volumes:
      - ${XDG_DATA_HOME:-~/.local/share}/erigon:/home/erigon/.local/share/erigon
    ports:
      - "8545:8545"
    restart: unless-stopped

Are you sure your erigon is on v2021.12.03? From the "incompatible db" it seems you ran erigon from the devel branch (unstable) which AFAIK triggers a db schema update. And since otterscan is built (currently) only from the stable tags, this causes this error check.
So, some questions:

how did you build/run the erigon executable? got it from docker hub or build it locally from the sources? if you build it locally, can you confirm the branch/tag?
do you intend to run erigon from devel/unstable?

@wmitsuda I got it from dockerhub. This is what it shows in my erigon logs:
[INFO] [01-11|16:24:12.861] Build info  git_branch=devel git_tag=v2021.12.03 git_commit=8203cdf4cd814a73af4608b54c4f8c57a6dd5aa1

I ran the docker compose as it was setup in the erigon repo and it pulled the latest image, tagged v2021.12.03

ok, good to know it worked! I'll close this issue then.
Also, I noticed from the log you posted the git_branch=devel message, I found it worrying that you got the unstable branch by accident.


TurbSim

Hi everyone, I installed TurbSim and I would like to create .bts file starting from a .inp one that I have...how can I do it?
Thanks a lot!

I tried to run my .inp file, but the following error occured:
C:\Users\167544>turbsim64 C:\Users\167544\Desktop\FAST30\WindFiles\GoF_NWP_5400s_8mps.inp
C:\Users\167544>C:\TurbSim\TurbSim64.exe C:\Users\167544\Desktop\FAST30\WindFiles\GoF_NWP_5400s_8mps.inp
Running TurbSim (v1.50, 25-Sep-2009).
Linked with the NWTC Subroutine Library (v1.01.09, 18-Jan-2008).
Reading the input file "C:\Users\167544\Desktop\FAST30\WindFiles\GoF_NWP_5400s_8mps.inp".
RanLux luxury level set by RLuxGo :    3 p =          223
RanLux initialized by RLuxGo from Seeds      123456           0           0
The number of the IEC 61400-x standard must be 1, 2, or 3.
Aborting TurbSim.
But, as you can see from the file attached, the number of the IEC 61400-x is 1, so what is the problem?
Thanks to all!
GoF_NWP_5400s_8mps.txt

Dear @Tommi1198,
Input IECstandard should be in quotes, e.g., "1".  You can also add the edition number, e.g., "1-Ed2".
Best regards,

Dear @jjonkman, as you can see from the file attached yesterday, IECstandard is 1...so, what is the problem?

Dear @Tommi1198,
Looking again, I also see that you are running TurbSim v1.50.  (Your input file says that it is formatted for TurbSim v1.21, but it doesn't actually look like your file is formatted for TurbSim v1.21.)  Regardless, your input file has two extra lines that are not used by TurbSim v1.50.  You must remove these lines for compatibility running TurbSim v1.50:
Line 31, UserFile
Line 37, ProfileFile
Alternatively, you could run the newest version of TurbSim available in OpenFAST, which would require that you upgrade your input file to the most recent format.
Best regards,

Dear @jjonkman, now TurbSim is running, but the .bts file is only 1kb...how is that possible?

Dear @Tommi1198,
Has TurbSim finished running?  Looking again at your TurbSim input file (assuming you didn't change anything else), your UsableTime = 999999999 s.  TurbSim will increase the AnalysisTime to accommodate this large UsableTime.  But such is simulation is so long that TurbSim may never finish.  It is much more common to run TurbSim for about 600 s.
Best regards,

Dear @jjonkman, TurbSim has finished to run and the .bts file is only 1kb…moreover, I had to use a very large UsableTime to prevent the exhaustion of the wind when running my simulation with FAST.Farm (I tried to put as UsableTime “All”, but it gave me an error, I don’t know why)

Dear @Tommi1198,
What does TurbSim say when you run it from the command prompt when a 1kb *.bts file is generated?
TurbSim v1.50, which you are running, does not support the UsableTime = "ALL" option.  For that, you must upgrade to TurbSim v2.0 or newer.  The original release of TurbSim v2.0 is available here: https://www.nrel.gov/wind/nwtc/turbsim.html.  This version was incorporated into this OpenFAST github repository and a few updates have been made since.  So, you can also use the version of TurbSim available in this github repository.
Best regards,


FileChannelVol.tryAvailable(long) does not test if storage is available (1.0.7)

The existing tryAvailable method looks like this:
        @Override
        public boolean tryAvailable(long offset) {
            if(hasLimit && offset>sizeLimit) return false;
            if(offset% chunkSize !=0)
                offset += chunkSize - offset% chunkSize; //round up to multiply of chunk size

            if(offset>size)synchronized (growLock){
                try {
                    channel.truncate(offset);
                    size = offset;
                } catch (IOException e) {
                    throw new IOError(e);
                }
            }
            return true;
        }

The FileChannel.truncate method does not grow the file if the new size is larger than the current size. As far as I can tell the call to truncate above will never do anything and does not fulfill the methods contract to test if the storage will be available.
I believe that:
channel.truncate(offset)
should be replaced with
raf.setLength(offset)
For reference, the JavaDoc:
https://docs.oracle.com/javase/7/docs/api/java/io/RandomAccessFile.html#setLength(long)
https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html#truncate(long)

Fixed in 2.0 branch. Not going to fix in 1.0 branch, since this is not critical error (FileChannel grows if data are written beyond EOF). And it could cause change in behaviour


Values reducers

Instead of native javascript arithmetic operators, use ops.add and ops.sub in reducers. This ensures that the operations are type-checked and that adding incompatible types or types for which the operation is not defined, throws an error, instead of being silently converted.

cc: @dmajda

I went through reducers.js and I think these instances still need to be converted:

result in avg — one can average e.g. durations since duration + duration and duration / number are both defined
result in sigma — I don’t think this line can get executed for anything but number (because all other types will fail to type check in update already), but I don’t think we want to embed this knowledge into the reducer

+1 with the comments addressed

+1

For the record, we had a discussion on slack whether math.sqrt in sigma should use JS native Math or Juttle wrapper. We decided to call Juttle sqrt for consistency (i.e. reducers should not care / be aware of which operations are defined for which types - that's a responsibility of modules/ops).
Will merge on green.


replaceDirnamePlugin will crash while string includes import.meta.url

I find bundle-require uses a custom esbuild plugin to handle path, source code is:
export const replaceDirnamePlugin = (): EsbuildPlugin => {
  return {
    name: "bundle-require:replace-path",
    setup(ctx) {
      ctx.onLoad({ filter: JS_EXT_RE }, async (args) => {
        const contents = await fs.promises.readFile(args.path, "utf-8")
        return {
          contents: contents
            .replace(/[^"'\\]\b__filename\b[^"'\\]/g, (match) =>
              match.replace("__filename", JSON.stringify(args.path)),
            )
            .replace(/[^"'\\]\b__dirname\b[^"'\\]/g, (match) =>
              match.replace(
                "__dirname",
                JSON.stringify(path.dirname(args.path)),
              ),
            )
            .replace(/[^"'\\]\bimport\.meta\.url\b[^"'\\]/g, (match) =>
              match.replace(
                "import.meta.url",
                JSON.stringify(`file://${args.path}`),
              ),
            ),
          loader: inferLoader(path.extname(args.path)),
        }
      })
    },
  }
}

Here has a problem while a string including keywords like __filename, __dirname or import.meta.data, the path in string is handled too. See the example in tsup.config.ts using banner: https://github.com/egoist/tsup/issues/719.
So maybe we need to add logic to exclude path in string. But regexp match seems hard to judge if the path reference is in string or not. Is there any quick way to fix this problem? AST may do, but it complicates the logic of the plugin.

figured out a way to do this using esbuild's define option and inject some code by peeking Vite's source code: https://github.com/vitejs/vite/blob/51ed05915ae1fcebacd5bcebca76559a2b8e4473/packages/vite/src/node/config.ts#L1013


zero tendencies

We found that we need to zero out the tendencies in the beginning of implicit_tendency_special as the EDMF components are coded assuming that it is done before updating tendencies.

bors r+

Bors r+

bors r+


Netty threads taking huge amount of heap space

We are facing an issue in production where netty threads take hug amount of heap leading to lots of Full GCs. We are using Redisson- java client which uses Netty.
We noticed netty threads used by redisson consumed 12GB of total 16GB of allocated heap space in one application and 8GB out of 16 GB on another application . One observation i have made is that netty threads had created around 275,000 thread local maps with each consuming around a fixed amount of heap. It all seems to be related with weblogic context aware classloader because it is shared by all the netty threads.
Since the heap dump is quite large I will be attching the screenshots.
Netty Version : 4.1.25
JDK - 1.8
App server - weblogic 12c

Which version of netty? Also I think I will need the heap dump to analyze this :(

Am 10.03.2019 um 09:06 schrieb shivendra-kumar<EMAIL_ADDRESS>We are facing an issue in production where netty threads take hug amount of heap leading to lots of Full GCs. We are using Redisson- java client which uses Netty.
We noticed netty threads used by redisson consumed 12GB of total 16GB of allocated heap space in one application and 8GB out of 16 GB on another application . One observation i have made is that netty threads had created around 275,000 thread local maps with each consuming around a fixed amount of heap. It all seems to be related with weblogic context aware classloader because it is shared by all the netty threads.
Since the heap dump is quite large I will be attching the screenshots.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Netty version is 4.1.25. Heap dump is very large around 30 Gbs. Also this issue persists even after multiple system restarts. I can share screenshots of specifc areas if that can help.

That’s a very old netty version. Please upgrade to the latest first and let me know if the problem still persist

Am 10.03.2019 um 09:32 schrieb shivendra-kumar<EMAIL_ADDRESS>Netty version is 4.1.25. Heap dump is very large around 30 Gbs. Also this issue persists even after multiple system restarts. I can share screenshots of specifc areas if that can help.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Is there a reported issue of memory consumption in Netty - 4.1.25? If yes, can you please share the link?

@shivendra-kumar we removed the ObjectCleaner as it had some problems. So yes it could been fixed already.

have you been live reloading your app frequently while it's running?

No we don't use live reloading. We just deployed the application and let users use it for a day. By the end of the day netty memory consumption reaches almost 80-90% of the total allocated heap.

Simulated it locally now attaching the heap dump.

@shivendra-kumar like I said please upgrade netty and report back. ObjectCleaner was removed due various issues.

Attaching Heap Dump Screenshots

@shivendra-kumar again please upgrade first and then re-run.

Closing as no response from reporter for almost 3 weeks. Please re-open if you still think there is any issue.

@normanmaurer Sorry for the super late response.  Upgarding netty did certainly help as the above issue is not coming anymore

@shivendra-kumar thanks a lot for confirming :)


[docs] Fix documentation for building Flink with Scala 2.11 or 2.10

This fixes some dead anchor links and aligns the text in "Build Flink for a specific Scala version" against the commands required by the current master.

Looks good to me. +1 for merging. Maybe we could also add some documentation for how to change the Scala version with your favourite IDE as described by you in https://issues.apache.org/jira/browse/FLINK-2858.

+1 looks good to merge.

Will merge this PR

:+1:

@tillrohrmann The list of TODOs from the JIRA issue is referring concretely to IntelliJ. Is there some Eclipse user who can see what the corresponding actions for Eclipse would be after I make a PR for that.
Also, I think that this belongs to a part of the documentation related to development.

I agree, it makes sense to add the documentation as a separate PR. I think here in the office, all of us are using IntelliJ by now. But I could try to get my old Eclipse setup going again.


Add webhoook report.

Add a report for a webhoook when an extraction queues/starts/finishes(success/error). This will tie into discordnotifier.com.

A working beta version is available now that should resolve this issue. Please try it out and let me know how it works out for you. https://github.com/davidnewhall/unpackerr/releases/tag/v0.7.2-beta7
On docker: docker pull golift/unpackerr:0.7.2-beta7


PROBLEMS WITH LIBRARY | ERRORS ON CODE

Hi, im using your library, well im making tests with it and i found errors on the code.
On the file S3MediaTrait.php i found this:
FIRST: Is not $this->hasError is $this->hasErrors();
ORIGINAL WRONG CODE:

GOOD CODE:

SECOND: the variable result doesnt exists
ORIGINAL WRONG CODE:

GOOD CODE

both things on the same function "saveUploadedFile"
By the way, thanks for the extension

Checking the repo files, It seems to ve something with the composer. I installed the latest version.


Examples are not parsed correctly

I am trying to fetch the english definition for "to grapple".
When doing so, wiktionary parses the examples incorrectly. In particular, in the first row there are all the examples separated by a '\n', and then from the second row on the same examples are repeated:

            "examples": [
                "to grapple with one's conscience",
                "Hakluyt\nThe gallies were grappled to the Centurion.Shakespeare\nGrapple them to thy soul with hoops of steel.",
                "The gallies were grappled to the Centurion.",
                "Grapple them to thy soul with hoops of steel."
            ],

Should be fixed in 6e2643526b1012b2ad7fb0f281af7eef2b847d48


What am i doing wrong when im getting 403 Forbidden?

Some context
Im running a graphql api that runs on unauthenticated service.
That graphql shall call an authenticated service in the same cloud run project.
Iv added my project has as env to the service

Im adding runsd to the container, and it runs fine, as I reach the container and get a response about the url is forbidden

May the problem be sub urls? Or how can I go about debugging where the issue might be?

Sub url did not change anything
I dont know where this HTML is coming from, as my service dont serve up anything like that.
It does seem some kind of google reposnse

It is generic response from Google's frontend.

did you configure the appropriate svc account and permissions on it so that that svc account you use to run the service can call this service?
there are ways to increase logs from runsd described in Troubleshooting section on readme. but I don't think that is the problem here.

@ahmetb ah, yeah I did not do the service account bit of the authorization. Was unclear if that was needed from the documentation here. I will try that out tomorrow.

Yes I am referring to that.
This tool makes only predictable URLs and not having to write authentication code easier.
IF you customize svc accounts you still will need to configure IAM with the right permissions.

@ahmetb thanks for leading me in the right direction. I got it to work just fine
It might help to add some toughts on it in the readme if noobs like me keep popping into this tool. I will add my path below.
I first created a service account because this will be used different places. I do this via terraform.
I then added that service account to the run service: https://cloud.google.com/sdk/gcloud/reference/run/deploy#--service-account. I use terraform so simply initiate and add on the same place of code.
I then had to do what was mentioned before: https://cloud.google.com/run/docs/authenticating/service-to-service#gcloud. But since I use terraform, I could do this easily after the run service creation with this command: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_service_iam, by fetching the service account data using this attribute and ID: https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/service_account
Hope this can help other noobs