VS Code doesn't fail unittest test if a segmentation fault occurs during test

Type: Bug

Behaviour
Expected vs. Actual
Expected:
If a segmentation fault occurs during a test, VS Code should fail the test and show a red cross next to it.
Actual:
If a segmentation fault occurs during a test, VS Code does not necessarily show the test as failed. Specifically, if the test passed the last time it was run, VS Code continues to show a green tick next to it.
Steps to reproduce:
I encountered this issue while working on a Cython extension. A minimal working example can be found in this repositiory. It needs Cython and can be compiled with python setup.py build_ext --inplace --force

Compile as is and run the test from the testing panel. We expect the test to pass and VS Code correctly shows the test as passing with a green tick.

To confirm it passed, we can run it manually from the command line using python -m unittest discover .

(venv) user@computer:~/temp/vscode-python-bug$ python -m unittest discover .
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
(venv) user@computer:~/temp/vscode-python-bug$

Uncomment # ptr[0] = 5 in testissue.pyx in an attempt to cause a segmentation fault by dereferencing a null pointer and recompile. If we rerun the test in VS Code, VS Code continues to show a green tick

To confirm the test did not pass, we can run it manually from the command line. On my system I get a segmentation fault

(venv) user@computer:~/temp/vscode-python-bug$ python -m unittest discover .
Segmentation fault (core dumped)
(venv) user@computer:~/temp/vscode-python-bug$

Diagnostic data

Python version (& distribution if applicable, e.g. Anaconda): 3.11.2
Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-07-19 12:37:59.186 [info] Experiment 'pythonPromptNewFormatterExt' is active
2023-07-19 12:37:59.186 [info] Experiment 'pythonPromptNewToolsExt' is active
2023-07-19 12:37:59.186 [info] Experiment 'pythonTestAdapter' is active
2023-07-19 12:37:59.186 [info] VS Code was launched from an activated environment: 'venv', selecting it as the interpreter for workspace.
2023-07-19 12:37:59.186 [info] Test server listening.
2023-07-19 12:37:59.186 [info] Python interpreter path: ./venv/bin/python
2023-07-19 12:37:59.725 [info] > ./venv/bin/python -I ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/interpreterInfo.py
2023-07-19 12:37:59.759 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
2023-07-19 12:37:59.759 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
2023-07-19 12:37:59.759 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
2023-07-19 12:38:01.874 [info] Starting Pylance language server.
2023-07-19 12:38:02.330 [info] Discover tests for workspace name: vscode-python-bug - uri: /home/tom/temp/vscode-python-bug
2023-07-19 12:38:02.330 [info] Running discovery for unittest using the new test adapter.
2023-07-19 12:38:02.495 [info] > . ./venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/printEnvVariables.py
2023-07-19 12:38:02.495 [info] shell: bash
2023-07-19 12:38:02.574 [info] Running unittests with arguments: /home/tom/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/discovery.py,--port,36469,--uuid,3b3fd3e7-6de2-4435-afce-bc879497de2d,--udiscovery,-v,-s,.,-p,test*.py

2023-07-19 12:38:02.574 [info] > ./venv/bin/python ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/discovery.py --port 36469 --uuid 3b3fd3e7-6de2-4435-afce-bc879497de2d --udiscovery -v -s . -p test*.py
2023-07-19 12:38:02.574 [info] cwd: .
2023-07-19 12:38:02.654 [info] Test server connected to a client.
2023-07-19 12:38:02.658 [info] Using result resolver for discovery
2023-07-19 12:38:12.815 [info] Running UNITTEST execution for the following test ids: test.Test_f.test
2023-07-19 12:38:12.815 [info] Server listening on port 40973
2023-07-19 12:38:12.842 [info] Discovering unittest tests with arguments: /home/tom/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py,--port,36469,--uuid,ebda9941-43af-4139-bbf3-ef66eb9ea365,--udiscovery,-v,-s,.,-p,test*.py

2023-07-19 12:38:12.842 [info] > ./venv/bin/python ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py --port 36469 --uuid ebda9941-43af-4139-bbf3-ef66eb9ea365 --udiscovery -v -s . -p test*.py
2023-07-19 12:38:12.842 [info] cwd: .
2023-07-19 12:38:12.921 [info] Test server connected to a client.
2023-07-19 12:38:12.927 [info] Client disconnected
2023-07-19 12:38:39.928 [info] Running UNITTEST execution for the following test ids: test.Test_f.test
2023-07-19 12:38:39.928 [info] Server listening on port 39645
2023-07-19 12:38:39.943 [info] Discovering unittest tests with arguments: /home/tom/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py,--port,36469,--uuid,3463e9fa-c3f7-42f9-98b3-3ec957fbd36e,--udiscovery,-v,-s,.,-p,test*.py

2023-07-19 12:38:39.943 [info] > ./venv/bin/python ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py --port 36469 --uuid 3463e9fa-c3f7-42f9-98b3-3ec957fbd36e --udiscovery -v -s . -p test*.py
2023-07-19 12:38:39.943 [info] cwd: .
2023-07-19 12:38:40.200 [info] Client disconnected
2023-07-19 12:45:53.769 [info] Running UNITTEST execution for the following test ids: test.Test_f.test
2023-07-19 12:45:53.771 [info] Server listening on port 44185
2023-07-19 12:45:53.803 [info] Discovering unittest tests with arguments: /home/tom/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py,--port,36469,--uuid,2994032d-1b5e-428f-96ca-0a10eae8376e,--udiscovery,-v,-s,.,-p,test*.py

2023-07-19 12:45:53.803 [info] > ./venv/bin/python ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py --port 36469 --uuid 2994032d-1b5e-428f-96ca-0a10eae8376e --udiscovery -v -s . -p test*.py
2023-07-19 12:45:53.803 [info] cwd: .
2023-07-19 12:45:53.879 [info] Test server connected to a client.
2023-07-19 12:45:53.889 [info] Client disconnected
2023-07-19 13:07:33.104 [info] Running UNITTEST execution for the following test ids: test.Test_f.test
2023-07-19 13:07:33.104 [info] Server listening on port 40019
2023-07-19 13:07:33.186 [info] > . ./venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/printEnvVariables.py
2023-07-19 13:07:33.186 [info] shell: bash
2023-07-19 13:07:33.252 [info] Discovering unittest tests with arguments: /home/tom/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py,--port,36469,--uuid,c2d59644-df4b-4e1e-addc-93d5f0620213,--udiscovery,-v,-s,.,-p,test*.py

2023-07-19 13:07:33.252 [info] > ./venv/bin/python ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py --port 36469 --uuid c2d59644-df4b-4e1e-addc-93d5f0620213 --udiscovery -v -s . -p test*.py
2023-07-19 13:07:33.252 [info] cwd: .
2023-07-19 13:07:33.339 [info] Test server connected to a client.
2023-07-19 13:07:33.355 [info] Client disconnected
2023-07-19 13:08:12.259 [info] Running UNITTEST execution for the following test ids: test.Test_f.test
2023-07-19 13:08:12.260 [info] Server listening on port 37219
2023-07-19 13:08:12.285 [info] Discovering unittest tests with arguments: /home/tom/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py,--port,36469,--uuid,a3e291fd-43a5-49f6-866b-387641187548,--udiscovery,-v,-s,.,-p,test*.py

2023-07-19 13:08:12.286 [info] > ./venv/bin/python ~/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/unittestadapter/execution.py --port 36469 --uuid a3e291fd-43a5-49f6-866b-387641187548 --udiscovery -v -s . -p test*.py
2023-07-19 13:08:12.286 [info] cwd: .
2023-07-19 13:08:12.533 [info] Client disconnected

User Settings

languageServer: "Pylance"

testing
• unittestArgs: "<placeholder>"
• unittestEnabled: true

Extension version: 2023.12.0
VS Code version: Code 1.80.1 (74f6148eb9ea00507ec113ec51c489d6ffb4b771, 2023-07-12T17:22:25.257Z)
OS version: Linux x64 6.2.0-25-generic
Modes:

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-4570T CPU @ 2.90GHz (4 x 3292)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okvideo_decode: enabledvideo_encode: disabled_softwarevulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: disabled_off

Load (avg)
1, 1, 1

Memory (System)
3.72GB (1.34GB free)

Process Argv
. --crash-reporter-id 20f7e162-f0a7-4674-99e7-f4ed2b83828b

Screen Reader
no

VM
0%

DESKTOP_SESSION
xubuntu

XDG_CURRENT_DESKTOP
XFCE

XDG_SESSION_DESKTOP
xubuntu

XDG_SESSION_TYPE
x11

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931cf:30280410
vshan820:30294714
vstes263:30335439
vscoreces:30445986
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30410667
py29gd2263:30792226
vsclangdc:30486549
c4g48928:30535728
dsvsc012cf:30540253
pynewext54:30695312
azure-dev_surveyone:30548225
vsccc:30610678
282f8724:30602487
f6dab269:30613381
showlangstatbar:30737416
vsctsb:30748421
03d35959:30757346
pythonfmttext:30731395
pythoncmv:30756943
fixshowwlkth:30771522
showindicator:30785052
pythongtdpath:30769146
i26e3531:30792625
pythonnosm12tcf:30779713
pythonidxpt:30784022
pythonnocebcf:30776496
h7j2d465:30786200
dsvsc013:30789517
cmakesidepanelv2:30779593

will investigate thanks!


Updated so the Alert shows when you only have closed courses.

Thought of a case where this doesn't work as intended.

I tested this case and it works. I was worried when a closed course isn't hidden and the alert would still show up. It doesn't show up in this case. So perfect.

That is correct.


Error while building on Linux

Hi,
When trying to build the new version v3.1 on linux, I get this error:
/usr/bin/ld: /tmp/user/0/ccg9Nw8T.o: in function `GuiTextBoxMulti':
rfxgen.c:(.text+0x5350): undefined reference to `GetCodepointNext'
/usr/bin/ld: /tmp/user/0/ccg9Nw8T.o: in function `GetTextWidth':
rfxgen.c:(.text+0xbab7): undefined reference to `GetCodepointNext'
/usr/bin/ld: /tmp/user/0/ccg9Nw8T.o: in function `GuiDrawText':
rfxgen.c:(.text+0xc5dc): undefined reference to `GetCodepointNext'
collect2: error: ld returned 1 exit status

I build it using this oneliner from one of the previous issues:
gcc -o rfxgen rfxgen.c external/tinyfiledialogs.c -s -Iexternal -lraylib -lGL -lm -lpthread -ldl -lrt -lX11 -L../raylib/release/libs/linux -std=c99 -no-pie -D_DEFAULT_SOURCE
Although, raylib is already installed, so any ideas ?

@jJit0 You have to use latest version of raylib from github master branch. GetCodepointNext() was added a couple of weeks ago.


chore: convert to proper Typescript composite project

This project has pretty cumbersome TypesScript setup with use of /dev/null as outDir and lerna for parallel compilation. While all these workarounds are results in some output there are proper documented way to do this better.
This PR converts the TS setup into a composite project improving the DX and IDE performance for type checking.
In additional, as we are here, we increasing strictness of TS checks and that catches few minor problem (unused arguments and reference to DOM type) and one major problem (require of ESM module). Major problem is just silenced as I'm not sure there it's a dead code or should be mitigated in a separated effort.
As part of this PR we also upgrading to TS 5.3 (that requires upgrade of eslint-config-standard-with-typescript and @typescript-eslint/* packages) and improving ESLint settings related to TS.
This is also moves typescript from peerDependencies (where it doesn't belong) to devDependencies.

@tinovyatkin are we going to do something here?


Could you give a specific example about "$ ./prep_all_data.sh <HUMAN_FRIENDLY_DATASET_NAME> path/to/data/file.json"

I don’t understand how to fill in the last two parameters.
Can you give a specific example?
Thanks.

Sure, here is an example command:
./prep_all_data.sh clothing ./reviews_Clothing_Shoes_and_Jewelry_5.json

With this command, my intent is to preprocess the 5-core version of Amazon's clothing, shoes, and jewelry dataset downloaded from Prof. McAuley's webpage. The script will then create the data files under the directory ./data/clothing/ since we provided the human-friendly name as clothing.
Does that help? If not, please feel free to comment with any further questions you have.
Cheers,
Noveen

Thank you very much for solving my problem!


Reduce allocations for request retry logic.

Using ioutil.ReadAll() to read the request body can introduce a large
number of allocations increasing heap pressure.  For cases where the
content length of the request is known we can read the body into an
allocated buffer of exact size.
Update go-autorest dependency to 8.1.0 to use RetriableRequest feature.

Beautiful :)

@marstr @mcardosos can you please take a look at the latest iteration?


Error 500 when scanning directory

Description
When we need to scan a directory, we get an error 500.
Screenshots

Environment

[x] Production
[x] Staging

@laurent-laporte-pro Can you precise what's the version whom fixe this issue ? Because it's still present

I have tested the scanner and it's now fixed in staging and integration environment but not in production. That's because the branch dev contains the fix but not the branch master. @laurent-laporte-pro, is there a particular reason for that ?

This issue was classified as a bugfix rather than a hotfix, so the correction is currently only available in the dev branch. @makdeuneuv, should we reconsider the priority of this issue and treat it as "hot"? In that case, I will create a branch hotfix/v2.14.5 to include the fix for this bug.


Make DynamoDB tables an output from cf templates

Is is possible to set the dynamodb table name/streamarn as an output from the amplify cli. I need access to the tables from another CF based deployment.

@benoj currently the table name and region is exported on aws-exports.js file on the aws_dynamodb_table_schemas attribute.

Thanks @elorzafe - I'm trying to write a lambda function which listens to the dynamodb stream using the serverless framework.  This unfortunately requires that I reference the cf output or ARN - which is a timestamp value and therefore likely to continuously change. By adding the tablename, and stream arn to the outputs in the CF template will allow me to reference the stream arn by referencing the cf-stack in aws.
I won't be able to use the aws-exports.js from my serverless.yml

I am marking this as an enhancement. We do store the ARN in  <project-root>/amplify/backend/amplify-meta.json which you could use in the mean time

+1 for the feature request.
Also, I don't see the aws_dynamodb_table_schemas in aws-exports.json neither that in <project-root>/amplify/backend/amplify-meta.json.
My tables are AppSync resolvers.
I actually don't see the table names anywhere in my project folder.

+1. Also currently need lambda created within amplify to read from the dynamodb table created by the graphql transformer.

We released this feature as a part of our latest CLI version release - 4.16.1.
You can checkout the documentation out here for more information - https://aws-amplify.github.io/docs/cli-toolchain/quickstart#as-a-part-of-the-graphql-api-types-with-model-annotation


Add Questions to form

@stariel,
-Completed adding questions to google form and added to form.html

Further questions added to form.


Update java-util to 0.27.11

More deterministic unmapping (https://github.com/metamx/java-util/pull/51)
GzipInputStream attempts to use super.available() first (https://github.com/metamx/java-util/pull/53)
Remove reflection from ByteBufferUtils (https://github.com/metamx/java-util/pull/52)
Add syntactic sugar for catching bad ordering of log messages (https://github.com/metamx/java-util/pull/44)

@drcrallen would it be worth putting this in 0.9.2 or should we hold this until that branch is cut?

@gianm I don't think anything here has to be in 0.9.2. Or more precisely, there isn't anything here that would warrant risking a regression bug in 0.9.2 this late in the release cycle, as such I have labeled it for 0.9.3.

This is no longer needed


Adding a hoistable attr interface to allow attaching attrs to hoists.

This could use a bit of refinement but allows for scoped attributes like device affinity to be associated with the globals and initializers created by hoisting. In the future when we perform device placement later on we'll not require this but we'll always want to allow a user to indicate a particular value should live on a particular device. We may want loosen the behavior by only hoisting the attribute to the global and letting placement decide on which device it should be initialized.

To start device placement is not done - the user must assign devices to all top-level ops and any without an assignment will fall back to using the module-level default. This is about hoisting where we pluck an operation out of a random container op and create a new global and initializer for it - we need that new global and initializer to have the same device as the original op did.
In the future when we do placement ourselves (or partial placement) we can allow the user to leave some ops without a device specified - but that's future stuff. This interface could still be useful for other attrs we stick on functions (rounding modes, whatever), and will always be needed to preserve the explicit behavior (if the user says a particular linalg op needs to run on a particular device and we hoist it we need to ensure it stays on that device, regardless of any other automatic placement we may do around it).


cannot create gestures

create_gestures.py is giving an error when opening the camera and trying to find contours which it cannot find.

create_gestures.py is giving an error when opening the camera and trying to find contours which it cannot find.

did u find the solution ????

For this issue you need to change the line in create_gestures.py
from
contours = cv2.findContours(thresh.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)[1]
to
contours = cv2.findContours(thresh.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)[0]
also you need to click 'c' as soon as the camera shows up.


Decompiling Farming Simulator 17 .l32/.l64 scripts

Hi,
I figure you probably don't work on this anymore but you are the best lead I have right now ^^.
I saw this and managed to get some of the FS15 scripts decompiled using ldj 2.1, Thanks! and great work.
I moved my attention to FS17 and it seems that this game has moved to a new model of storing their Luc data. Each script is now represented by a .l32 and .l64 file.
When using your ldj 2.1 branch I get the error:
Unknown flags set:<PHONE_NUMBER>00 Failed to read raw-dump header
It figures i guess that they have changed the headers between game versions.
I was wonder if you have time maybe you could have a quick look over these files and perhaps point me in the right direction?
I am hoping it may only require that I add new or modified headers to the ldj2.1 branch.
FS15 Samples:
FS15 luc files.zip
FS17 Samples:
FS17 luc files.zip
Thank you :)

When using your ldj 2.1 branch I get the error:

On which files?
Does this work with *.l32 files and not *.l64 files in fs17 or
does it work with fs15 files but not fs17 files?

Hey, Sorry it works for FS15 and not FS17, Neither .l32 or .l64 decompiled for me.

@jjdredd If you are working on a decompiler for files from FS17, I can provide source files and compiled from this game (version <IP_ADDRESS>).

I have very limited time so at this point I just hope it's an easy fix.

@KITT3000 What do you mean by Source files? the .l32/.l64 files or the Lua source?? :O
@jjdredd I understand, thanks for any help you can provide :)

Ok, I've added the new flag, however that doesn't seem to help.

@thefoofighter In one update, Giants left the ".lua" files before the compilation.

@KITT3000 so they did! Thank you for that info. Having the source files should help alot when making decompiler. Not expecting you to do it @jjdredd but perhaps someone someday might get a chance,
@KITT3000 you hardly have binary/ida/x64dbg knowledge? I have used them on a few occasions to unlock abandonware but reverse engineering a compiled and maybe even encrypted format looks tough..
@jjdredd what was the new flag?

@thefoofighter Unfortunately not.

They could've changed the bytecode dump format. A binary of the game would help if it isn't stripped.
And try not to break the tos/law by posting such files on GH.

well it isn't really "new" https://github.com/jjdredd/ljd/commit/90fefbd61d5326d28854240bac3cd41cc4f12a24

Hello,
I have a problem using the decompiler, can you help me?
`Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 17:26:49) [MSC v.1900 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.

== RESTART: C:\Users\pierre-antoine\Downloads\ljd-ljv2.1\ljd-ljv2.1\main.py ==
Traceback (most recent call last):
File "C:\Users\pierre-antoine\Downloads\ljd-ljv2.1\ljd-ljv2.1\main.py", line 123, in
retval = main()
File "C:\Users\pierre-antoine\Downloads\ljd-ljv2.1\ljd-ljv2.1\main.py", line 75, in main
file_in = sys.argv[1]
IndexError: list index out of range

C:/Users/pierre-antoine/Downloads/FS17.luc.files/FS17 luc files/Gui.l32
SyntaxError: invalid syntax
`

@Pierrot08  That is because the decompiler doesn't currently work on FS17 compiled lua code.

ha sorry
my error above that I launched the decomplileur in python and not in CMD
For luc no worries ^^ but for the same error as you, hoping that you find a solution
Sorry for the inconvenience

@Pierrot08 Np, if you want to have a look at the source get a hold of the 1.3.1 update and using universal extractor dump the contents to a folder, find the .gar files and extract with quickbms & giants config.
The .lua's are intact and i doubt there are too many major changes to 1.5.1
Good source to learn how Giants made some things to help with modding ^^
Good Luck :)

Any update on this and progress towards reading FS19's L32/L64 files?


Improve package building and packing for @fluid-experiemental/property-inspector-table

Reduce the usage of webpack
Using similar webpack configuration to other react libraries in the repo
Remove hacks to load the SVGs

Note - for now webpack is used only for running the storybook and bundling SVGs due to the way its consumed in the library itself - this may change in the future, however, it will require further investigation.
@CraigMacomber FYI

@evaliyev  PTAL

@CraigMacomber @DLehenbaue can we merge this PR ?


Where is the documentation for VSTS REST API 3.0?

How to find previous version of VSTS REST API
Recently the VSTS REST API docs were updated to version 4.1, for example: https://docs.microsoft.com/en-us/rest/api/vsts/git/pull requests/get pull requests
Where can I find the version 3.0? I have clicked in the Previous Version Docs link in the button of the page, but I cannot find the previous versions for VSTS docs. I am guessing there should be a GitHub repository as the contributing page mentions, but I cannot find the Edit button for these pages.
Any idea how can I get to the previous _apis/git/repositories/repoId/pullRequests docs?
Thanks!

Older versions of our rest apis can be found here https://docs.microsoft.com/en-us/vsts/integrate/legacy-apis/overview?view=vsts .

Hi @wnjenkin , can you help me find the old documentation again? The link now returns 404 Not Found.

https://docs.microsoft.com/en-us/vsts/integrate/previous-apis/overview?view=vsts .

@kiewic Whitney provide a new link. Please take a look.

Nice! Thank you guys

Hello @wnjenkin, if you can help me with the documentation for below REST API.
POST https://{accountName}.portalext.visualstudio.com/_apis/continuousdelivery/provisioningconfigurations?api-version=3.2-preview.1
Thanks


Replace psycopg2 requirement with psycopg2-binary

Its better because we don't have to build it ourselves.

yeh got it

,,


Retrieve the correct metadata boilerplate if "Status" is specified.

In commit 9410d77, config.retrieveBoilerplateFile was changed to, among other things, use doc.md.status rather than doc.md.rawStatus as the default status specifier. However, when the default and computed-metadata boilerplates are retrieved, doc.md.status is set to None since the implicit metadata hasn't been computed yet, which causes the default boilerplate files to load rather than those for the specified status.
Fixes #1818 and fixes whatwg/whatwg.org#348.

Nice catch, thanks!


tagdetails: View details of a tag (PROJQUAY-3545)

Made the following changes:

Added page to view details about a particular tag
Manifest list tags can switch between architectures
Taglist table now links to tag detailspage
TagResource now accepts specificName parameter to get details for a single tag
NavigationPath now has helper methods for getting domain and constructing tagdetails path

Running locally:
Run with MOCK_API=true npm start and go to the organizations/testorg/testrepo page. Selecting a tag there will go to the tag details page.

few comments:

is the OS column mocked? If we don't have data for it, we should remove it.
can you make the Size column to be human readable maybe use something like (this lib)[https://www.npmjs.com/package/pretty-bytes] or a small function in utils.js
can you replace the expires-mock with actual value
The colum name for Last Modified gets hidden is it possible to wrap it to a new line?
In the filter dropdown, are we doing filtering by OS and Size? If not please feel free to remove them

for some reason I can't see values on this page

few comments:

is the OS column mocked? If we don't have data for it, we should remove it.
can you make the Size column to be human readable maybe use something like (this lib)[https://www.npmjs.com/package/pretty-bytes] or a small function in utils.js
can you replace the expires-mock with actual value
The colum name for Last Modified gets hidden is it possible to wrap it to a new line?
In the filter dropdown, are we doing filtering by OS and Size? If not please feel free to remove them

These are items that we're aware of and will be addressed in a future PR. This PR is specifically for the tag details page.

for some reason I can't see values on this page

Is this using the mock API or a real DB?

@bcaton85  The creation is a timestamp, is that expected? Can you convert that to a string

Updated with conversion to datetime string considering timezone

We will look into configuring the hostname at runtime in a separate PR


at_hash is needed

I recently updated the library from 1.x to 2.x to use the refresh trick with the iFrame.
But after the update a problem occured: The tryLogin function wouldn't accept my tokens any more because they don't provide a at_hash.
Has anybody else experienced this issue? Is there a workaround?
Unfortunately I'm not in controll of how the token is built on the server side...

Which solution do you use on server side?
This validation is done b/c the oidc standard demands for it.
Would it help to provide a flag that allows to disable this?

One more question: Do you just receive an id_token or an id_token as well as an access_token?

We are using an Identity Service for Cloud Foundry (https://docs.cloudfoundry.org/api/uaa)
I think a flag would pretty much solve it for my case.
I recieve both an access_token and an id_token.

Ok, finde. I will include it into the next release, I'm currently working on. Should be ready during the next week.

Thank you very much!

The next version that will land on monday morning will have a config property disableAtHashCheck. You can set it to true to bypass this check. In addition to that, you could also use the NullValidationHandler instead of the JwksValidationHandler to avoid further validation steps.
But please keep in mind that you are disabling all the additional security aspects of OIDC. Technically, it's just OAuth2 + unchecked additional information defined by OIDC.

The release just landet. If you need further infos or adaptions, don't hesitate to contact me.

We have the same issue using ADFS 5.0 (Windows Server 2019).

I have the same issue as @RS-Sautter.

TypeError: Cannot read property 'replace' of undefined
at JwksValidationHandler. (angular-oauth2-oidc.js:122)

While using codeflow with JwksValidationHandler.


For each does not work with Databricks SQL endpoint

The [for each logic] does not work when using Databricks with a SQL endpoint

SODA-1296

Hi @vijaykiran and @jmarien : what is the plan for this bug? Can you communicate a timeline?

@JCZuurmond  Puling this into the latest sprint, fix will get into the next release.

Please test with 3.0.16 and let us know if this is fixed.


Mobile View - Number of Pages

If the number of pages for the collection of samples is more than can fit in the normal width for the screen, it will go off-screen and cause odd horizontal scrolling.

@v-aaronbode  - reopening as as missing any conclusions if the issue is fixed or when it has been released. Let's be a bit more clear on the communications, so that both teams knows what is happening and that the work done for testing is properly appreciated... and acknowledged and fixes properly communicated.
When we fix something - please let us know if things were fixed, skipped and when it's available in the web site. Just typical open-source communication model on being respectful on the people spending time here.


Comments following review

I'm copying a few comments here that I originally sent via e-mail, just so they're a bit easier to track and resolve. Responses from @cristian-recoseanu are noted underneath in italics.

connection_lost - I didn't understand this at first. I’d suggest adding a brief description and/or link to MQTT WILL. My interpretation (having read other docs) is that the connection_lost is sent on startup to tell the broker what to send out to clients if an ungraceful disconnect is detected. This intentionally has no timing data as it isn't known when it will get sent out (I wondered why this was missing at first). Do you also need to specify the topic this will be sent to? This may be the same topic as used for normal messages.

Your assumption is correct. Should we link to a more detailed explanation of MQTT WILL messages? The topic used is the same across all messages and is defined in the IS-05 parameter “broker_topic”

I’d suggest being clear that for WebSocket comms, even though there is one URL, each client gets its own communication channel with unique data. This is probably the standard expectation for a WebSocket, but it threw me slightly at first and I suspect it would throw others too.

Ok so basically clarify that each connection is independent and considered its own unique sessions with unique subscriptions and events sent?

The example event type bool/enum/OnOff seems inconsistent, should it be boolean/enum/OnOff?

Well spotted. That is indeed a mistake.

I’d appreciate recommendations for frequency of access to the API to get bounds of a numeric event type for example. Is this allowed to change over its lifetime without disconnecting all clients - it may be easier if not, then a client knows to re-query this upon reconnection?

I’m not sure I fully understand this one. Could you give an example?

Be clear with wildcards that you can't do stuff like number/Tempera* or number/*/C. I assume the * should replace whole fields only and always goes at the end?

Understood. The documentation does mention the wildcard always goes at the end.

I’d suggest re-wording the core models section slightly to assume that those changes are already part of IS-04/05 and specify the version of 04 and 05 required (v1.3 and v1.1 respectively).

Would that require we have the feature requests logged on Github for both IS-04 and IS-05? Is your suggestion to still keep the section to highlight the changes but write it as if it had already happened?

Remove AMQP from core models diagram

Again well spotted

'channels' needs removing from the core models Source example as this relates to the 'audio' schema

Understood

Is _nmos-mqtt TCP or UDP? Guessing it should be '_nmos-mqtt._tcp'? Others seem to use _mqtt._tcp, should we be using that too, or is there a reason you'd want a specific NMOS MQTT server?

It is indeed TCP. I think we wanted to be clear that it is the MQTT broker which is involved in NMOS. I don’t know what the odds are of a customer already having an MQTT broker or not.
I don’t have a strong opinion on this and we just made a decision so we could move on with the work. Would you recommend using the simple _mqtt._tcp instead?

A few more comments. Sorry for the volume, just want to make sure there's a note of these before the deadline:

The definition of the origin_timestamp is a little confusing. It sounds very similar to the creation_timestamp (I appreciate IS-04 isn't clear on this). If it helps the ID and Timing work is aiming at a definition which includes just one core timestamp (which sounds like how you've defined the creation_timestamp). Any additional timestamps are likely to be part of the Flow payload.

The flow_id isn't currently included in the messages sent over the MQTT/WebSocket link, only the source_id. As I think I've mentioned elsewhere previously, this means it's impossible to work out the Flow ID associated with the data. The most straightforward solution would be to include both the source_id and the flow_id in the 'identity' part of the message.

Is there any way the 'type' definitions could be done as JSON schemas rather than using a freshly defined way of identifying maximums, minimums etc? I feel like I may have asked this previously, so it might be that this isn't possible for the same reason that IS-05 has to define its own constraints. If that's the case, then just ensuring as much consistency as possible with IS-05's constraints mechanism would be good so that they can be merged into JSON schemas to aid validation, rather than writing our own validators.

Where the WebSocket health messages are defined, would it be worth using the same default times as for IS-04? I note that a 5 second periodicity is already used for health messages, but the timeout is specified as when two consecutive health messages are missed. Could a specific time be associated with this, ie. 12 seconds is used for IS-04 to cover this two missed heartbeats plus 2 additional seconds to account for inaccurate clocks and transport latency.

It might be worth copying a few of the boilerplate API docs from the IS-04 repository, not least the CORS requirements and trailing slash policy in order to keep things consistent. In time we'll hopefully put this in a separate spec, but for now it might be sensible to be explicitly clear that this applies here too.

Hi Andrew,
I quick reply to number 10. One example is in the health messages for websocket.
The creation_timestamp is sent by the server at the time of generating the response whereas the origin_timestamp is the original timestamp sent by the client in the health command.
You could use this information to measure a rough round-trip time.

Hi Andrew,
Regarding item 12 - the idea was to have something that would be simple to use and to allow for the recipient to easily figure out what the allowed values would be. If that would have been done through a JSON schema, it could potentially end up with very complex structures a vendor on the sending side could use in such a schema that would be easily used for validation, but not so much for getting the list of possible values on the receiving end. (e.g. there would be a number of ways of defining possible enum values in a JSON schema, especially using RegEx). The other thing I would like us to achiev in the future is easily building schemas for complex types what would just require the introduction of an "object and an "array" which would then allow for properties to be of one the basic types or another array or object (that was left out of the v1.0 spec to keep the scope limited). Unfortunately, that would not be compatible with the IS-05 way of defining constraints.
While I think that writing a validator that would validate such simple schemas (it is just three basic types with very limited constraints) is trivial, I will have a look if we might be able to define something that would be a subset of the JSON schema and make it immediately usable for validation (as it would be a JSON schema itself) or at least come with a straight forward translation of that into a valid JSON schema that could be then used for validation.
It might be also useful to discuss this over a call, maybe early next week by which time we should ideally be able to finalize all the details.

I am very late to the party in reviewing Event & Tally, sorry - not enough bandwidth!
Andrew's item 12 jumped out at me as well. Using JSON Schema for type definitions, or a similar kind of cut-down JSON Schema that we adopted for IS-05 constraints, looks to me like it ought to work well here. The first thing that jumped out at me was seeing “min_length” rather than “minLength” for strings, and then the different way of defining enums, and rationals.

broker_topic vs. rest_api_url
I'm not an MQTT expert; would it make sense to add "events" before "source" in the broker_topic, to clearly indicate this spec, and possibly the version too?
E.g.
{
  "broker_topic": "x-nmos/events/v1.0/sources/{sourceId}",
  "rest_api_url": "http://hostname/x-nmos/events/v1.0/sources/{sourceId}/"
}

I also wonder whether using "href", or "events_href", rather than "rest_api_url", would be more consistent with the naming of similar URL properties in IS-04?

Regarding item 11: I have now added flow_id to the messages sent over one of the transports. Please note that no flow_id will be returned in the JSON when returning values through the REST API as there is not necessarily a 1:1 relationship between a source and a flow (the API is defined on the source as that is where the state is held).

The first thing that jumped out at me was seeing “min_length” rather than “minLength” for strings

The spelling of that field was done with underscores so it would be compatible with the rest of NMOS where multi-word property names are spelled that way. The purpose of the type definition is to communicate the metadata related to the state in a simple and easy readable way and not to provide validation.

@cristian-recoseanu > The more up to date versions are in my pull request
OK, thanks! I knew I had a lot to catch up with. :-)
The transport parameter names have got even longer; what's the reason for the "ext_is_07_" prefix?
@mjeras > The purpose of the type definition is to communicate the metadata related to the state in a simple and easy readable way and not to provide validation.
Neither are the IS-05 constraints about validation so much as setting expectation. It seems a good fit to me, but I may have missed some subtlety.

@garethsb-sony
ext_ is a prefix for externally defined parameters.
This is a conclusion reached after discussions with Andrew and the team behind IS-05 to allow for externally defined transport parameters that can be carried in the same patch request.
ext_ parameters will have their schemas defined in IS-07 as opposed to IS-05.
IS-05 maintains generic/general purpose use of parameters.
The schemas and most of everything else about these will follow the IS-05 way of doing things closely but they will sit within this specification as opposed to IS-05.


"Cannot find module './'" error

$ head-it --help
module.js:327
    throw err;
    ^

Error: Cannot find module './'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/usr/local/lib/node_modules/head-it/cli.js:4:14)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)

Node version?

Sorry, node v4.4.5 @ os x 10.11, npm 2.15.5
Module was installed globally using sudo.

Strange, it works well for me, for the same version and os combinations, but with nvm.
Can you change /usr/local/lib/node_modules/head-it/cli.js to require('./index.js') specifically and do you notice index.js in ``/usr/local/lib/node_modules/head-it/` ?

No, there's no index.js in head-it folder:
$ ls
-rw-r--r--   1 nobody  staff   1,1K 31 май 16:04 license
-rwxr-xr-x   1 nobody  staff   432B 31 май 17:12 cli.js*
-rw-r--r--   1 nobody  staff    13K 31 май 17:16 head.js
-rw-r--r--   1 nobody  staff   1,1K 31 май 17:24 readme.md
-rw-r--r--   1 root    staff   1,6K  8 июн 12:03 package.json
drwxr-xr-x   3 nobody  staff   102B  8 июн 12:03 node_modules/
drwxr-xr-x  17 24561   wheel   578B  8 июн 12:03 ../
drwxr-xr-x   8 nobody  staff   272B  8 июн 12:03 ./

I wonder why there is no index.js for you, the package.json has included all the files npm link on my local worked fine.
//cc @mklabs had you seen something like this before, main in mandatory?

Sorry. Not sire I understand the issue here.
The main field is mandatory only if you want your package to be requirable, but I a agree it seems a bit weird.
Package name@version ?

<EMAIL_ADDRESS>seems like a npm bug?

Looks like a known issue
@Konfuze Can you confirm if it's working in the latest version?

@hemanth yes, it's 1.0.1

@Konfuze No, the latest is 1.0.2

@hemanth There's no 1.0.2 package:
npm install -g<EMAIL_ADDRESS>...
npm ERR! No compatible version found<EMAIL_ADDRESS>npm ERR! Valid install targets:
npm ERR! 1.0.1, 1.0.0

@Konfuze Yikes!
$ npm publish
+<EMAIL_ADDRESS>

Yeah!
$head-it
Expected a string, one of: 360, min, elm, meta, link, fav, fbog, fbia, twitter, g+, oembed, ios, safari, android, chrome, ie, uc, qq, applinks

Phew!


PRE RELEASE - Triage JDK 17 Primary Platform Pipeline

PRE-RELEASE
TRSS link
Build URL https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk17-pipeline/185/
Started by user Martijn Verburg
⚠  refactor_openjdk_release_tool has a build result of undefined ⚠
Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_0
jdk_security_infra_1 => deep history 1/2 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_mac_testList_2
jvm_compiler_1 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_2
jvm_compiler_1 => deep history 0/3 passed | possible issues
jdk_net_0 => deep history 2/3 passed | possible issues
jdk_net_1 => deep history 2/3 passed | possible issues
jdk_nio_1 => deep history 2/3 passed | possible issues
jdk_jmx_0 => deep history 1/3 passed | possible issues
jdk_rmi_0 => deep history 2/3 passed | possible issues
Post Test => deep history 2/3 passed | [possible issues](https://trss.adoptopenjdk.net/possibleIssues?buildId=614033d9c6182d021720b393&buildName=Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_2&testId=61403488c6182d021720b61d&testName=Post Test)
Test_openjdk17_hs_extended.openjdk_x86-64_mac_testList_1
jvm_compiler_0 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_mac
⚠ Test Job Failed ⚠
Test_openjdk17_hs_extended.openjdk_x86-64_windows_testList_2
jvm_compiler_1 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_windows_testList_1
jvm_compiler_0 => deep history 0/3 passed | possible issues
jdk_tools_1 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_1
jvm_compiler_0 => deep history 0/2 passed | possible issues
jdk_tools_1 => deep history 0/2 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_linux
⚠ Test Job Failed ⚠
Test_openjdk17_hs_extended.openjdk_x86-64_mac_testList_0
jdk_instrument_0 => deep history 2/3 passed | possible issues
Test_openjdk17_hs_special.functional_x86-64_linux
Post Test => deep history 2/3 passed | [possible issues](https://trss.adoptopenjdk.net/possibleIssues?buildId=61403304c6182d021720ade9&buildName=Test_openjdk17_hs_special.functional_x86-64_linux&testId=614033b6c6182d021720b1e1&testName=Post Test)
Test_openjdk17_hs_extended.openjdk_x86-64_windows_testList_0
jdk_other_0 => deep history 1/3 passed | possible issues
jdk_tools_0 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_windows
⚠ Test Job Failed ⚠

Linux:
TRSS link
Build URL https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk17-pipeline/185/
Started by user Martijn Verburg
⚠  refactor_openjdk_release_tool has a build result of undefined ⚠
Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_0
jdk_security_infra_1 => deep history 1/2 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_2
jvm_compiler_1 => deep history 0/3 passed | possible issues
jdk_net_0 => deep history 2/3 passed | possible issues
jdk_net_1 => deep history 2/3 passed | possible issues
jdk_nio_1 => deep history 2/3 passed | possible issues
jdk_jmx_0 => deep history 1/3 passed | possible issues
jdk_rmi_0 => deep history 2/3 passed | possible issues
Post Test => deep history 2/3 passed | [possible issues](https://trss.adoptopenjdk.net/possibleIssues?buildId=614033d9c6182d021720b393&buildName=Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_2&testId=61403488c6182d021720b61d&testName=Post Test)
Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_1
jvm_compiler_0 => deep history 0/2 passed | possible issues
jdk_tools_1 => deep history 0/2 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-64_linux
⚠ Test Job Failed ⚠
[Test_openjdk17_hs_special.functional_x86-64_linux]: (https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_special.functional_x86-64_linux/29/) : OK to release : Failure Exception: java.io.IOException: Remote call on test-aws-rhel8-x64-1 at post stage, either the network issue or machine issue
Post Test => deep history 2/3 passed | [possible issues](https://trss.adoptopenjdk.net/possibleIssues?buildId=61403304c6182d021720ade9&buildName=Test_openjdk17_hs_special.functional_x86-64_linux&testId=614033b6c6182d021720b1e1&testName=Post Test)

@karianna - it appears you are pulling TRSS report from an old TRSS instance, is that the case?
This is the release report generated from the public TRSS instance at: https://trss.adoptium.net/ (which has the latest updates to generate the report faster and put it in a more pleasing order)...  Noting the that 'undefined' child jobs are most likely due to the bot not having read access to those jobs...

Release Summary Report for openjdk17-pipeline
TRSS Build and TRSS Grid View
Jenkins Build URL https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk17-pipeline/186/
Started by user Martijn Verburg at 9/13/2021, 3:05:15 PM

create_installer_linux ❌ undefined ❌
create_installer_windows ❌ undefined ❌
refactor_openjdk_release_tool ❌ undefined ❌
sign_build ❌ undefined ❌
sign_installer ❌ undefined ❌

Test_openjdk17_hs_extended.openjdk_aarch64_linux ⚠ UNSTABLE ⚠
jdk_management_0 => deep history 0/3 passed | possible issues
jvm_compiler_0 => deep history 0/3 passed | possible issues
jvm_compiler_1 => deep history 0/3 passed | possible issues
jdk_management_1 => deep history 0/3 passed | possible issues
jdk_tools_0 => deep history 2/3 passed | possible issues
jdk_tools_1 => deep history 2/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_arm_linux ⚠ UNSTABLE ⚠
Test_openjdk17_hs_extended.openjdk_arm_linux_testList_0 ⚠ UNSTABLE ⚠
jvm_compiler_2 => deep history 0/3 passed | possible issues
jdk_other_2 => deep history 2/3 passed | possible issues
jdk_management_2 => deep history 0/3 passed | possible issues
jdk_tools_2 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_arm_linux_testList_1 ⚠ UNSTABLE ⚠
jdk_jfr_2 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_arm_linux_testList_2 ⚠ UNSTABLE ⚠
jdk_nio_2 => deep history 0/3 passed | possible issues
jdk_imageio_2 => deep history 2/3 passed | possible issues
jdk_vector_2 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_ppc64_aix ⚠ UNSTABLE ⚠
Test_openjdk17_hs_extended.openjdk_ppc64_aix_testList_0 ⚠ UNSTABLE ⚠
jdk_nio_0 => deep history 0/1 passed | possible issues
jdk_tools_0 => deep history 0/1 passed | possible issues
Test_openjdk17_hs_extended.openjdk_ppc64_aix_testList_1 ⚠ UNSTABLE ⚠
jvm_compiler_0 => deep history 0/1 passed | possible issues
jdk_beans_0 => deep history 0/1 passed | possible issues
jdk_tools_1 => deep history 0/1 passed | possible issues
Test_openjdk17_hs_extended.openjdk_ppc64_aix_testList_2 ⚠ UNSTABLE ⚠
jdk_net_1 => deep history 0/1 passed | possible issues
jvm_compiler_1 => deep history 0/1 passed | possible issues
jdk_net_0 => deep history 0/1 passed | possible issues
jdk_nio_1 => deep history 0/1 passed | possible issues
Test_openjdk17_hs_extended.openjdk_ppc64le_linux ⚠ UNSTABLE ⚠
Test_openjdk17_hs_extended.openjdk_ppc64le_linux_testList_0 ⚠ UNSTABLE ⚠
jdk_nio_0 => deep history 1/3 passed | possible issues
jdk_management_1 => deep history 1/3 passed | possible issues
jdk_tools_0 => deep history 1/3 passed | possible issues
jdk_jmx_1 => deep history 1/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_ppc64le_linux_testList_1 ⚠ UNSTABLE ⚠
jvm_compiler_0 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_ppc64le_linux_testList_2 ⚠ UNSTABLE ⚠
jvm_compiler_1 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_s390x_linux ⚠ UNSTABLE ⚠
Test_openjdk17_hs_extended.openjdk_s390x_linux_testList_0 ⚠ UNSTABLE ⚠
jdk_other_0 => deep history 0/3 passed | possible issues
jdk_jdi_1 => deep history 2/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_s390x_linux_testList_1 ⚠ UNSTABLE ⚠
jvm_compiler_0 => deep history 0/3 passed | possible issues
jdk_other_1 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_s390x_linux_testList_2 ⚠ UNSTABLE ⚠
jvm_compiler_1 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-32_windows ⚠ UNSTABLE ⚠
Test_openjdk17_hs_extended.openjdk_x86-32_windows_testList_0 ⚠ UNSTABLE ⚠
jvm_compiler_2 => deep history 0/3 passed | possible issues
jdk_tools_2 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-32_windows_testList_1 ⚠ UNSTABLE ⚠
jdk_jdi_2 => deep history 0/3 passed | possible issues
jdk_jfr_2 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_extended.openjdk_x86-32_windows_testList_2 ⚠ UNSTABLE ⚠
runtime_nestmate_2 => deep history 0/3 passed | possible issues
jdk_nio_2 => deep history 0/3 passed | possible issues
Test_openjdk17_hs_sanity.openjdk_ppc64_aix ⚠ UNSTABLE ⚠
jdk_lang_0 => deep history 3/4 passed | possible issues
jdk_lang_1 => deep history 1/4 passed | possible issues
jdk_util_0 => deep history 3/4 passed | possible issues
jdk_util_1 => deep history 3/4 passed | possible issues
Test_openjdk17_hs_sanity.openjdk_x86-32_windows ⚠ UNSTABLE ⚠
jdk_lang_2 => deep history 0/6 passed | possible issues
Test_openjdk17_hs_sanity.system_ppc64_aix ⚠ UNSTABLE ⚠
TestJlmRemoteMemoryNoAuth_1 => deep history 3/4 passed | possible issues

I'm going to close this issue to avoid confusion although it can be used for reference.


Upgrading Tamazight

@BoFFire i'm closing this one, and opening a new one with the latest and greatests:
https://github.com/AnySoftKeyboard/LanguagePack/pull/202

Makes sense?


setup.py issue

I noticed that the reason why it was failing like a death star ;) was due to when running python3 setup.py installthe install_require was not being executed as distutils.core package does not support install_require but its successor setuptools that

Neat! I actually fixed it by editing the travis.yml on dev to manually install pyjwt and requests before installing via pip, but logically that's not helping.
I'd love to merge it, but changes should be made to dev only - otherwise the git network graph is gonna look like steaming pile of poop again.
Could you submit the PR for dev instead? That goes for all your future submissions, btw.
I should probably note that down, somewhere.
Again, a big thanks for the submission!


May CheckContraints be used without setting up Regular Expressions for SQL Server?

I'm very keen to use EFCore.CheckConstraints on SQL Server, yet I wouldn't want to use or require Regular Expression constraints on an on-premise SQL Server installation because that would involve additional setup efforts.
Can I disable regular expression check constraints even when RegularExpressionAttribute is present in entity property?

ping - Anyone?

Currently you can only disable all validation attribute check constraints together. Once https://github.com/dotnet/efcore/issues/214 is implemented you can create a type derived from ValidationCheckConstraintConvention and override SupportsRegex

Ah, very interesting!
In the meantime, I implemented an update to the current version where I added this optional feature, plus full-fledged XML documentation comments. I'm currently using that version for now.
So, I guess, you wouldn't want to pull my PRs for the time being until /dotnet/efcore/issues/214 gets implemented?


VideoCore integration doesn't work as expected

I'm trying to integrate VideoCore with React Native.
I've build a project to demonstrate my issue.
Basically the previewView from VideoCore works once in a while, probably because the way React Native build the views.
Could somebody give me some help? Thanks in advance.

I've found some issues with the help of the community on facebook group. Here is the module I published: https://github.com/GuilhermeMedeiros/react-native-videocore


Provide Jetpack Compose support

After the official Jetpack Compose alpha release a good friend showed us part of the compose testing API we could use to implement screenshot testing on top of Shot. There are a few pull requests we will have to send in order to provide proper support, but it is nothing impossible 💪
We will start with the public ScreenshotTest API, accepting a SemanticsNodeInteraction, then we will modify the android library core to be able to gather the tests execution metadata and save the execution report. Once this is ready, the only task missing is to modify the Shot gradle plugin to be able to fetch this information, compare the screenshots and generate reports.
I'll start working on this as soon as I finish a bug I have pending.

Implemented and released in Shot 5.0.1!


DO Ember install

Something is wrong with peerDependencies on the droplet. :fire: npm :fire:

The real problem is that npm is running out of memory when trying to install all of ember-cli. If I can cut the tree down into chunks (like, install Babel separately), then maybe I can make it fit. It's a hack, but it's cheaper than doubling the memory or an even worse hack of adding a swap.

I had to go with a 1GB swapfile to get it work in Vagrant. I'll confirm tonight if it fixed the issue for the DO box.


GIL

TODO does GIL apply only with multi-threading?

Extend image class in python.

Depends on #6 and #7 (writer)


[openapi-gen][bug][java-client] Nullable field failing when explicitly set to "null"

Recently openapi generator fixed bug in java pojo template : https://github.com/OpenAPITools/openapi-generator/issues/12549
This fix is not available in any stable version hence we have to move to latest image until a new stable version is released.

An alternative would be to patch the OAS3 (3.0) spec to indicate that these fields are nullable: true, although I don't know if the issue referred to in the issue description handles that scenario any better.

So before fix mentioned in https://github.com/OpenAPITools/openapi-generator/issues/12549 following is code pattern that would be generated.
if ((jsonObj.get("description") != null && !jsonObj.get("description").isJsonPrimitive()) {
        throw new IllegalArgumentException(String.format("Expected the field `description` to be a primitive type in the JSON string but got `%s`", jsonObj.get("description").toString()));
      }

Here variable jsonObj is of type gson.JsonObject which returns JsonNull.Instance instead of null when its corresponding parameter is null. Now JsonNull.Instance is not equal to null and if its a primitive type then exception inside if block is thrown.
Correct flavour would be to check if returned value is not JsonNull, which is the produced code after the fix went in :
if ((jsonObj.get("description") != null && !jsonObj.get("description").isJsonNull()) && !jsonObj.get("description").isJsonPrimitive()) {
        throw new IllegalArgumentException(String.format("Expected the field `description` to be a primitive type in the JSON string but got `%s`", jsonObj.get("description").toString()));
      }


Allow connect function to replace current connection

Add replace=CONN to allow a newly created connection to replace an existing connection. Also shorthanded option @replace will allow replacing the current connection.
This feature is not strictly necessary since the same effect can be achieved using remove and move. So there is question of how much value it adds. And because we should try to be minimal, so this may not be a thing we want. In addition, it does not solve #99.

The feature is not very useful as discussed above.


Feature/jackson datatypes

Add Jackson serializers and deserializers for common Interledger and CryptoCondition classes to enable Conditions, Fulfillments, and InterledgerAddresses to be easily used in JSON representations.

Codecov Report

Merging #62 into master will increase coverage by 0.33%.
The diff coverage is 80.98%.

@@             Coverage Diff              @@
##             master      #62      +/-   ##
============================================
+ Coverage     74.28%   74.62%   +0.33%
- Complexity      440      482      +42
============================================
  Files            92      105      +13
  Lines          2691     2833     +142
  Branches        298      304       +6
============================================
+ Hits           1999     2114     +115
- Misses          518      539      +21
- Partials        174      180       +6

Impacted Files
Coverage Δ
Complexity Δ

...g/interledger/quilt/jackson/InterledgerModule.java
100% <100%> (ø)
3 <3> (?)

...edger/quilt/jackson/cryptoconditions/Encoding.java
100% <100%> (ø)
1 <1> (?)

...er/quilt/jackson/InterledgerAddressSerializer.java
100% <100%> (ø)
3 <3> (?)

.../quilt/jackson/InterledgerAddressDeserializer.java
100% <100%> (ø)
3 <3> (?)

...ledger/quilt/jackson/InterledgerAddressModule.java
100% <100%> (ø)
1 <1> (?)

...uilt/jackson/cryptoconditions/SerializerUtils.java
54.54% <54.54%> (ø)
2 <2> (?)

...kson/cryptoconditions/FulfillmentDeserializer.java
69.23% <69.23%> (ø)
4 <4> (?)

...ackson/cryptoconditions/ConditionDeserializer.java
69.23% <69.23%> (ø)
4 <4> (?)

...ckson/cryptoconditions/CryptoConditionsModule.java
77.77% <77.77%> (ø)
1 <1> (?)

...erledger/quilt/jackson/InterledgerSerializers.java
81.81% <81.81%> (ø)
4 <4> (?)

... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9692f9c...0d97fcd. Read the comment docs.

Closing in favor of #63


Added custom spawn and destroy handlers and removed built in pooling support

Codecov Report

Merging #169 into master will increase coverage by 0.2%.
The diff coverage is 0%.

@@           Coverage Diff            @@
##           master    #169     +/-   ##
========================================
+ Coverage    5.99%   6.19%   +0.2%
========================================
  Files          56      51      -5
  Lines       11779   11211    -568
  Branches     1853    1747    -106
========================================
- Hits          706     695     -11
+ Misses      10976   10455    -521
+ Partials       97      61     -36

Impacted Files
Coverage Δ

MLAPI/Data/MLAPIConstants.cs
0% <ø> (ø)
:arrow_up:

...rComponents/Core/InternalMessageHandler.Receive.cs
0% <ø> (ø)
:arrow_up:

MLAPI/MonoBehaviours/Core/NetworkedObject.cs
0% <ø> (ø)
:arrow_up:

...rkingManagerComponents/Core/NetworkSceneManager.cs
0% <0%> (ø)
:arrow_up:

MLAPI/MonoBehaviours/Core/NetworkingManager.cs
0% <0%> (-5.12%)
:arrow_down:

...I/NetworkingManagerComponents/Core/SpawnManager.cs
0% <0%> (ø)
:arrow_up:

...NetworkingManagerComponents/Core/MessageManager.cs
0% <0%> (-2.29%)
:arrow_down:

...ingManagerComponents/Cryptography/EllipticCurve.cs

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9043316...3977cf7. Read the comment docs.


Affter "git clone" the repository, run "demo.ipynb". I received error messages: ModuleNotFoundError: No module named 'SuperGluePretrainedNetwork.models'

Thank you for your marvelous work!
However, after "git clone" the repository, run "demo.ipynb".
I received error messages:

[2023/01/07 13:59:11 hloc INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
'output': 'feats-superpoint-n4096-r1024',
'preprocessing': {'grayscale': True, 'resize_max': 1024}}

ModuleNotFoundError                       Traceback (most recent call last)
Cell In[5], line 1
----> 1 extract_features.main(feature_conf, images, image_list=references, feature_path=features)
2 pairs_from_exhaustive.main(sfm_pairs, image_list=references)
3 match_features.main(matcher_conf, sfm_pairs, features=features, matches=matches)
File lib/python3.8/site-packages/torch/autograd/grad_mode.py:27, in _DecoratorContextManager.call..decorate_context(*args, **kwargs)
24 @functools.wraps(func)
25 def decorate_context(*args, **kwargs):
26     with self.clone():
---> 27         return func(*args, **kwargs)
File GitHub/Hierarchical-Localization/hloc/extract_features.py:245, in main(conf, image_dir, export_dir, as_half, image_list, feature_path, overwrite)
242     return feature_path
244 device = 'cuda' if torch.cuda.is_available() else 'cpu'
--> 245 Model = dynamic_load(extractors, conf['model']['name'])
246 model = Model(conf['model']).eval().to(device)
248 loader = torch.utils.data.DataLoader(
249     dataset, num_workers=1, shuffle=False, pin_memory=True)
File Hierarchical-Localization/hloc/utils/base_model.py:39, in dynamic_load(root, model)
37 def dynamic_load(root, model):
38     module_path = f'{root.name}.{model}'
---> 39     module = import(module_path, fromlist=[''])
40     classes = inspect.getmembers(module, inspect.isclass)
41     # Filter classes defined in the module
File Hierarchical-Localization/hloc/extractors/superpoint.py:8
5 from ..utils.base_model import BaseModel
7 sys.path.append(str(Path(file).parent / '../../third_party'))
----> 8 from SuperGluePretrainedNetwork.models import superpoint  # noqa E402
11 # The original keypoint sampling is incorrect. We patch it here but
12 # we don't fix it upstream to not impact exisiting evaluations.
13 def sample_descriptors_fix_sampling(keypoints, descriptors, s: int = 8):
ModuleNotFoundError: No module named 'SuperGluePretrainedNetwork.models'
The we run incremental Structure-From-Motion and display the reconstructed 3D model.

I have referenced the previously solved issue:
https://github.com/cvg/Hierarchical-Localization/issues/228
Nevertheless, copying files from this repository to the current folder did not work.
I would appreciate any help.

I figured it out and solved it.
Thanks~

I figured it out and solved it. Thanks~

Hello, could you tell me how you solved it?


implement PartialEq for NamedField

This change is needed to allow checking for equality when comparing
NamedFields in tracing-core.

@carllerche


CRASHING ON DISCONNECT

When running disconnect command , it is causing the application to crash with Message from debugger: Terminated due to signal 13

I am also facing the same issue.
Started debugging and found that transport_unref_locked method in transport.c caused this crash. Specifically this line of code D("transport: %s unref (count=%d)\n", t->serial, t->ref_count);
After commenting out that line, it's not crashing anymore. But I am not sure what this method does.


saveurls fails - see error messages

I had no luck creating HTML files from URLs with saveurls - see output below.
C:\saveurls-test>saveurls -i urls.txt
2022/01/07 16:48:03 rename C:\Users\jimsp\AppData\Local\Temp\temp3524864199 failed: create file Browse code samples | Microsoft Docs.html fail: open Browse code samples | Microsoft Docs.html: The filename, directory name, or volume label syntax is incorrect.
2022/01/07 16:48:03 rename C:\Users\jimsp\AppData\Local\Temp\temp2941290506 failed: create file Browse code samples | Microsoft Docs.html fail: open Browse code samples | Microsoft Docs.html: The filename, directory name, or volume label syntax is incorrect.
2022/01/07 16:48:04 rename C:\Users\jimsp\AppData\Local\Temp\temp1233669729 failed: rename C:\Users\jimsp\AppData\Local\Temp\temp1233669729 Turn EML to HTML Web files online and without registration!.1.html: The process cannot access the file because it is being used by another process.
2022/01/07 16:48:05 rename C:\Users\jimsp\AppData\Local\Temp\temp297682893 failed: rename C:\Users\jimsp\AppData\Local\Temp\temp297682893 xplorer² blog: search filter for EPUB ebooks.1.html: The process cannot access the file because it is being used by another process.
2022/01/07 16:48:05 rename C:\Users\jimsp\AppData\Local\Temp\temp3567353523 failed: rename C:\Users\jimsp\AppData\Local\Temp\temp3567353523 xplorer² customer support.2.html: The process cannot access the file because it is being used by another process.
contents of urls.txt:
https://gallery.technet.microsoft.com/convert-emlx-to-pst-f45d2687
https://gallery.technet.microsoft.com/eml-to-html-725c8ad2
https://www.coolutils.com/online/EML-to-HTML
https://www.zabkat.com/blog/epub-ifilter-windows-search.htm
https://www.zabkat.com/x2support.htm

Hi, @jimspoon
saveurls v1.1.0 should be good to go with Windows.

saveurls v.1.1.0 is  working in Windows, thanks very much!


Re-structure analytics pipeline to include a EventHubProcessor

Re-structure analytics pipeline to include a EventHubProcessor that will process, enrich and route incomming messages to correct locations in blob storage and/or another event hub for further analysis.

Still to do: #275 and #276

@krist00fer - should we close this?


MCRZ gate does not alllow to bind parameters

Environment

Qiskit Terra version:  0.19.2
Python version:  3.8
Operating system:  Windows 10/Ubuntu 20.04

What is happening?
If I use the multi-controlled Z gate rotation I cannot use parameter binds to run a circuit. It fails to create the QOBJ.
How can we reproduce the issue?
from qiskit.circuit import QuantumCircuit, Parameter
from qiskit import QuantumCircuit, QuantumRegister, AncillaRegister, ClassicalRegister
from qiskit import Aer
from qiskit import transpile, assemble

states = QuantumRegister(5, 'states')
ancilla = QuantumRegister(1, 'ancilla')
output = ClassicalRegister(1, 'reg_class')

# create the parameter
phi = Parameter('phi')

qc = QuantumCircuit(states,ancilla,output)

# parameterize the rotation
qc.mcrz(phi, states[:-1],states[-1])
qc.measure(ancilla,output)

simulator_backend = Aer.get_backend('aer_simulator')

qc = transpile(qc,simulator_backend)

qobj = assemble(qc,
                        shots=2,
                        parameter_binds = [{phi: 3.14}])

results = simulator_backend.run(qobj).result()

all_counts = results.get_counts()

print(all_counts)

What should happen?
If we ran the code, the exped result would be the counts of the outputs of the measured value. However, we get a message saying it failed to convert from python to C++ and returns a empty string.
ERROR: Failed to load qobj: Unable to cast Python instance to C++ type (compile in debug mode for details)
Any suggestions?
The controlled Z-gste seems to be working fine, maybe inspect it.

Thanks for the report!  It's pretty arcane, but this is actually caused by CUGate rather than the multi-controlled rz - it's to do with misalignments of expectations between how CUGate.params works, and how other parts of Terra expect to be able to mutate those lists.  There's more information in #7326 - we don't have a work-around yet unfortunately, but it is a bug we need to squash.


[SPARK-22000][SQL] Address missing Upcast in JavaTypeInference.deserializerFor

What changes were proposed in this pull request?
Spark expects the type of column and the type of matching field is same when deserializing to Object, but Spark hasn't actually restrict it (at least for Java bean encoder) and some users just do it and experience undefined behavior (in SPARK-22000, Spark throws compilation failure on generated code because it calls .toString() against primitive type.
It doesn't produce error in Scala side because ScalaReflection.deserializerFor properly inject Upcast if necessary. This patch proposes applying same thing to JavaTypeInference.deserializerFor as well.
Credit to @srowen, @maropu, and @cloud-fan since they provided various approaches to solve this.
How was this patch tested?
Added UT which query is slightly modified based on sample code in attachment on JIRA issue.

cc. @maropu @kiszk @srowen since they had discussion on JIRA issue.

Can one of the admins verify this patch?

ok to test

Thanks @maropu I addressed your comments. The code got much longer - in Java world I had to deal with Row directly, as well as I need to add equals() / hashCode() (this is actually optional but best practice would implement along with equals()) / toString(). Hope this is OK.

Can one of the admins verify this patch?

Test build #102584 has started for PR 23854 at commit 443e74f.

Test build #102584 has finished for PR 23854 at commit 443e74f.

This patch fails Java style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102584/
Test FAILed.

First of all, I'm not a catalyst expert so I might be wrong, so please correct me if I'm mistaken.

The issue concerned calling .toString on primitive types, not String.

Why it was happening is IntegerType is matched to String type of field when deserializing to Java Bean. If it was matched to Integer type of field, Integer.valueOf() would be called.
Looks like Spark doesn't just restrict matching type should be same. It handles some implicit type conversion via leveraging Java API, and primitive type was not compatible with String (case it calls .toString()) whereas it can be matched in other way (String.valueOf()).

This change alters behavior a little bit. Now a null is deserialized as "null" not null. I'm not sure we want that, nor was that an issue before; can this be called on null anyway?

Unless I misread javadoc or javadoc is wrong, StaticInvoke will take care of that. This change would not incur calling String.valueOf(null), otherwise passing null should be happening to other types in same if statement as well and it will just throw NumberFormatException for numeric types.
https://github.com/apache/spark/blob/17d0cfcaa4a43fd55b81065d907538a9c1bf569b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala#L199-L220
Btw I guess the case can be added in new UT - we don't need to worry about it after we apply the case in UT. I'll try.

Test build #102598 has started for PR 23854 at commit d869bba.

I agree with your argument about null; it would never have worked here anyway. It feels like there should be an assertion about it to make sure, as if a null reaches here it returns "null" rather than an exception.
I also don't know this part well enough, but primitives are matched to string? That sounds weird. It seems like primitives are already handled in
      case c if !inferExternalType(c).isInstanceOf[ObjectType] => path

It wasn't clear from the JIRA, initially, how this even happened. Now there's a simple test case attached. Could we paste that in as a test here, and see if it even fails without the change? and if so, how.
The test case suggests that a String field is the problem, but it has toString and isn't handled by this part.

Now there's a simple test case attached. Could we paste that in as a test here, and see if it even fails without the change? and if so, how.

The added UT was slightly modified from attached test case: I just modified it based on review comments. Previously added test failed without the change, and the revised test remains the same.

I agree with your argument about null; it would never have worked here anyway. It feels like there should be an assertion about it to make sure, as if a null reaches here it returns "null" rather than an exception.

No I meant null has been handled well and modified code also works well with null. Please find String.valueOf from generated code - it is guarded with checking null. If the value is null, String.valueOf is not called and the result is null. That's what propagateNull does for us.
07:13:10.684 DEBUG org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection: code for initializejavabean(newInstance(class test.org.apache.spark.sql.JavaBeanDeserializationSuite$RecordSpark22000), (setBooleanField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[6, boolean, true], true, false)), (setLongField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[2, bigint, true], true, false)), (setIntField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[1, int, true], true, false)), (setStringField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[5, string, true], true, false)), (setDoubleField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[4, double, true], true, false)), (setShortField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[0, smallint, true], true, false)), (setFloatField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[3, float, true], true, false)), (setTimestampField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[7, timestamp, true], true, false)), (setNullIntField,staticinvoke(class java.lang.String, ObjectType(class java.lang.String), valueOf, input[8, int, true], true, false))):
/* 001 */ public java.lang.Object generate(Object[] references) {
/* 002 */   return new SpecificSafeProjection(references);
/* 003 */ }
/* 004 */
/* 005 */ class SpecificSafeProjection extends org.apache.spark.sql.catalyst.expressions.codegen.BaseProjection {
/* 006 */
/* 007 */   private Object[] references;
/* 008 */   private InternalRow mutableRow;
/* 009 */   private boolean resultIsNull_0;
/* 010 */   private boolean argValue_0;
/* 011 */   private boolean resultIsNull_1;
/* 012 */   private long argValue_1;
/* 013 */   private boolean resultIsNull_2;
/* 014 */   private int argValue_2;
/* 015 */   private boolean resultIsNull_3;
/* 016 */   private boolean resultIsNull_4;
/* 017 */   private double argValue_3;
/* 018 */   private boolean resultIsNull_5;
/* 019 */   private short argValue_4;
/* 020 */   private boolean resultIsNull_6;
/* 021 */   private float argValue_5;
/* 022 */   private boolean resultIsNull_7;
/* 023 */   private long argValue_6;
/* 024 */   private boolean resultIsNull_8;
/* 025 */   private int argValue_7;
/* 026 */   private UTF8String[] mutableStateArray_0 = new UTF8String[1];
/* 027 */
/* 028 */   public SpecificSafeProjection(Object[] references) {
/* 029 */     this.references = references;
/* 030 */     mutableRow = (InternalRow) references[references.length - 1];
/* 031 */
/* 032 */
/* 033 */   }
/* 034 */
/* 035 */   public void initialize(int partitionIndex) {
/* 036 */
/* 037 */   }
/* 038 */
/* 039 */   public java.lang.Object apply(java.lang.Object _i) {
/* 040 */     InternalRow i = (InternalRow) _i;
/* 041 */     final test.org.apache.spark.sql.JavaBeanDeserializationSuite$RecordSpark22000 value_1 = false ?
/* 042 */     null : new test.org.apache.spark.sql.JavaBeanDeserializationSuite$RecordSpark22000();
/* 043 */     test.org.apache.spark.sql.JavaBeanDeserializationSuite$RecordSpark22000 javaBean_0 = value_1;
/* 044 */     if (!false) {
/* 045 */       initializeJavaBean_0_0(i, javaBean_0);
/* 046 */       initializeJavaBean_0_1(i, javaBean_0);
/* 047 */       initializeJavaBean_0_2(i, javaBean_0);
/* 048 */     }
/* 049 */     if (false) {
/* 050 */       mutableRow.setNullAt(0);
/* 051 */     } else {
/* 052 */
/* 053 */       mutableRow.update(0, value_1);
/* 054 */     }
/* 055 */
/* 056 */     return mutableRow;
/* 057 */   }
/* 058 */
/* 059 */
/* 060 */   private void initializeJavaBean_0_2(InternalRow i, test.org.apache.spark.sql.JavaBeanDeserializationSuite$RecordSpark22000 javaBean_0) {
/* 061 */
/* 062 */     resultIsNull_6 = false;
/* 063 */     if (!resultIsNull_6) {
/* 064 */       boolean isNull_15 = i.isNullAt(3);
/* 065 */       float value_15 = isNull_15 ?
/* 066 */       -1.0f : (i.getFloat(3));
/* 067 */       resultIsNull_6 = isNull_15;
/* 068 */       argValue_5 = value_15;
/* 069 */     }
/* 070 */
/* 071 */     boolean isNull_14 = resultIsNull_6;
/* 072 */     java.lang.String value_14 = null;
/* 073 */     if (!resultIsNull_6) {
/* 074 */       value_14 = java.lang.String.valueOf(argValue_5);
/* 075 */     }
/* 076 */     if (!isNull_14) {
/* 077 */       javaBean_0.setFloatField(value_14);
/* 078 */     }
/* 079 */
/* 080 */     resultIsNull_7 = false;
/* 081 */     if (!resultIsNull_7) {
/* 082 */       boolean isNull_17 = i.isNullAt(7);
/* 083 */       long value_17 = isNull_17 ?
/* 084 */       -1L : (i.getLong(7));
/* 085 */       resultIsNull_7 = isNull_17;
/* 086 */       argValue_6 = value_17;
/* 087 */     }
/* 088 */
/* 089 */     boolean isNull_16 = resultIsNull_7;
/* 090 */     java.lang.String value_16 = null;
/* 091 */     if (!resultIsNull_7) {
/* 092 */       value_16 = java.lang.String.valueOf(argValue_6);
/* 093 */     }
/* 094 */     if (!isNull_16) {
/* 095 */       javaBean_0.setTimestampField(value_16);
/* 096 */     }
/* 097 */
/* 098 */     resultIsNull_8 = false;
/* 099 */     if (!resultIsNull_8) {
/* 100 */       boolean isNull_19 = i.isNullAt(8);
/* 101 */       int value_19 = isNull_19 ?
/* 102 */       -1 : (i.getInt(8));
/* 103 */       resultIsNull_8 = isNull_19;
/* 104 */       argValue_7 = value_19;
/* 105 */     }
/* 106 */
/* 107 */     boolean isNull_18 = resultIsNull_8;
/* 108 */     java.lang.String value_18 = null;
/* 109 */     if (!resultIsNull_8) {
/* 110 */       value_18 = java.lang.String.valueOf(argValue_7);
/* 111 */     }
/* 112 */     if (!isNull_18) {
/* 113 */       javaBean_0.setNullIntField(value_18);
/* 114 */     }
/* 115 */
/* 116 */   }
/* 117 */
/* 118 */
/* 119 */   private void initializeJavaBean_0_1(InternalRow i, test.org.apache.spark.sql.JavaBeanDeserializationSuite$RecordSpark22000 javaBean_0) {
/* 120 */
/* 121 */     resultIsNull_3 = false;
/* 122 */     if (!resultIsNull_3) {
/* 123 */       boolean isNull_9 = i.isNullAt(5);
/* 124 */       UTF8String value_9 = isNull_9 ?
/* 125 */       null : (i.getUTF8String(5));
/* 126 */       resultIsNull_3 = isNull_9;
/* 127 */       mutableStateArray_0[0] = value_9;
/* 128 */     }
/* 129 */
/* 130 */     boolean isNull_8 = resultIsNull_3;
/* 131 */     java.lang.String value_8 = null;
/* 132 */     if (!resultIsNull_3) {
/* 133 */       value_8 = java.lang.String.valueOf(mutableStateArray_0[0]);
/* 134 */     }
/* 135 */     if (!isNull_8) {
/* 136 */       javaBean_0.setStringField(value_8);
/* 137 */     }
/* 138 */
/* 139 */     resultIsNull_4 = false;
/* 140 */     if (!resultIsNull_4) {
/* 141 */       boolean isNull_11 = i.isNullAt(4);
/* 142 */       double value_11 = isNull_11 ?
/* 143 */       -1.0 : (i.getDouble(4));
/* 144 */       resultIsNull_4 = isNull_11;
/* 145 */       argValue_3 = value_11;
/* 146 */     }
/* 147 */
/* 148 */     boolean isNull_10 = resultIsNull_4;
/* 149 */     java.lang.String value_10 = null;
/* 150 */     if (!resultIsNull_4) {
/* 151 */       value_10 = java.lang.String.valueOf(argValue_3);
/* 152 */     }
/* 153 */     if (!isNull_10) {
/* 154 */       javaBean_0.setDoubleField(value_10);
/* 155 */     }
/* 156 */
/* 157 */     resultIsNull_5 = false;
/* 158 */     if (!resultIsNull_5) {
/* 159 */       boolean isNull_13 = i.isNullAt(0);
/* 160 */       short value_13 = isNull_13 ?
/* 161 */       (short)-1 : (i.getShort(0));
/* 162 */       resultIsNull_5 = isNull_13;
/* 163 */       argValue_4 = value_13;
/* 164 */     }
/* 165 */
/* 166 */     boolean isNull_12 = resultIsNull_5;
/* 167 */     java.lang.String value_12 = null;
/* 168 */     if (!resultIsNull_5) {
/* 169 */       value_12 = java.lang.String.valueOf(argValue_4);
/* 170 */     }
/* 171 */     if (!isNull_12) {
/* 172 */       javaBean_0.setShortField(value_12);
/* 173 */     }
/* 174 */
/* 175 */   }
/* 176 */
/* 177 */
/* 178 */   private void initializeJavaBean_0_0(InternalRow i, test.org.apache.spark.sql.JavaBeanDeserializationSuite$RecordSpark22000 javaBean_0) {
/* 179 */
/* 180 */     resultIsNull_0 = false;
/* 181 */     if (!resultIsNull_0) {
/* 182 */       boolean isNull_3 = i.isNullAt(6);
/* 183 */       boolean value_3 = isNull_3 ?
/* 184 */       false : (i.getBoolean(6));
/* 185 */       resultIsNull_0 = isNull_3;
/* 186 */       argValue_0 = value_3;
/* 187 */     }
/* 188 */
/* 189 */     boolean isNull_2 = resultIsNull_0;
/* 190 */     java.lang.String value_2 = null;
/* 191 */     if (!resultIsNull_0) {
/* 192 */       value_2 = java.lang.String.valueOf(argValue_0);
/* 193 */     }
/* 194 */     if (!isNull_2) {
/* 195 */       javaBean_0.setBooleanField(value_2);
/* 196 */     }
/* 197 */
/* 198 */     resultIsNull_1 = false;
/* 199 */     if (!resultIsNull_1) {
/* 200 */       boolean isNull_5 = i.isNullAt(2);
/* 201 */       long value_5 = isNull_5 ?
/* 202 */       -1L : (i.getLong(2));
/* 203 */       resultIsNull_1 = isNull_5;
/* 204 */       argValue_1 = value_5;
/* 205 */     }
/* 206 */
/* 207 */     boolean isNull_4 = resultIsNull_1;
/* 208 */     java.lang.String value_4 = null;
/* 209 */     if (!resultIsNull_1) {
/* 210 */       value_4 = java.lang.String.valueOf(argValue_1);
/* 211 */     }
/* 212 */     if (!isNull_4) {
/* 213 */       javaBean_0.setLongField(value_4);
/* 214 */     }
/* 215 */
/* 216 */     resultIsNull_2 = false;
/* 217 */     if (!resultIsNull_2) {
/* 218 */       boolean isNull_7 = i.isNullAt(1);
/* 219 */       int value_7 = isNull_7 ?
/* 220 */       -1 : (i.getInt(1));
/* 221 */       resultIsNull_2 = isNull_7;
/* 222 */       argValue_2 = value_7;
/* 223 */     }
/* 224 */
/* 225 */     boolean isNull_6 = resultIsNull_2;
/* 226 */     java.lang.String value_6 = null;
/* 227 */     if (!resultIsNull_2) {
/* 228 */       value_6 = java.lang.String.valueOf(argValue_2);
/* 229 */     }
/* 230 */     if (!isNull_6) {
/* 231 */       javaBean_0.setIntField(value_6);
/* 232 */     }
/* 233 */
/* 234 */   }
/* 235 */
/* 236 */ }

So it's completely safe to use String.valueOf() for String type, even other types in if statement.

      case c if !inferExternalType(c).isInstanceOf[ObjectType] => path

This would match when the type of field in Java class is primitive. c is not for type of column in Dataset but for Java class (and fields in class, with considering nested maybe?) we are creating Encoder. I guess you've got confusion here.

OK agree about null.
I'm still trying to understand why the change fixes it, though I believe it does. Is it accidental or actually solving the problem?
Before your change, which field has the wrong generated code -- what type is it? the original test case suggests the String field is the problem, not the primitive problem. But then I don't get where the problem is coming from.
Just trying to understand if we're actually understanding why it happens?

Before your change, which field has the wrong generated code -- what type is it?

The field/column was ref: the type from ref column in Dataset was IntegerType and the type of ref field in Java class was String.
To be honest I have no idea this is intended to support implicit type conversions when deserializing, but any types to String would sound convenient, and not much weird. Without knowing history/intention I guess I can't explain more.

@cloud-fan Could you help putting some knowledge around this issue, since you look to be author of Javabean Encoder? Thanks in advance!

Looks like Encoder is not aware of the type of columns in Dataset: then I understand why it doesn't have strict type check on columns and fields in Java bean. Still not sure what's the intention: expect only StringType for String field so safe to call .toString() there, or handle any types for String field so take .toString() to work with various types.

Test build #102598 has finished for PR 23854 at commit d869bba.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
 public final static class RecordSpark22000

btw, why don't we cast internal data into compatible typed data before deserializing them into Bean instead of this weird type handling like int-> string in deserializer?
Probably, it seems we can add cast exprs before collecting data?
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L3360

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102598/
Test PASSed.

That's probably a better design; this at least fixes the current design and bug. WDYT about proceeding anyway?

Yea, sure. +1 for the fix in the current design. cc: @cloud-fan

Sure I think it sounds much better, but as a view of newcomer of Spark SQL it doesn't seem to be trivial.

Probably, it seems we can add cast exprs before collecting data?
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L3360

I guess we also need to handle Dataset.rdd as well, and both path looks like going through CatalystSerde.deserialize to create DeserializeToObject. So I looked through how to deal with types of logical plan and deserializer, but deserializer has type of fields as JavaType (like ObjectType(java.lang.String) for String field) which we need to match and decide against Spark SQL type. Does Spark have some code to match, or even handle cast?

We've already got SQL inferred types for java beans by JavaTypeInference.inferDataType, so I think its ok to just cast internal data into the inferred types somewhere.

The thing is, inferred types are not propagated into deserializer (which is InitializeJavaBean). It only has setters which StaticInvoke is passed for the String field, and it doesn't have information which type is required, only have information for return type of StaticInvoke which is wrapped with ObjectType.
So if we want to leverage inferred SQL DataType to avoid dealing with ObjectType, it may need to be also propagated in StaticInvoke (and also Invoke maybe) which also looks like non-trivial thing (the code change might be small but just not sure this is what we want).

Never mind. I realized encoderFor[T] has schema information. I'll try to apply the new approach.

Test build #102656 has started for PR 23854 at commit c4d611c.

@maropu Thanks for the idea. I applied your suggestion.
@srowen I just rolled back the change of JavaTypeInference, and confirmed new test passed without this.
I couldn't inject Project(Exec) in CatalystSerde.deserialize since there's some case (ExpressionEncoder.resolveAndBind()) which doesn't use returned DeserializeToObject as LogicalPlan - it just leverages deserializer in DeserializeToObject which injected Project(Exec) is not effective.

Test build #102656 has finished for PR 23854 at commit c4d611c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102656/
Test FAILed.

It turns out to be more complicated than expected. I need to apply projection with cast only when deserialization happens, but some of cases I had to play with physical plan which I had to rebuild deserializer.
After the fix, nothing leverages Dataset.deserializer so I've removed it. According to comment, in sample case Spark should do type check and provide error, but as we've seen the error is not from type check so I'm wondering eagerly initializing deserializer does the type check properly.

Test build #102665 has started for PR 23854 at commit 4063a37.

Test build #102665 has finished for PR 23854 at commit 4063a37.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102665/
Test FAILed.

Test build #102679 has started for PR 23854 at commit 661786b.

Now I have another version of the path which is based on @cloud-fan's suggestion.
https://github.com/apache/spark/compare/master...HeartSaVioR:SPARK-22000-approach-cloud-fan
Please also compare with ScalaReflection when reading above change.
Please let me know which approach we prefer: I'll rebase this branch when we would prefer new approach, as well as update title and description on PR. Thanks in advance!

Test build #102679 has finished for PR 23854 at commit 661786b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102679/
Test PASSed.

Test build #102692 has started for PR 23854 at commit 60ca088.

Never mind. I just had hard reset to commit which follows @cloud-fan approach since it's non-intrusive to Dataset. Old approach is stored in https://github.com/apache/spark/compare/master...HeartSaVioR:SPARK-22000-approach-v1

Test build #102692 has finished for PR 23854 at commit 60ca088.

This patch fails Java style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102692/
Test FAILed.

Test build #102707 has started for PR 23854 at commit 9eeb391.

Test build #102707 has finished for PR 23854 at commit 9eeb391.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
 public static final class RecordSpark22000

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102707/
Test PASSed.

Test build #102736 has started for PR 23854 at commit ca124b9.

d4c2060 only changes parameter names from ca124b9 so the build result would be expected to be same.
Btw, while d4c2060 is refactoring on deserializerFor between ScalaReflection and JavaTypeInference, I'm seeing similar needs on serializerFor, as well as walkedTypePath. Would we want to have them refactored similarly? If then I'll work on that in separate JIRA & PR.
Thanks!

Test build #102737 has started for PR 23854 at commit d4c2060.

Test build #102739 has started for PR 23854 at commit 9c040e2.

Test build #102739 has finished for PR 23854 at commit 9c040e2.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds the following public classes (experimental):
 public static final class RecordSpark22000FailToUpcast

Test build #102736 has finished for PR 23854 at commit ca124b9.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  * For example, we build an encoder forcase class Data(a: Int, b: String) and the real type

Test build #102737 has finished for PR 23854 at commit d4c2060.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  * For example, we build an encoder forcase class Data(a: Int, b: String) and the real type

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102739/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102737/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102736/
Test FAILed.

Test build #102751 has started for PR 23854 at commit 4d564ab.

Test build #102751 has finished for PR 23854 at commit 4d564ab.

This patch fails Java style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102751/
Test FAILed.

Java style test may be false alarm: failure is not due to violation but inaccessible of dtd link.
Just a submitted fix for this: #23887

Retest this please.

Test build #102762 has started for PR 23854 at commit 4d564ab.

Test build #102762 has finished for PR 23854 at commit 4d564ab.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102762/
Test FAILed.

Test build #102768 has started for PR 23854 at commit e01bfe6.

Test build #102768 has finished for PR 23854 at commit e01bfe6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102768/
Test PASSed.

Test build #102782 has started for PR 23854 at commit 8cbad26.

Test build #102782 has finished for PR 23854 at commit 8cbad26.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102782/
Test FAILed.

retest this please

Test build #102785 has started for PR 23854 at commit 8cbad26.

Test build #102785 has finished for PR 23854 at commit 8cbad26.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102785/
Test PASSed.

Test build #102796 has started for PR 23854 at commit 24a1b19.

Test build #102796 has finished for PR 23854 at commit 24a1b19.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102796/
Test PASSed.

thanks, merging to master!

Thanks all for reviewing, providing nice approaches, and finally merging!


CB-16699 Bring your own DNS zone - Rename endpoint body field to prep…

…are endpoint for multiple private DNS zones
On azure the only service accepting private endpoints is the postgres server. In the future, however, it might be possible that there will be more, and the customer will be able to specify his own private DNS zone for them.
The request body has currently one field: privateDnsZoneId. This name does not reflect the service for which the customer specifies the private DNS zone.
This commit renames privateDnsZoneId to databasePrivateDnsZoneId, and with that all related fields within the Bring your own DNS zone feature, including DB persistence.
See detailed description in the commit message.

This PR's schema compatibility check is failing, but that is expected, as an endpoint for an unreleased feature is renamed.