Proper way to deploy an update to a Cosmos Db stored procedure

I can't find in the documentation any recommandation on how to do CI /CD with Cosmos Db stored procedures. The documentation mentionnes how to create a stored procedure with the .NET SDK but that's not really something we can do in a CI/CD pipeline (Azure pipelines for instance). What is the recommanded mechanism to deploy modifications to a store procedure.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 239e3b2d-4a09-c6a1-ab36-70b60a18f8b5
Version Independent ID: 10a323f6-5a10-8b38-6339-48ee7a4c54b3
Content: Register and use stored procedures, triggers, and user-defined functions in Azure Cosmos DB SDKs
Content Source: articles/cosmos-db/how-to-use-stored-procedures-triggers-udfs.md
Service: cosmos-db
GitHub Login: @markjbrown
Microsoft Alias: mjbrown

@TechWatching Thank you for your feedback! We will review and provide an update as appropriate.

@TechWatching Thank you for the detailed feedback.
This functionality is currently planned for a future release based on user feedback.
Please upvote and follow the link to get updates when the feature is live.

Thanks for the answer. Do you have hany release date planned ? Because the Azure Cosmos Db Teams answer is from August 2019.

Unfortunately, we do not have a release date to communicate.
Meanwhile did you check the deployment options via Powershell or CLI.
You should be able to easily integrate these with your CI/CD process.

In fact my infrastructure is already managed by another team using Terraform. So my CosmosDb account, the database and containers are already deployed with Terraform. The only parts missing are the indexes and the stored procedures. I am not sure it's a good idea or even possible to mix Terraform and ARM templates scripts in that case, especially as it's not my team which managed the creation of the database and containers.
I have seen that there are some samples to create stored procedures and indexes with  .Net using Azure.Management SDK so a solution would be to create an executable that would be launched in a pipeline but I am afraid it won't be idempotent.

That User Voice needs to be marked as complete. ARM Templates have support for stored procedures, triggers and UDF's. PowerShell will be coming soon followed by CLI. Terraform you would need to ask them to add support for it. This is supported in our Resource Provider so there is nothing blocking them from adding this support.
Thanks.

btw, here is a sample, https://docs.microsoft.com/en-us/azure/cosmos-db/manage-sql-with-resource-manager#create-sproc

Thanks Mark. Are the samples you commited on github cosmos-management-net repo also a good way to deploy stored procedures, triggers and UDF ? I was wondering if using the Azure Management SDK as you did was indempotent ? The fluent API seems nice to use but I need to be sure that I can execute again the same code that creates store procedures, indexes ... and that it will work fine.

These are not ARM templates. These API's are essentially wrappers around our Resource Provider REST API. The API for the generated samples are auto-generated from our swagger spec. The fluent version is a wrapper around that. The interface for the fluent API's is not fully complete and we are going to make changes as these are getting too deep due to the depth and complexity of our RP.
With the generated API, what is important to understand with these is our RP only supports PATCH at the database account only. Database and container level resources must first be read and copied into a new resource object with specific properties updated, before you do call update on the resource.

@TechWatching Thank you for opening an issue on the Terraform providers repo.
We will proceed to close this issue now.
If there are further questions regarding this matter, please comment and we will gladly continue the discussion.

btw, here is a sample, https://docs.microsoft.com/en-us/azure/cosmos-db/manage-sql-with-resource-manager#create-sproc

@markjbrown - thanks for this example - I was able to follow it and deploy a stored procedure with an ARM template. One shortcoming that I'm stuck trying to address is that it seems to require you to put the actual code of your stored procedure into the arm template, e.g.:
{
   "resource": {
      "id": "myStoredProcedure",
       "body": "function () { var context = getContext(); var response = context.getResponse(); response.setBody('Hello, World'); }"
   }
}

Maybe this is obvious to someone who is more familiar with ARM templates and other deployment techniques, but is there a way that I could store the body of the stored procedure in a separate file (where I could get intelligence with VSCode) and still have automatable deployments? I was hoping somehow inject the file contents into the ARM template when it's deployed, but I can't figure out how to do that (if it's possible at all). Having a JSON file with thousands of lines of JavaScript inserted as a string isn't a great developer experience.

Hi @astegmaier, completely agree embedding js into an ARM template is a terrible developer experience. Unfortunately there is no way to put the js for a stored proc into a separate file then refer to it in the main ARM template.
This is an ask I've had with the ARM team since we added support for deploying stored procs in our ARM templates.
What I do today is copy/paste the js into a minimizer then paste that into my ARM template, just to avoid any issues.
If you're using stored procedures to do bulk inserts definitely take a look at using bulk mode in our .NET v3 SDK or if using them for transactions take a look at our transactional batch support also in our .NET v3 SDK.
Thanks.


Not changing priority for pool members

When saving pool members and altering priority, API is not calling the external script to change configuration in LB.
POST /api/pools/save_reals/

closing as there is issue #36 already open.


Overlap NamedEntities more carefully

@maxaalexeeva, would you like to try this out to see if it fixes the problems you were seeing with the locally published master branch?  This lets longer custom named entities overwrite shorter ones.

Thanks @kwalcock !

I tried this out on Habitus and it did fix the problems.  Habitus with processor master failed 11 tests and after this fix they all went away except for the one that specifically tests for the incorrect behavior.  I'm ready to merge unless @maxaalexeeva has objections.  It's probably a good time to republish as well so we don't get used to the bug in the currently published version.

@kwalcock thank you! Sorry didn't get to this yet - had to get a couple other things out the way. If tests in main now pass, it's most likely good, but I can check in a couple of hours.

@maxaalexeeva, waiting is easy enough :-)  Want to be sure it's working the way you expect it to.  It's TestHabitusProcessor behavior of "acronym" that will fail because it is checking for the incorrect behavior.

@kwalcock I have not found any additional issues! Thank you!

@kwalcock let me know when you release processors and I'll adjust habitus main (in addition to changes to autocloser and lazy ner, there are a bunch of tests that are supposed to pass with the new processors and I will want to go through them, see if they actually pass)

@maxaalexeeva, will do.  There are updates to *.md files which means it will take a couple of hours still.


fix parsing for rfc822 dates with American timezones

fix https://github.com/Cumulus/Syndic/issues/63
modified:   lib/syndic_date.ml

Sorry for lines 29-30, it's because there are some tabs there...
It'd be nice if all tabs were removed...

Thanks :+1: !


Add support for multi-domain docker registry

Support multi-domain registry (several dots. i.e. access.host.com).
Add tests for multi-domain registry with and without port.
Bug Url: https://bugzilla.redhat.com/show_bug.cgi?id=1365878

@zeari @moolitayer @simon3z PTAL
@miq-bot add_label providers/containers, bug, darga/no

LGTM :+1:

@zeari yes we want this for darga
@zgalor can you check the relevant code in docker? They're doing the same parsing there and we probably want to stick close to their logic.
@miq-bot add_label darga/yes

@miq-bot remove_label darga/no
LGTM :+1:

spec/models/manageiq/providers/kubernetes/container_manager/refresh_parser_spec.rb

Line 119, Col 121 - Metrics/LineLength - Line is too long. [134/120]

@zgalor Please fix this

@miq-bot assign chessbyte
LGTM :+1: ready for merge/backport

:tada: @zgalor

Darga backport details:
commit d6ecf3a3c8fa97e88418b2635fdfefc2814a8f5b
Author: Oleg Barenboim<EMAIL_ADDRESS>Date:   Tue Aug 16 14:11:33 2016 -0400

    Merge pull request #10464 from zgalor/handle_multi_domain_registry

    Add support for multi-domain docker registry
    (cherry picked from commit 4f3cf7e8a810e0dd315002723c9ada08d68ded20)


cryptsetup luksFormat failed

https://github.com/AMDESE/sev-guest/blob/62317d7de4d79d4ca887b357dddf072082b0b078/scripts/create-luks-qemu-img.sh#L233
[guest vm|ubuntu-22.04] root@bytedance:~/sev/scripts# cryptsetup --debug luksFormat /dev/nbd0p3
# cryptsetup 2.4.3 processing "cryptsetup --debug luksFormat /dev/nbd0p3"
# Running command luksFormat.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.
# Allocating context for crypt device /dev/nbd0p3.
# Trying to open and read device /dev/nbd0p3 with direct-io.
# Initialising device-mapper backend library.

WARNING!
========
This will overwrite data on /dev/nbd0p3 irrevocably.

Are you sure? (Type 'yes' in capital letters): yes
Operation aborted.

# Releasing crypt device /dev/nbd0p3 context.
# Releasing device-mapper backend.
# Unlocking memory.
Command failed with code -1 (wrong or missing parameters).

should use one of the following:
echo -n 'helloworld' | cryptsetup luksFormat /dev/nbd0p3 -d -
cryptsetup  -q luksFormat /dev/nbd0p3 # need type-in password

'Are you sure? (Type 'yes' in capital letters): yes'
You need 'YES' not 'yes' - see the prompt!

'Are you sure? (Type 'yes' in capital letters): yes'
You need 'YES' not 'yes' - see the prompt!

Oh..., thanks.  @dagrh


How import pickle to index?

Hello!
Tell me please how import pickle to index?
I try:

import pickle
from pynndescent import NNDescent
with open('dataset_faces.dat', 'rb') as f:
	all_face_encodings = pickle.load(f)

index = NNDescent(all_face_encodings)

I get error:
AttributeError: 'list' object has no attribute 'shape'

What data type is all_face_encodings? I feel like this should work, but it isn't exactly clear to me whart is getting handed to NNDescent, so it is hard to judge quite what has gone astray...

@lmcinnes  Hello! =)
I use https://github.com/ageitgey/face_recognition
I encodings get for images with faces with this library.
Encoding - a list of 128 real-valued numbers
**Like this:**
    known_face_encoding = [-0.09634063,  0.12095481, -0.00436332, -0.07643753,  0.0080383,
                            0.01902981, -0.07184699, -0.09383309,  0.18518871, -0.09588896,
                            0.23951106,  0.0986533 , -0.22114635, -0.1363683 ,  0.04405268,
                            0.11574756, -0.19899382, -0.09597053, -0.11969153, -0.12277931,
                            0.03416885, -0.00267565,  0.09203379,  0.04713435, -0.12731361,
                           -0.35371891, -0.0503444 , -0.17841317, -0.00310897, -0.09844551,
                           -0.06910533, -0.00503746, -0.18466514, -0.09851682,  0.02903969,
                           -0.02174894,  0.02261871,  0.0032102 ,  0.20312519,  0.02999607,
                           -0.11646006,  0.09432904,  0.02774341,  0.22102901,  0.26725179,
                            0.06896867, -0.00490024, -0.09441824,  0.11115381, -0.22592428,
                            0.06230862,  0.16559327,  0.06232892,  0.03458837,  0.09459756,
                           -0.18777156,  0.00654241,  0.08582542, -0.13578284,  0.0150229 ,
                            0.00670836, -0.08195844, -0.04346499,  0.03347827,  0.20310158,
                            0.09987706, -0.12370517, -0.06683611,  0.12704916, -0.02160804,
                            0.00984683,  0.00766284, -0.18980607, -0.19641446, -0.22800779,
                            0.09010898,  0.39178532,  0.18818057, -0.20875394,  0.03097027,
                           -0.21300618,  0.02532415,  0.07938635,  0.01000703, -0.07719778,
                           -0.12651891, -0.04318593,  0.06219772,  0.09163868,  0.05039065,
                           -0.04922386,  0.21839413, -0.02394437,  0.06173781,  0.0292527 ,
                            0.06160797, -0.15553983, -0.02440624, -0.17509389, -0.0630486 ,
                            0.01428208, -0.03637431,  0.03971229,  0.13983178, -0.23006812,
                            0.04999552,  0.0108454 , -0.03970895,  0.02501768,  0.08157793,
                           -0.03224047, -0.04502571,  0.0556995 , -0.24374914,  0.25514284,
                            0.24795187,  0.04060191,  0.17597422,  0.07966681,  0.01920104,
                           -0.01194376, -0.02300822, -0.17204897, -0.0596558 ,  0.05307484,
0.07417042, 0.07126575, 0.00209804]

**I put this in pickle file with code:**

import face_recognition
import pickle
all_face_encodings = {}
img1 = face_recognition.load_image_file("obama.jpg")
all_face_encodings["obama"] = face_recognition.face_encodings(img1)[0]
img2 = face_recognition.load_image_file("biden.jpg")
all_face_encodings["biden"] = face_recognition.face_encodings(img2)[0]
... etc ...
with open('dataset_faces.dat', 'wb') as f:
pickle.dump(all_face_encodings, f)
And then i want to import this encodings to index for pynndescent.
And search a pynndescent index for the 5 nearest neighbors of one query image.
Do you understand me,friend? Sorry for my English..

I think I understand. You should not be getting the error you are getting,
but it is entirely unclear to me where the error actually is. Is it
possible for you to post the full traceback rather than just the immediate
error? It would be helpful to know on which lines of code it is failing.
On Thu, Dec 6, 2018 at 12:36 PM psyapathy<EMAIL_ADDRESS>wrote:

@lmcinnes https://github.com/lmcinnes Hello! =)
I use https://github.com/ageitgey/face_recognition
I encodings get for images with faces with this library.
Encoding - a list of 128 real-valued numbers
Like this:
known_face_encoding = [-0.09634063,  0.12095481, -0.00436332, -0.07643753,  0.0080383,
0.01902981, -0.07184699, -0.09383309,  0.18518871, -0.09588896,
0.23951106,  0.0986533 , -0.22114635, -0.1363683 ,  0.04405268,
0.11574756, -0.19899382, -0.09597053, -0.11969153, -0.12277931,
0.03416885, -0.00267565,  0.09203379,  0.04713435, -0.12731361,
-0.35371891, -0.0503444 , -0.17841317, -0.00310897, -0.09844551,
-0.06910533, -0.00503746, -0.18466514, -0.09851682,  0.02903969,
-0.02174894,  0.02261871,  0.0032102 ,  0.20312519,  0.02999607,
-0.11646006,  0.09432904,  0.02774341,  0.22102901,  0.26725179,
0.06896867, -0.00490024, -0.09441824,  0.11115381, -0.22592428,
0.06230862,  0.16559327,  0.06232892,  0.03458837,  0.09459756,
-0.18777156,  0.00654241,  0.08582542, -0.13578284,  0.0150229 ,
0.00670836, -0.08195844, -0.04346499,  0.03347827,  0.20310158,
0.09987706, -0.12370517, -0.06683611,  0.12704916, -0.02160804,
0.00984683,  0.00766284, -0.18980607, -0.19641446, -0.22800779,
0.09010898,  0.39178532,  0.18818057, -0.20875394,  0.03097027,
-0.21300618,  0.02532415,  0.07938635,  0.01000703, -0.07719778,
-0.12651891, -0.04318593,  0.06219772,  0.09163868,  0.05039065,
-0.04922386,  0.21839413, -0.02394437,  0.06173781,  0.0292527 ,
0.06160797, -0.15553983, -0.02440624, -0.17509389, -0.0630486 ,
0.01428208, -0.03637431,  0.03971229,  0.13983178, -0.23006812,
0.04999552,  0.0108454 , -0.03970895,  0.02501768,  0.08157793,
-0.03224047, -0.04502571,  0.0556995 , -0.24374914,  0.25514284,
0.24795187,  0.04060191,  0.17597422,  0.07966681,  0.01920104,
-0.01194376, -0.02300822, -0.17204897, -0.0596558 ,  0.05307484,
0.07417042, 0.07126575, 0.00209804]
I put this in pickle file with code:
import face_recognition
import pickle
all_face_encodings = {}
img1 = face_recognition.load_image_file("obama.jpg")
all_face_encodings["obama"] = face_recognition.face_encodings(img1)[0]
img2 = face_recognition.load_image_file("biden.jpg")
all_face_encodings["biden"] = face_recognition.face_encodings(img2)[0]
... etc ...
with open('dataset_faces.dat', 'wb') as f:
pickle.dump(all_face_encodings, f)
And then i want to import this encodings to index for pynndescent.
And search a pynndescent index for the 5 nearest neighbors of one query
image.
Do you understand me,friend? Sorry for my English..
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lmcinnes/pynndescent/issues/8#issuecomment-444960468,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALaKBaQSKPGNE2evBQas4ZKYsoNqBrqUks5u2VWbgaJpZM4ZBjgG
.


pcre2: Update to upstream version 10.39

Changelog: https://github.com/PhilipHazel/pcre2/blob/pcre2-10.39/ChangeLog

Cherry-picked for 3.5.


After unlock - No connection available

Very helpful walkthrough. Thanks.
Created passwords and seeds, all smooth. Upon attempting to create a new address, or even getinfo, I am getting this error.

[lncli] rpc error: code = Unavailable desc = grpc: the connection is unavailable

I checked the lnd log:

2018-03-25 00:00:33.154 [INF] RPCS: password RPC server listening on <IP_ADDRESS>:10009
2018-03-25 00:00:33.154 [INF] LTND: Waiting for wallet encryption password. Use lncli create to create wallet, or lncli unlock to unlock already created wallet.
2018-03-25 00:07:13.458 [INF] LNWL: Opened wallet
2018-03-25 00:07:13.921 [ERR] SRVR: invalid password
2018-03-25 00:08:14.567 [INF] LTND: Version 0.4.0-alpha
2018-03-25 00:08:14.567 [INF] LTND: Active chain: Bitcoin (network=testnet)
2018-03-25 00:08:14.591 [INF] CHDB: Checking for schema update: latest_version=0, db_version=0
2018-03-25 00:08:14.618 [INF] RPCS: password RPC server listening on <IP_ADDRESS>:10009
2018-03-25 00:08:14.619 [INF] RPCS: password gRPC proxy started at <IP_ADDRESS>:8080
2018-03-25 00:08:14.619 [INF] LTND: Waiting for wallet encryption password. Use lncli create to create wallet, or lncli unlock to unlock already created wallet.

I can go on an unlocking, successfully, and then attempting to create a new address, but this cycle keeps happening.
Help would be much appreciated.

I am getting the same thing.

The log says invalid password, any chance of a mixup or typo?
If nothing helps (and you do not have any funds commited yet), I would recommend deleting the /mnt/hdd/lnd directory, recreating it with the config file, let sync lnd directly on mainnet and creating a brand-new wallet.
The reason for a complete delete is that quite a few files are encrypted using this password (eg. *.macaroon, data/macaroons.db and of course the wallet.db in the chains subdirectories). You can try to delete these individually, but I would just start anew.

Wiped the slate clean and started from the beginning, (had done this prior to original post also.)
I can unlock the wallet. No problem. lnd.log:

Mar 25 16:05:51 raspberrypi lnd[793]: 2018-03-25 16:05:51.929 [INF] LNWL: Opened wallet
Mar 25 16:05:52 raspberrypi lnd[793]: 2018-03-25 16:05:52.335 [INF] LTND: Primary chain is set to: bitcoin
Mar 25 16:05:52 raspberrypi lnd[793]: 2018-03-25 16:05:52.335 [INF] LTND: Initializing bitcoind backed fee estimator
Mar 25 16:05:58 raspberrypi lnd[793]: 2018-03-25 16:05:58.598 [INF] LNWL: Opened wallet
Mar 25 16:05:58 raspberrypi systemd[1]: lnd.service: Main process exited, code=exited, status=1/FAILURE
Mar 25 16:05:58 raspberrypi systemd[1]: lnd.service: Unit entered failed state.
Mar 25 16:05:58 raspberrypi systemd[1]: lnd.service: Failed with result 'exit-code'.

sooo... checked my lnd.service file. The initial issue I see is the reference to the lnd.pid file, but there is no corresponding file in that directory.
Why or how? No idea, hopefully someone can help.
Thanks

Ok... fresh install. Same problem. Came to realize that I cannot, at least with this setup, simultaneously run bitcoind mainnet and testnet, which is where I believe the problem was. I was just assuming testnet3 was running as planned, but it was giving me an error about bitcoin.conf, and I didn't bother to listen.
So changed bitcoin.conf back to testnet / reboot / boots into testnet / lncli unlock / now catching up...
I assume this has resolved the issue, though still catching up.


How can we get the car-object-latents and car-nerf-state-dict at the first place?

Hi MARS team, the project is a great work in the NERF simulator area!
I noticed in issue #5 that you provided the car-object-latents and car-nerf-state-dict for some scenes in KITTI-MOT and vKITTI.
My question is that how we can get the car-object-latents and car-nerf-state-dict from scratch, like if we want to use your default car-nerf foreground model on our own dataset?
Thanks!

Hi, the CarNeRF is actually another recent work of our lab (URL), please refer to their project page for more information.

Hi, the CarNeRF is actually another recent work of our lab (URL), please refer to their project page for more information.

Thank you!

Hey guys, as a kind reminder, the CarStudio paper is accepted by RAL and their code is available now at https://github.com/lty2226262/Car_studio


StackOverflow calling very large generated function

Repro steps

Build enclosed project: [StackOverflowExample.zip](https://github.com/fsharp/fsharp/files/2627592/StackOverflowExample.zip .

Run it. It crashes with stack overflow exception between starting call to update and actually getting into it (first printfn in it).

Expected behavior
It should not crash with stack overflow exception.
Actual behavior
StackOverflowException
Known workarounds
None.
Related information

Windows
Visual Studio Community 2017
ModelData.fs is a generated file with approximately 170K LOC.
This could be related to this ticket: https://github.com/fsharp/fsharp/issues/882

You might get a better response using the official repo: https://github.com/Microsoft/visualfsharp

Or maybe this is a duplicate of this mono bug? https://github.com/mono/mono/issues/11933

Closing out issues in this repo, if necessary please reopen at https://github.com/dotnet/fsharp, which is now the unified repository for F# work


Release Definition Visual Studio Test task v2.* does not find test assemblies

Description
When trying to run Visual Studio Test task v. 2.* within some Release Definition (after the deployment task) it fails  - test assemblies not found.

2018-03-11T13:45:58.9299389Z ##[section]Starting: VsTest - testAssemblies
2018-03-11T13:45:58.9308588Z ==============================================================================
2018-03-11T13:45:58.9308898Z Task         : Visual Studio Test
2018-03-11T13:45:58.9309304Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test runner. Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run. Tests can be distributed on multiple agents using this task (version 2).
2018-03-11T13:45:58.9309987Z Version      : 2.3.22
2018-03-11T13:45:58.9310218Z Author       : Microsoft Corporation
2018-03-11T13:45:58.9310499Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-03-11T13:45:58.9310806Z ==============================================================================
2018-03-11T13:46:00.2276627Z Run the tests locally using vstest.console.exe
2018-03-11T13:46:00.2277087Z ========================================================
2018-03-11T13:46:00.2277775Z Test selector : Test assemblies
2018-03-11T13:46:00.2278155Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2018-03-11T13:46:00.2278744Z Test filter criteria : TestCategory=GUITests
2018-03-11T13:46:00.2279050Z Search folder : D:\a\r1\a
2018-03-11T13:46:00.2279394Z Run settings file : D:\a\r1\a
2018-03-11T13:46:00.2279964Z Run in parallel : false
2018-03-11T13:46:00.2280285Z Run in isolation : false
2018-03-11T13:46:00.2281981Z Path to custom adapters : null
2018-03-11T13:46:00.2282309Z Other console options : null
2018-03-11T13:46:00.2282694Z Code coverage enabled : false
2018-03-11T13:46:00.2283486Z Rerun failed tests: false
2018-03-11T13:46:00.2283876Z VisualStudio version selected for test execution : 14.0
2018-03-11T13:46:00.3538282Z ========================================================
2018-03-11T13:46:00.3606879Z ##[warning]No test assemblies found matching the pattern: **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**.
2018-03-11T13:46:00.4897315Z ##[section]Finishing: VsTest - testAssemblies

The same task in a Build Definition runs just fine, thus tried to add a path to custom test assemblies from that successful Build Definition task (F:\VSTSBuild\workspace\8\s\VS$PROJECTNAME\bin\Release) but it does not help:
2018-03-11T11:24:14.1592030Z ##[section]Starting: VsTest - testAssemblies
2018-03-11T11:24:14.1599679Z ==============================================================================
2018-03-11T11:24:14.1600003Z Task         : Visual Studio Test
2018-03-11T11:24:14.1600437Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test runner. Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run. Tests can be distributed on multiple agents using this task (version 2).
2018-03-11T11:24:14.1600997Z Version      : 2.3.22
2018-03-11T11:24:14.1601205Z Author       : Microsoft Corporation
2018-03-11T11:24:14.1601467Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-03-11T11:24:14.1601755Z ==============================================================================
2018-03-11T11:24:14.8054390Z Run the tests locally using vstest.console.exe
2018-03-11T11:24:14.8054796Z ========================================================
2018-03-11T11:24:14.8055468Z Test selector : Test assemblies
2018-03-11T11:24:14.8055818Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2018-03-11T11:24:14.8056150Z Test filter criteria : TestCategory=GUITests
2018-03-11T11:24:14.8056455Z Search folder : D:\a\r1\a
2018-03-11T11:24:14.8056858Z Run settings file : D:\a\r1\a
2018-03-11T11:24:14.8058062Z Run in parallel : false
2018-03-11T11:24:14.8058411Z Run in isolation : false
2018-03-11T11:24:15.0560015Z ##[error]Error: Path to custom adapters '**F:\VSTSBuild\workspace\8\s\VS\$PROJECTNAME\bin\Release**' should be a directory and it should exist.
2018-03-11T11:24:15.1786711Z ##[section]Finishing: VsTest - testAssemblies

The 'F:\VSTSBuild\workspace\8\s\VS$PROJECTNAME\bin\Release' path taken from the logs of the Build Definition test task which runs properly.
It looks like "D:\a\r1\a" is the root directory and the path to the custom assembles is calculated related to the root directory - because 'F:\VSTSBuild\workspace\8\s\VS$PROJECTNAME\bin\Release' does exist on file system.
Is that possible to set the root directory in some way? Because considering even drive letters are different I cannot calculate the relative path properly.
Steps to reproduce
Create Visual Studio Test task v. 2.* within some Build Definition Agent Phase with criteria (TestCategory!=GUITests) and default assemblies names ($(BuildConfiguration)*test*.dll;-:\obj**)
Create Visual Studio Test task v. 2.* within some Release Definition Agent Phase with criteria
(TestCategory=GUITests) and default values for test assemblies (*test.dl, !*TestAdapter.dll, !\obj**)**
Make Build Definition run Release Definition
Run Build Definition
Expected behavior
Build Definition tests run
Release definition test run
Actual behavior
Build Definition tests run
Release definition test fails while not finding the test assemblies
Environment
VSTS 2017
Release Definition Visual Studio Test task v. 2.*
Account, project and definition names can be provided upon request

@Koshak118 Can you run your release with release variable "system.debug=true" and provide us the logs at devops_tools[at]microsoft[dot]com. Thanks.

Hi @Koshak118,
Couple of things:

Path to custom adapters is not the same as path to test assemblies. This input field is something else. It is used to provide test adapters required to run tests of different frameworks.

You will have to change the Search folder input to point explicitly point to the test dlls.

Also the way you are going about it seems wrong. Your build definition is supposed to upload the test dlls as artifacts. And your release definition is supposed to download them into the artifacts folder and then proceed ahead. If you are not uploading the test dlls in the build definition then I suggest you run the tests in the build definition itself.

I can take a look at your build and release definitions if you want to know the right way to go about it. You can attach screenshots here. It would also help if you can tell me what exactly is your requirement with the build definition and release definition and where you want to run your tests and why.

Hey Shreys
Thanks for the answer
So it seems there is a fundamental issue with how I think things should
work
Can we do an interactive session to solve the problem, like Skype session
with screen sharing?
On Mar 19, 2018 8:34 AM, "Shreyas R"<EMAIL_ADDRESS>wrote:

Hi @Koshak118 https://github.com/koshak118,
Couple of things:

Path to custom adapters is not the same as path to test assemblies.
This input field is something else. It is used to provide test adapters
required to run tests of different frameworks.
You will have to change the Search folder input to point explicitly
point to the test dlls.
Also the way you are going about it seems wrong. Your build definition
is supposed to upload the test dlls as artifacts. And your release
definition is supposed to download them into the artifacts folder and then
proceed ahead. If you are not uploading the test dlls in the build
definition then I suggest you run the tests in the build definition itself.
I can take a look at your build and release definitions if you want to
know the right way to go about it. You can attach screenshots here. It
would also help if you can tell me what exactly is your requirement with
the build definition and release definition and where you want to run your
tests and why.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vsts-tasks/issues/6680#issuecomment-374114128,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGdW7G4AMXckcsFqlCbqkisB-bi_PQZAks5tf1FhgaJpZM4SpF9D
.

@Koshak118 we can. Can you share your skype handle with me? Either here or to<EMAIL_ADDRESS>I will initiate a call.

Can we schedule it in 4 hours from now please?
On Mar 19, 2018 12:30 PM, "Shreyas R"<EMAIL_ADDRESS>wrote:

@Koshak118 https://github.com/koshak118 we can. Can you share your
skype handle with me? Either here or to<EMAIL_ADDRESS>I will initiate a call.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vsts-tasks/issues/6680#issuecomment-374166837,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGdW7KRUijLv3PNb6UCHqhfhesVCfJipks5tf4jagaJpZM4SpF9D
.

@Koshak118 That is outside my working hours (I'm available from 11am to 5pm IST).
We can correspond through mails also. Screen shots should help. Or you can give me<EMAIL_ADDRESS>permissions to a clone of your definition and I can take a look and give suggestions.

OK, so I'll ping you when I get back to the office and we'll see how we
proceed
Thanks for the fast responses meanwhile
On Mar 19, 2018 12:55 PM, "Shreyas R"<EMAIL_ADDRESS>wrote:

@Koshak118 https://github.com/koshak118 That is outside my working
hours (I'm available from 11am to 5pm IST).
We can correspond through mails also. Screen shots should help. Or you can
give me<EMAIL_ADDRESS>permissions to a clone of your definition
and I can take a look and give suggestions.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vsts-tasks/issues/6680#issuecomment-374173126,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGdW7CiIhBLStm40RmedCkzzyjFHu_00ks5tf46RgaJpZM4SpF9D
.

Well, I tried to add you as a project contributor but with no success
Anyway, here are the 2 screenshots: the first of the working build definition, which was pretty much straightforward

and the second is for the broken release definition (after removing custom paths)

They looks pretty much the same for a reason  - I'd like to run the same assemblies but with different configuration - (TestCategory!=GUITests) for the first and (TestCategory=GUITests) for the latter, which are literally Selenium GUI tests which should run after the Azure deployment task

@Koshak118 You'll need to add me to the AAD before adding me to the vsts account.

Yeah so you're going about it the wrong way. Follow how to create a build definition from here and how to create a linked release definition from here

Your build definition should upload the built binaries and your release definition is meant to download those uploaded binaries via artifacts.

If you are not doing this then there is no point in having a separate release definition as you clearly expect the steps in the release definition to run on the same machine where the build ran since you are expecting the test dlls to be present on the machine. You might as well have the steps you are performing in the release definition in the build definition itself.

But if you are bent on having both then just use the search folder input to hardcode the path to the built test dlls. But this is clearly not the recommended way to go about this and it will most definitely not scale.

Ok, I'll take a look on it shortly. The motivation of doing it as I did is to run some integration tests locally, then upload the binaries to some some test Azure app and then run the some Selenium tests - from the same assemblies

Yes then you will need to run the integration tests in your build definition and then publish these binaries as build artifacts and link your release definition to your build and consume these artifacts there.

Speaking of Search Folder - that what I was meant by "changing the root path" . Didin't find it in "Variables" section of the definition though, where it is hiding?

Regarding the linking build and release definitions - so yeah, there is a "After Release" trigger which starts the release definition which defines "Artifacts filter" to be artifacts from that specific build definition which doing its testing job well
Should I define something else?

you should add a task to upload those artifacts. just check the link on creating a build definition or use the default template for building .Net apps. ideally you shouldn't have to change the search folder input. You can just type in whatever value you want. It doesn't have to be a build variable. Please take a look at some sample templates. Also please take a look at other blog posts regarding the vstest task usage as well as for creating proper build definitions and release definitions

@Koshak118 once you add the task to upload the build artifacts and get your release to download them just set the Search folder to $(System.ArtifactsDirectory) and things should work.

Well so the build definition does have the "publish artifacts" task at the last one which looks like that:

So there are 2 question right now:

How to make the release definition to download those artifacts or the v2.* does it automatically and nothing should be done?
Where is that "Search Folder" variable which should be set?

@Koshak118 Search Folder is an input. Please take a look at the task and read the input descriptions and the blogs. Everything is clearly documented there.
How to make the release definition to download those artifacts or the v2.* does it automatically: This is also documented please search for it online. When you create a release definition if you can read the on screen messages and take a look at the UI you will find out all this on your own.
I am closing this issue now. The task is intuitive enough to use, please report back if you get any actual issues.

Guys, if things were so straightforward and intuitive I wouldn't open this issue.
So right, "Search Folder" is an input field of the task, but when I set it to "F:\VSTSBuild\workspace\8\s\VS\Tests\bin\Release" things end up with error
"##[error]Error: Search folder: 'F:\VSTSBuild\workspace\8\s\VS\Tests\bin\Release' should be a directory and it should exist."  - despite the fact that it is an existing folder
So I set the "Search Field" to " $(System.ArtifactsDirectory)" , added the task of unzipping the artifacts after downloading of them and that what I am getting
Artifacts downloaded:
2018-03-21T11:24:50.8817225Z ##[section]Starting: Download artifact - Rule Them All
2018-03-21T11:24:50.9016987Z ==============================================================================
2018-03-21T11:24:50.9017278Z Task         : Download Build Artifacts
2018-03-21T11:24:50.9017506Z Description  : Download Build Artifacts
2018-03-21T11:24:50.9017705Z Version      : 0.131.1
2018-03-21T11:24:50.9017920Z Author       : Microsoft Corporation
2018-03-21T11:24:50.9018426Z Help         :
2018-03-21T11:24:50.9018697Z ==============================================================================
2018-03-21T11:24:51.9973046Z Linked artifacts count:  1
2018-03-21T11:24:51.9982405Z Downloading artifacts from: https://say2eat.visualstudio.com//_apis/resources/Containers/1335576?itemPath=drop&isShallow=true&api-version=4.1-preview.4
2018-03-21T11:24:52.2623781Z Downloading drop/Say2eat.API.deploy-readme.txt to D:\a\r1\a\Rule Them All\drop\Say2eat.API.deploy-readme.txt
2018-03-21T11:24:52.2659074Z Downloaded drop/Say2eat.API.deploy-readme.txt to D:\a\r1\a\Rule Them All\drop\Say2eat.API.deploy-readme.txt
2018-03-21T11:24:52.3186680Z Downloading drop/Say2eat.API.deploy.cmd to D:\a\r1\a\Rule Them All\drop\Say2eat.API.deploy.cmd
2018-03-21T11:24:52.3198933Z Downloaded drop/Say2eat.API.deploy.cmd to D:\a\r1\a\Rule Them All\drop\Say2eat.API.deploy.cmd
2018-03-21T11:24:52.3907509Z Downloading drop/Say2eat.API.SetParameters.xml to D:\a\r1\a\Rule Them All\drop\Say2eat.API.SetParameters.xml
2018-03-21T11:24:52.3926635Z Downloaded drop/Say2eat.API.SetParameters.xml to D:\a\r1\a\Rule Them All\drop\Say2eat.API.SetParameters.xml
2018-03-21T11:24:52.4584799Z Downloading drop/Say2eat.API.SourceManifest.xml to D:\a\r1\a\Rule Them All\drop\Say2eat.API.SourceManifest.xml
2018-03-21T11:24:52.4585209Z Downloaded drop/Say2eat.API.SourceManifest.xml to D:\a\r1\a\Rule Them All\drop\Say2eat.API.SourceManifest.xml
2018-03-21T11:24:52.6287773Z Downloading drop/Say2eat.API.zip to D:\a\r1\a\Rule Them All\drop\Say2eat.API.zip
2018-03-21T11:24:53.7194635Z Downloading drop/Say2eat.BotFramework.SourceManifest.xml to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.SourceManifest.xml
2018-03-21T11:24:53.7198522Z Downloaded drop/Say2eat.BotFramework.SourceManifest.xml to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.SourceManifest.xml
2018-03-21T11:24:53.7199943Z Downloading drop/Say2eat.BotFramework.SetParameters.xml to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.SetParameters.xml
2018-03-21T11:24:53.7200630Z Downloaded drop/Say2eat.BotFramework.SetParameters.xml to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.SetParameters.xml
2018-03-21T11:24:53.7201858Z Downloading drop/Say2eat.BotFramework.deploy.cmd to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.deploy.cmd
2018-03-21T11:24:53.7203279Z Downloaded drop/Say2eat.BotFramework.deploy.cmd to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.deploy.cmd
2018-03-21T11:24:53.7208179Z Downloading drop/Say2eat.BotFramework.deploy-readme.txt to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.deploy-readme.txt
2018-03-21T11:24:53.7211257Z Downloaded drop/Say2eat.BotFramework.deploy-readme.txt to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.deploy-readme.txt
2018-03-21T11:24:53.7214534Z Downloading drop/Say2eat.BotFramework.zip to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.zip
2018-03-21T11:24:53.7214942Z Downloaded drop/Say2eat.API.zip to D:\a\r1\a\Rule Them All\drop\Say2eat.API.zip
2018-03-21T11:24:53.7221062Z Downloaded drop/Say2eat.BotFramework.zip to D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.zip
2018-03-21T11:24:54.4104278Z Total Files: 10, Processed: 10, Skipped: 0, Failed: 0, Download time: 2.41 secs, Download size: 26.456MB
2018-03-21T11:24:54.5625900Z Successfully downloaded artifacts to D:\a\r1\a/Rule Them All
2018-03-21T11:24:54.5653087Z ##[section]Finishing: Download artifact - Rule Them All

Artifacts unzipped (by another task)
2018-03-21T11:29:04.2066782Z ##[section]Starting: Unzip artifacts
2018-03-21T11:29:04.2069292Z ==============================================================================
2018-03-21T11:29:04.2069556Z Task         : Unzip file
2018-03-21T11:29:04.2069793Z Description  : extracts files from a zip, a pattern can be applied to extract only parts of the file.
2018-03-21T11:29:04.2070039Z Version      : 0.9.5
2018-03-21T11:29:04.2070223Z Author       : Yod Labs
2018-03-21T11:29:04.2070420Z Help         :
2018-03-21T11:29:04.2070749Z ==============================================================================
2018-03-21T11:29:04.4239792Z (node:2888) fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version.
2018-03-21T11:29:04.4398671Z unzipping D:\a\r1\a\Rule Them All\drop\Say2eat.BotFramework.zip using pattern  to folder D:\a\r1\a
2018-03-21T11:29:05.0995516Z ##[section]Finishing: Unzip artifacts

No assemblies found:

2018-03-21T11:29:05.1033908Z ##[section]Starting: VsTest - testAssemblies
2018-03-21T11:29:05.1039447Z ==============================================================================
2018-03-21T11:29:05.1039720Z Task         : Visual Studio Test
2018-03-21T11:29:05.1040171Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test runner. Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run. Tests can be distributed on multiple agents using this task (version 2).
2018-03-21T11:29:05.1040569Z Version      : 2.3.27
2018-03-21T11:29:05.1040795Z Author       : Microsoft Corporation
2018-03-21T11:29:05.1041238Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-03-21T11:29:05.1041532Z ==============================================================================
2018-03-21T11:29:05.7511862Z Run the tests locally using vstest.console.exe
2018-03-21T11:29:05.7512177Z ========================================================
2018-03-21T11:29:05.7512642Z Test selector : Test assemblies
2018-03-21T11:29:05.7513006Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2018-03-21T11:29:05.7513358Z Test filter criteria : TestCategory=GUITests
2018-03-21T11:29:05.7513692Z Search folder : D:\a\r1\a
2018-03-21T11:29:05.7514019Z Run settings file : D:\a\r1\a
2018-03-21T11:29:05.7514444Z Run in parallel : false
2018-03-21T11:29:05.7514759Z Run in isolation : false
2018-03-21T11:29:05.7516401Z Path to custom adapters : null
2018-03-21T11:29:05.7516725Z Other console options : null
2018-03-21T11:29:05.7517302Z Code coverage enabled : false
2018-03-21T11:29:05.7517987Z Rerun failed tests: false
2018-03-21T11:29:05.7518424Z VisualStudio version selected for test execution : 14.0
2018-03-21T11:29:06.2141783Z ========================================================
2018-03-21T11:29:06.2243879Z ##[warning]No test assemblies found matching the pattern: **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**.
2018-03-21T11:29:06.3682176Z ##[section]Finishing: VsTest - testAssemblies

The weird issue that "D:\a\r1\a" does not even exists and I cannot see what actually is going on the file system. There are no "a" folder on partition "D" - neither system nor hidden

Re-opening based on comments

@bryanmacfarlane just for the record this issue seems more related to using the build and release flows more than using the vstest task.

@Koshak118 You don't need to add an explicit task to download the artifacts. Release does this for you. You can link an artifact to a release and it downloads and unzips them to the System.Artifacts folder.
Are you using a private agent or a Hosted agent?
Seeing as to how long this issue seems going, If you can gimme a skype call during my working hours I can go out of my way (Coz the issue you are facing is not an issue with the VsTest task which we own, but with just getting your test dlls on to the machine to execute) and help you set up your Release for you. If this is not convenient then add me<EMAIL_ADDRESS>to the AAD of you org and give me permissions to your team project and I will be more than happy to set it up for you.

@ShreyasRmsft I've added you to the organization AAD, you were supposed to get the invitation
You are a contributor of a project now
Also a Skype credentials were sent to<EMAIL_ADDRESS>Let's get rid of this weirdo

@Koshak118, sent an invite and waiting for acceptance.

OK, so we are all set now
The main issue was that the Release definition has been run by Hosted agent rather than our own one
@ShreyasRmsft - thanks for the assistance by Skype, great job

hello good night, I have the following problem when I integrate a project of test studio and I attach the evidence of the configuration:

The release can not detect the tests that were assembled in the build
2018-07-12T01:00:50.1666758Z DiscoveryMessage : Logging TestHost Diagnostics in file: D:\a\_temp\Slice-71_VSTest_TestPlatform.host.18-07-12_01-00-48_62511_6.diag 2018-07-12T01:00:50.7601461Z ##[debug]ProxyDiscoveryRequest: HandleDiscoveryComplete Entered 2018-07-12T01:00:50.7601949Z ##[debug]ProxyDiscoveryRequest: HandleDiscoveredTests Entered 2018-07-12T01:00:50.7615019Z ##[debug]TestCaseFilter.FilterTests : Test Filtering invoked. 2018-07-12T01:00:50.7640128Z ##[debug]TestCaseFilter.GetFilteredTestCases : Starting filtering of discovered test cases. 2018-07-12T01:00:50.7641983Z ##[debug]TestCaseFilter.FilterTests : Test Filtering completed. 2018-07-12T01:00:50.7642414Z ##[debug]ProxyDiscoveryRequest: Dispose Entered 2018-07-12T01:00:50.7642741Z ##[debug]ProxyDiscoveryRequest: Abort Entered 2018-07-12T01:00:50.7643043Z ##[debug]PERF WARNING: DiscoverTests.PerformDiscovery : CreateDiscoveryRequest: took 3042.3294 ms 2018-07-12T01:00:50.7656225Z ##[debug]PERF WARNING: DiscoverTests.PerformDiscovery : VstestUtilities.GetTestPlatform: took 16299.5239 ms 2018-07-12T01:00:50.7724727Z ##[debug]PERF WARNING: DiscoverTests.PerformDiscovery : TestPlatformClient.PublisherMefHost.Instance: took 16306.8078 ms 2018-07-12T01:00:50.7851993Z Number of testcases discovered : 0 2018-07-12T01:00:50.7868043Z ##[debug]PERF: DiscoverTests:AddTestCases: took 0.003 ms 2018-07-12T01:00:50.8040303Z Discovered tests 0 from sources 2018-07-12T01:00:50.8040488Z ================================================================= 2018-07-12T01:00:50.8040776Z ##[debug]DiscoverTests.PerformDiscovery : Test Discovery completed. 2018-07-12T01:00:50.8045936Z ##[debug]PERF WARNING: DiscoverTests.PerformDiscovery: took 16957.0379 ms 2018-07-12T01:00:50.8212012Z [RunStatistics]This execution slice with id '71', received '1' testcases to execute out of which '0' is discovered. 2018-07-12T01:00:50.8212467Z ##[debug]PERF: Execution:FetchPreviousPhaseData: took 16985.0774 ms 2018-07-12T01:00:50.8235256Z ##[debug]PERF: Execution:Run: took 1.3988 ms 2018-07-12T01:00:50.8368483Z ##[debug]ExecuteAndPublish.Run : Cleanup invoked. 2018-07-12T01:00:50.8368843Z ##[debug]Cleanup.Cleanup : Creating new cleanup instance. 2018-07-12T01:00:50.8374337Z ##[debug]Cleanup.Run : Initialize cleanup 2018-07-12T01:00:50.8391239Z ##[debug]Processed: ##vso[task.uploadfile]D:\a\_temp\Slice-71_VSTest_TestPlatform.diag 2018-07-12T01:00:50.8391915Z ##[debug]Processed: ##vso[task.uploadfile]D:\a\_temp\Slice-71_VSTest_TestPlatform.host.18-07-12_01-00-48_62511_6.diag 2018-07-12T01:00:50.8396240Z ##[debug]Cleanup.Run : Cleanup finished 2018-07-12T01:00:50.8396467Z ##[debug]PERF WARNING: ExecutionAndPublish:Run: took 17121.8137 ms 2018-07-12T01:00:51.7541349Z ##[error]An exception occurred while executing the task. So the task is not executed. System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: No test assemblies found on the test machine matching the source filter criteria or no tests discovered matching test filter criteria. Verify that test assemblies are present on the machine and test filter criteria is correct. 2018-07-12T01:00:51.7552512Z ##[debug]Processed: ##vso[task.logissue type=error;]An exception occurred while executing the task. So the task is not executed. System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: No test assemblies found on the test machine matching the source filter criteria or no tests discovered matching test filter criteria. Verify that test assemblies are present on the machine and test filter criteria is correct. 2018-07-12T01:00:51.7553042Z    at Microsoft.VisualStudio.TestService.VstestAdapter.Execution.Run(ExecutionStateContext stateModdelContext, CancellationToken cancellationToken) in D:\V2.0\A2\_work\37\s\Ta\Agent\VS.TestService.VstestAdapter\Phases\Execution.cs:line 53 2018-07-12T01:00:51.7553763Z    at Microsoft.VisualStudio.TestService.VstestAdapter.ExecutionAndPublish.Run(ExecutionStateContext stateModelContext, CancellationToken cancellationToken) in D:\V2.0\A2\_work\37\s\Ta\Agent\VS.TestService.VstestAdapter\Phases\ExecuteAndPublish.cs:line 35 2018-07-12T01:00:51.7554073Z    at System.Threading.Tasks.Task1.InnerInvoke()
2018-07-12T01:00:51.7554205Z    at System.Threading.Tasks.Task.Execute()
2018-07-12T01:00:51.7554330Z    --- End of inner exception stack trace ---
2018-07-12T01:00:51.7554649Z ---> (Inner Exception #0) System.InvalidOperationException: No test assemblies found on the test machine matching the source filter criteria or no tests discovered matching test filter criteria. Verify that test assemblies are present on the machine and test filter criteria is correct.
2018-07-12T01:00:51.7554998Z    at Microsoft.VisualStudio.TestService.VstestAdapter.Execution.Run(ExecutionStateContext stateModdelContext, CancellationToken cancellationToken) in D:\V2.0\A2_work\37\s\Ta\Agent\VS.TestService.VstestAdapter\Phases\Execution.cs:line 53
2018-07-12T01:00:51.7555638Z    at Microsoft.VisualStudio.TestService.VstestAdapter.ExecutionAndPublish.Run(ExecutionStateContext stateModelContext, CancellationToken cancellationToken) in D:\V2.0\A2_work\37\s\Ta\Agent\VS.TestService.VstestAdapter\Phases\ExecuteAndPublish.cs:line 35
2018-07-12T01:00:51.7555848Z    at System.Threading.Tasks.Task1.InnerInvoke() 2018-07-12T01:00:51.7556103Z    at System.Threading.Tasks.Task.Execute()<--- 2018-07-12T01:00:51.7556187Z  2018-07-12T01:00:51.7556475Z ##[debug]ExecutionStateResolver.SetNextPhase : Execution State Resolver called. Curren Phase: Execution
Please I would appreciate guidance to solve this problem

@arturo2019 please create a fresh issue. Problems like this are mostly unrelated. We will look into it. You can paste the link to the new issue here for speedy tracking.

@ShreyasRmsft this is the new https://github.com/Microsoft/vsts-tasks/issues/7708 thank you very much

I had a similar issue where the cause was I had copied a semi-colon separated search pattern in the "Test files" field from my v1 Visual Studio Test task definition to my v2 definition
(i.e. build/Tests/*Test.dll;-:**\obj\**) .
Apparently v2 does not support semi-colon separation so i changed to new-line separation and it worked.
Hope it will be of value to someone with this issue, otherwise let me know if i should post this somewhere else.

Hi,
I am experiencing the same issue. I tried to change the pattern so it will look for the test automation dll files for Easy Repro and for the dll files I have created.
2018-12-10T05:18:40.4786270Z ##[section]Starting: VSTest
2018-12-10T05:18:40.4793309Z ==============================================================================
2018-12-10T05:18:40.4793387Z Task         : Visual Studio Test
2018-12-10T05:18:40.4793438Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2018-12-10T05:18:40.4793480Z Version      : 2.143.7
2018-12-10T05:18:40.4793531Z Author       : Microsoft Corporation
2018-12-10T05:18:40.4793569Z Help         : More information
2018-12-10T05:18:40.4793605Z ==============================================================================
2018-12-10T05:18:41.7705771Z SystemVssConnection exists true
2018-12-10T05:18:41.7706866Z SystemVssConnection exists true
2018-12-10T05:18:41.8062113Z Running tests using vstest.console.exe runner.
2018-12-10T05:18:41.8062205Z ======================================================
2018-12-10T05:18:41.8062454Z Test selector : Test assemblies
2018-12-10T05:18:41.8062527Z Test filter criteria : null
2018-12-10T05:18:41.8062592Z Search folder : D:\a\1\s
2018-12-10T05:18:41.8063031Z VisualStudio version selected for test execution : latest
2018-12-10T05:18:41.9496412Z Run in parallel : false
2018-12-10T05:18:41.9497179Z Run in isolation : false
2018-12-10T05:18:41.9497462Z Path to custom adapters : null
2018-12-10T05:18:41.9498264Z Other console options : null
2018-12-10T05:18:41.9498520Z Code coverage enabled : false
2018-12-10T05:18:41.9498650Z Diagnostics enabled : false
2018-12-10T05:18:41.9503571Z SystemVssConnection exists true
2018-12-10T05:18:41.9533569Z Run the tests locally using vstest.console.exe
2018-12-10T05:18:41.9533672Z ========================================================
2018-12-10T05:18:41.9544588Z Test selector : Test assemblies
2018-12-10T05:18:41.9549084Z Test assemblies : \Release**UIAutomation.dll\obj**
2018-12-10T05:18:41.9549396Z Test filter criteria : null
2018-12-10T05:18:41.9553317Z Search folder : D:\a\1\s
2018-12-10T05:18:41.9555096Z Run settings file : D:\a\1\s
2018-12-10T05:18:41.9556942Z Run in parallel : false
2018-12-10T05:18:41.9558969Z Run in isolation : false
2018-12-10T05:18:41.9581736Z Path to custom adapters : null
2018-12-10T05:18:41.9582167Z Other console options : null
2018-12-10T05:18:41.9584097Z Code coverage enabled : false
2018-12-10T05:18:41.9585216Z Diagnostics enabled : false
2018-12-10T05:18:41.9590023Z Rerun failed tests: false
2018-12-10T05:18:41.9593102Z VisualStudio version selected for test execution : latest
2018-12-10T05:18:42.8699816Z ========================================================
2018-12-10T05:18:42.9759033Z ##[warning]No test assemblies found matching the pattern: \Release**UIAutomation.dll\obj**.
2018-12-10T05:18:42.9759424Z ======================================================
2018-12-10T05:18:43.0135733Z ##[section]Finishing: VSTest
Changed it on the yml file but no luck. Still getting the issue.

@kpzara please open a new issue. This is required as we no longer track closed issues. Thanks!


Pre-built packages seg fault on start on linux kernel version >= 6

When I just tried to use the cli again I got a core dump, I'm not sure if its specific to my system, but if I use go install to install v1.5.1 it does work
strace upctl
execve("/usr/local/bin/upctl", ["upctl"], 0x7ffca96f3d70 /* 74 vars */) = 0
arch_prctl(ARCH_SET_FS, 0xdf0bd0)       = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x1} ---
+++ killed by SIGSEGV (core dumped) +++
[1]    1054086 segmentation fault (core dumped)  strace upctl

Used AUR to install on 6.0.2-2-MANJARO

Hi, thanks for reporting this!
Did you try to run go install with latest version?
Is the 6.0.2 in 6.0.2-2-MANJARO the Linux kernel version? Seems that GitHub actions runner that we use for builds and tests uses kernel version 5.15.0-.... So that would be one difference between the environments.

Did you try to run go install with latest version?

That's impossible because it's on v2, you need to append /v2 to the go.mod for it to work with the go install cmd.

Is the 6.0.2 in 6.0.2-2-MANJARO the Linux kernel version?

yes

So that would be one difference between the environments.

If I clone and build myself, it does work. Perhaps you can change the AUR version to build locally instead of installing a pre-build version. Since compiling is super fast that shouldn't be an issue

The just released v2.3.0 now has v2 included in the go module name, so go install will now also work with upctl versions >= 2.3.0. That would likely be the easiest installation method until we look into how to make pre-built packages work also on >= 6 kernels.

Debian 12 ("Bookworm") runs on kernel version 6.1.0-9-amd64. Installed upctl version 2.7.1 from pre-built deb packages and confirmed it works as expected.

Works on another OS with series 6 Linux kernel as well. Closing this off but reach out if the problem still persists.


fix perftests

pkg/attributes and pkg/adapterManager do not have benchmarks anymore
pkg/api and pkg/il/interpreter are missing in preftest
fix pkg/api/perf_test.go's panic by adding the new dispatcher
fix interpreterBenchmark_test.go by fixing the IL opcodes

Release note:

This change is

/release-note-none

/assign @douglas-reid

Thanks for fixing this.
We need to write a perf test for pkg/runtime as well, which is equivalent to the prior pkg/adapterManager perf test.
/lgtm


Pre-release: 0.20.60-d994

Checklist

[X] Used a personal fork of the feedstock to propose changes
[X] Bumped the build number (if the version is unchanged)
[X] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[X] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

Hi @mohammad-akhlaghi
Here are the outcomes of my tests.

As discussed in https://github.com/conda-forge/gnuastro-feedstock/issues/28 there is a way to build pre-releases. AFAIK, this only means working on a dev branch and adding the -dev suffix to the package name. This is what I did on this PR.
Issue 1: the .tar.lz extension is not recognized by conda and therefore I need to manually download and extract the source code. Valid extensions are .tar.gz, .zip, .tar.bz2, .tar.xz. Would it be possible to release gnuastro using one of those?
Issue 2: conda didn't like the 0.20.60-d994 version number; in particular the -d994 suffix. If possible, avoid using these kind of suffixes in the future.
Issue 3: the astmknoise was no longer found, is this expected with this new version of gnuastro? I have temporarily removed from the test section.

Please let me know your thoughts.
Best regards,
Sebastian

Wonderful, thanks a lot :clap:
I will try to reproduce these steps on another tarball and if I was able to reproduce the build, then I'll close this merge request (I am traveling now).
About issue 3, indeed, the astmknoise executable has been removed in this version of Gnuastro (because its features were present in a more complete way in other executables :wink:). This is described in the announcement, so users will know about it :wink:.
Thanks again,
Mohammad

Thanks a lot @sebastian-luna-valero for the nice checklist! I was able to replicate the process in #38 and confirm the build :smiley:. I am therefore closing this test build.


Wfcore 5343

Thanks for submitting your Pull Request!
Please delete this text, and add a link to the Jira issue solved by this PR.
Remember to use the Jira issue ID in the PR title, and any commits.

Core - Full Integration Build 10936 outcome was FAILURE using a merge of ac2f4a42a9ca8757b19427a1dc5d54377a068310
Summary: Compilation error: Compiler (new) Build time: 00:00:47

Core - Full Integration Build 10839 outcome was FAILURE using a merge of ac2f4a42a9ca8757b19427a1dc5d54377a068310
Summary: Compilation error: Compiler (new) Build time: 00:01:54

Core - Full Integration Build 10840 outcome was FAILURE using a merge of e99dd8c31a2dc10b10a1e39819ec5c84f9292f19
Summary: Compilation error: Compiler (new) Build time: 00:04:29

Core - Full Integration Build 10937 outcome was FAILURE using a merge of e99dd8c31a2dc10b10a1e39819ec5c84f9292f19
Summary: Tests passed: 552, ignored: 3; compilation error: Compiler (new) Build time: 00:05:18

Core - Full Integration Build 10943 outcome was FAILURE using a merge of a61ad541db4eb1251dbc6b6610db36b76232a526
Summary: Tests passed: 552, ignored: 3; compilation error: Compiler Build time: 00:03:14

Core - Full Integration Build 10846 outcome was FAILURE using a merge of a61ad541db4eb1251dbc6b6610db36b76232a526
Summary: Compilation error: Compiler Build time: 00:02:44


Junit4 / Grizzly2 on 2.1.5

Just updated from 2.1.4 to 2.1.5, with a legacy junit4 test environment I needed to add junit-vintage-engine to get tests to run.  This is because the junit5/jupiter deps come in via the new org.glassfish.jersey.test-framework.providers:jersey-test-framework-provider-grizzly2:jar:2.39:test.  They did not in 2.37.  No big deal, just documenting for anyone else in the same situation.
        <!-- run junit4 tests in a junit5 world -->
        <dependency>
            <groupId>org.junit.vintage</groupId>
            <artifactId>junit-vintage-engine</artifactId>
        </dependency>

Thanks!

@johnv-git Thanks for pointing this out!
I've added this to the release notes of Dropwizard 2.1.5:

https://github.com/dropwizard/dropwizard/discussions/6734
https://github.com/dropwizard/dropwizard/releases/tag/v2.1.5

Jersey 2.38 migrated from JUnit 4.x to JUnit Jupiter (JUnit 5.x): https://github.com/eclipse-ee4j/jersey/pull/5123
If you're still using JUnit 4.x in your projects, you might need to include org.junit.vintage:junit-vintage-engine for running JUnit 4 tests on the JUnit Platform.
<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <scope>test</scope>
</dependency>

See also JUnit 5 User Guide - Migrating from JUnit 4.


Add m1 to build matrix and release

Adds m1 target as discussed in #3982.
NOTE: This doesn't adjust the install.sh script.
Locally, I ran the wasmtime tests on a 2020 macOS Monterey M1:
test result: ok. 641 passed; 0 failed; 20 ignored; 0 measured; 0 filtered out; finished in 13.92s

Thanks for this! I don't think that CI can run the tests since I believe it's x86_64 hardware, but we should probably be able to add a builder so long as it doesn't take unduly long in CI. I think though that the build-tarballs.sh script needs to be updated for the new target

This seems fine to me as long as we have some testing on the releases -- unfortunately GitHub Actions still does not support macOS/aarch64 (see actions/virtual-environments#2187) so we can't run our tests in CI. (This has been the blocker for M1 support so far.) Last I heard, Embark Studios folks were running their own internal CI on M1 systems (@bnjbvr can you confirm this is still active?) -- if we have that, and if #3955 goes in so we have two weeks between branching a release and publishing it, then we could rely on Embark to notify us if there is some breakage with a given release. Not optimal, but it's something. Thoughts?

If we have a release milestone, we can do manual testing on a m1 and sign-off for releases. Not ideal, but workable. I would gladly volunteer until an automated solution is put in place.
A different option would be to use a self-hosted runner. Now that AWS and other providers have aarch64-darwin VM's, we could kickoff a workflow for verifying the m1 binary. This issue seems a little closer to completion than the a hosted VM but that's just a guess https://github.com/actions/runner/issues/805.

I think even with a custom runner our hands are sort of tied because GitHub's own official recommendation is that for public repositories you shouldn't use self-hosted runners for security reasons. If you're ok manually verifying that builds are ok that may be the best way to go. With the release process from https://github.com/bytecodealliance/wasmtime/pull/3955 there will be a 2-week window between when a release branch is created and when it's actually published which should provide a good opportunity to test on non-CI-tested-platforms.
If you'd like it could also be set up to notify you. We could apply a tag to the PR and then using our labeling bot you'd get auto-cc'd on any PRs related to a major version bump

Confirming that we still do have some nightly CI job running the test suite (using the checked-in scripts) every day, using the latest commit on main at the time of running the CI job. Results are public, but only we at Embark Studios can start jobs etc.

That's fantastic. Then this might be ready to merge? I added a reference to embark's CI so that it will be part of the initial PR for releases.

Oh I think CI is still failing on this, the build-tarballs.sh script needs an update to use the right path to the artifacts (right now it assumes target/release/* since macOS previously didn't cross-compile but now it's sometimes in target/aarch64-apple-darwin/release/*)

I think the matrix entry for the test build may have snuck back in? Otherwise though the script changes look good to me, thanks!
To double-check, can you confirm the release on this page works locally for you? This should be the direct link there

I think the matrix entry for the test build may have snuck back in? Otherwise though the script changes look good to me, thanks!

Ack!

To double-check, can you confirm the release on this page works locally for you? This should be the direct link there

LGTM tried the example and ran a compile on one of my wasm modules.
./wasmtime --version
wasmtime 0.37.0


Rich component dirs config

This adds a "rich" component_dirs setting, which allows multiple component directories to be added and configured independently. For example:
class MyApp::Container < Dry::System::Container
  configure do |config|
    config.root = __dir__

    config.component_dirs.add "lib" do |dir|
      dir.auto_register = true    # defaults to true
      dir.add_to_load_path = true # defaults to true
      dir.default_namespace = "my_app"
    end

    config.component_dirs.add "another" do |dir|
      # etc.
    end
  end
end

The following top-level settings have been removed:

default_namespace
add_component_dirs_to_load_path
auto_register

Since these are now configurable per-component directory, they no longer make sense at the top-level (and really, they only mostly worked by happenstance until now, relying upon the fact that most usages of dry-system involved a single primary directory for auto-registration).
The component_dirs setting value is a custom object, a Dry::System::Config::ComponentDirs, which yields a Dry::System::Config::ComponentDir for each added directory. These are simple classes including Dry::Configurable and delegating all methods to the internal config object via #method_missing (sidenote: I've followed this pattern in so many places now I think dry-configurable should support it out of the box). The ComponentDirs setting value is made cloneable (which is required so that we don't needlessly propagate component dir configs across different containers) via a new cloneable option on dry-configurable settings (see https://github.com/dry-rb/dry-configurable/pull/102, which I'll release after some final testing of this branch of dry-system).
The component_dirs are then exposed as Container.component_dirs, which are instances of Dry::System::ComponentDir, representing the component directory placed within the container's configured root. These objects wrap the component dir's config, and importantly, provide a #component_file(component_path) method for finding component files (i.e. Ruby source files) within the directory.
This method is then used by another new method, Component.locate(identifier, component_dirs, **options), called from Container.component, which is the sole method for the container to build Component instances. .locate is passed the container's array of component_dis, and will return a Component instance for the first matching component file (in this way, component dirs act like a typical "path", in that the first match wins).
The benefit of this arrangement is that because Component.locate now has the full set of ComponentDir objects, which contain all their configuration, then the resulting Component instance can access that configuration as required! In particular, this means a component found within a component_dir configured with a default_namespace can be initialized with that namespace. This allows us to greatly simplify Container.load_local_component (more on that later) and reduce it from a two-pass operation (in which the method would call itself again with the default_namespace prepended if a component could not be found in the first pass) to a single-pass.
With this change, components are found directly from the file system in both main container use cases, (a) when finalizing the container and auto-registering all components, as well as (b) when components are lazily loaded in a non-finalized container. This allows us to provide a file_path arg when initializing a Component, which means every component can now be asked if its #file_exists?. This new consistency across components allows us to further simplify .load_local_component by doing away with the entire .require_component block method, whose only jobs were to (a) try and find a file matching the already-initialized component (no longer required, we know about the file up front now!), and (b) require that file (also no longer required, since this PR has Loader#call make a call to require! as a first step, keeping in mind that require! may be a no-op for the autoloading loader). This change reduces the amount of low-level implementation detail that Container needs to know about, and pushes that detail down into the place where it is most relevant, the Loader.
(A small supporting change to Loader is that it's now initialized with the whole component, rather than just the component's path. This allows it to check if the component.file_exists? before attempting to require it)
Another way we increase the consistency across lazily-loaded vs auto-registered components is by adding a check for a file's magic comment-based configuration as a step inside Component.locate. This means that if a component has a magic comment specifying auto_register: false, then it can also be skipped when lazily resolving, which was never the case before! I plan to take this further and completely remove the magic comment parsing from AutoRegistrar, but that will happen when I remove Container.auto_register! in a follow-up PR (this current PR is already big enough!).
A big upshot of all these changes is that they allow Container.load_component and .load_local_component to be dramatically simplified! .load_component now contains all the detail of deciding which loading mechanism to use. Instead of 4 levels of nesting, it now has only 1 level, with two early returns for the cases where a bootable component has already done the work of loading the specified component. .load_local_component drops from a 14-line method with multiple conditionals (and lots of overlap with .load_component) to a 3-line method with the simple job of registering the component or not. This pair of methods has confused me nearly every time I've encountered them over the last few years, and I'm so happy the changes for the component_dirs made this refactoring possible!
As for next steps here:

I'd love your review of this PR. It's an important change, so please take the time you need.
I'll test these changes with one of the larger dry-system-backed production apps I have access to
If that goes well, I'll release the dry-configurable changes in https://github.com/dry-rb/dry-configurable/pull/102 and update this branch to used the released version
Once all reviewers are happy, we can merge this PR

There's one follow-up change I'd like to make before this can be released in a new gem version, and that's to rip out Container.auto_register! given all auto-registration should now happen only over the configured component_dirs. This requires some changes to dry-rails, so I'd like to do that as a separate PR.

@solnic @flash-gordon @jodosha This is now ready for review, complete with a thorough PR description!

@solnic @flash-gordon @jodosha This is now ready for review, complete with a thorough PR description!

As a before/after comparison, here's how .load_component and .load_local_component used to look:
def load_component(key, &block)
  return self if registered?(key)

  component(key).tap do |component|
    if component.bootable?
      booter.start(component)
    else
      root_key = component.root_key

      if (root_bootable = component(root_key)).bootable?
        booter.start(root_bootable)
      elsif importer.key?(root_key)
        load_imported_component(component.namespaced(root_key))
      end

      load_local_component(component, &block) unless registered?(key)
    end
  end

  self
end

def load_local_component(component, default_namespace_fallback = false, &block)
  if booter.bootable?(component) || component.file_exists?(component_paths)
    booter.boot_dependency(component) unless finalized?

    require_component(component) do
      register(component.identifier) { component.instance }
    end
  elsif !default_namespace_fallback
    load_local_component(component.prepend(config.default_namespace), true, &block)
  elsif manual_registrar.file_exists?(component)
    manual_registrar.(component)
  elsif block_given?
    yield
  else
    raise ComponentLoadError, component
  end
end

And here's how they look after this PR:
def load_component(key)
  return self if registered?(key)

  component = component(key)

  if component.bootable?
    booter.start(component)
    return self
  end

  booter.boot_dependency(component)
  return self if registered?(key)

  if component.file_exists?
    load_local_component(component)
  elsif manual_registrar.file_exists?(component)
    manual_registrar.(component)
  elsif importer.key?(component.root_key)
    load_imported_component(component.namespaced(component.root_key))
  end

  self
end

private

def load_local_component(component)
  if component.auto_register?
    register(component.identifier) { component.instance }
  end
end

Hopefully this helps make it clear why I'm so excited about this PR!

As a before/after comparison, here's how .load_component and .load_local_component used to look:
def load_component(key, &block)
  return self if registered?(key)

  component(key).tap do |component|
    if component.bootable?
      booter.start(component)
    else
      root_key = component.root_key

      if (root_bootable = component(root_key)).bootable?
        booter.start(root_bootable)
      elsif importer.key?(root_key)
        load_imported_component(component.namespaced(root_key))
      end

      load_local_component(component, &block) unless registered?(key)
    end
  end

  self
end

def load_local_component(component, default_namespace_fallback = false, &block)
  if booter.bootable?(component) || component.file_exists?(component_paths)
    booter.boot_dependency(component) unless finalized?

    require_component(component) do
      register(component.identifier) { component.instance }
    end
  elsif !default_namespace_fallback
    load_local_component(component.prepend(config.default_namespace), true, &block)
  elsif manual_registrar.file_exists?(component)
    manual_registrar.(component)
  elsif block_given?
    yield
  else
    raise ComponentLoadError, component
  end
end

And here's how they look after this PR:
def load_component(key)
  return self if registered?(key)

  component = component(key)

  if component.bootable?
    booter.start(component)
    return self
  end

  booter.boot_dependency(component)
  return self if registered?(key)

  if component.file_exists?
    load_local_component(component)
  elsif manual_registrar.file_exists?(component)
    manual_registrar.(component)
  elsif importer.key?(component.root_key)
    load_imported_component(component.namespaced(component.root_key))
  end

  self
end

private

def load_local_component(component)
  if component.auto_register?
    register(component.identifier) { component.instance }
  end
end

Hopefully this helps make it clear why I'm so excited about this PR!

sidenote: I've followed this pattern in so many places now I think dry-configurable should support it out of the box

@timriley this must be added prior 1.0.0. dry-validation does this too and I suspect rom-rb's core config could benefit from this too.

sidenote: I've followed this pattern in so many places now I think dry-configurable should support it out of the box

@timriley this must be added prior 1.0.0. dry-validation does this too and I suspect rom-rb's core config could benefit from this too.

@flash-gordon Thanks so much for the detailed feedback! I'm very happy to receive this; big changes like this are well worth fine-tuning, especially considering our attention is likely to turn elsewhere after this is done.
I've incorporated all your direct suggestions and force-pushed a new history. Have also left you a question about how best to avoid the object allocations through the keyword param splats.

@flash-gordon Thanks so much for the detailed feedback! I'm very happy to receive this; big changes like this are well worth fine-tuning, especially considering our attention is likely to turn elsewhere after this is done.
I've incorporated all your direct suggestions and force-pushed a new history. Have also left you a question about how best to avoid the object allocations through the keyword param splats.


String generation from complex regex in integration tests

Closes #8593
Closes #8607
Contributes to #8447
This PR :

Fixed StringGen in integration tests to correctly generate long and complex strings from a regex. StringGen can't do this when the state space of a regex is very large, as described in #8593.

Implemented a decimal generation function to speed up the generation of decimals that were previously generated by sre_yield.

Updated _cache_repr of UniqueLongGen and RepeatSeqGen to make them simpler.

Some small changes in test cases that affected by the StringGen change:

In test_regexp_extract_all_idx_out_of_bounds, the error message is raised only if the given string matches the pattern, so it can't raise the error with mk_str_gen('[abcd]{0,3}') after StringGen changes because of bad luck. This behavior is matched between cpu and gpu, so I just change the gen to mk_str_gen('[a-d]{1,2}.{0,1}[0-9]{1,2}').

One regex used by test_json_tuple and test_get_json_object will generate numbers with leading zeros, which is not compatible between spark and plugin, as described in #8607. I changed it to not generate such numbers.

When processing string to sql in _convert_to_sql, there is a edge case that if there is a \' in string before replace, the result will be r\\', which will make the ' escape. So I replaced all r\ with r\\ first to prevent it.

[UPDATED] test_hash_groupby_collect_set_on_nested_type and test_hash_reduction_collect_set_on_nested_type faile after this PR, but they also fail when selecting different seeds, as described in #8716. I don't think it's related to this PR, so I  temporarily marked them as XFAIL.

[UPDATED] Some test results:
For DecimalGen, this change brings some performance improvement. I tested this one from current IT:
def test_special_decimal_division():
    for precision in range(1, 39):
        for scale in range(-3, precision + 1):
            print("PRECISION " + str(precision) + " SCALE " + str(scale))
            data_gen = DecimalGen(precision, scale)
            assert_gpu_and_cpu_are_equal_collect(
                    lambda spark : two_col_df(spark, data_gen, data_gen).select(
                        f.col('a') / f.col('b')))

Before this PR: 0:07:05, after this PR: 0:04:44
For StringGen, if we use ''.join(rand.choice(sre_yield.CHARSET + ['\n']) for _ in range(30)) to generate default strings, the speed is even slower, so I leave it unchanged. I tested this case:
@pytest.mark.parametrize('data_gen', [string_gen]*100, ids=idfn)
def test_string_gen_perf(data_gen):
    assert_gpu_and_cpu_are_equal_collect(
            lambda spark: unary_op_df(spark, data_gen))

Before this PR: 52.04s, First commit of this PR: 52.49s. Other string-related tests are also a bit slower. Thus, I will not close the sre_yield performance issue #8477.
The overall performance improvement will be small because it only affects DecimalGen.
Note

Now the DecimalGen will only generating numbers with the given precision rather than (1, precision) because it may cause precision limitations in some cases, leading to CPU/GPU mismatch.

test_str_to_map_expr_fixed_pattern_input and test_initcap are failed when using random() * length as index but passed now. I don't know why and I will investigate them later. Currently they don't block anything.

build

build


add bubblesort in java

For short term contributors: we understand that getting your commits well
defined like we require is a hard task and takes some learning. If you
look to help without wanting to contribute long term there's no need
for you to learn this. Just drop us a message and we'll take care of brushing
up your stuff for merge!
Fixes #{ISSUE_NUMBER}
By submitting this pull request I confirm I've read and complied with the
below declarations.

[ ] I have read the Contribution guidelines and I am confident that my PR reflects them.
[ ] I have followed the commit guidelines for this project.
[ ] My code follows the standard code structure.
[ ] This pull request has a descriptive title. For example, {Tag}: Add {Algorithm/DS name} [{Language}], not Update README.md or Added new code.
[ ] This pull request will be closed if I fail to update it even once in a continuous time span of 7 days.
[ ] This pull request shall only be reviewed and merged once all the checks passes. No maintainer or supporter shall be obliged to review it before this condition is met.
[ ] I have mentioned the issue number correctly (with hyperlink) in this pull request description.

After you submit your pull request, DO NOT click the 'Update Branch' button.

Your checks are breaking. Fix it. and include README table also. Feel free to contact.

don't know what is check breakin


[3462] add /sha endpoint to see the commit SHA in deployed env

Context
Sometimes it's important to be able to see what version of the code is actually running for debugging issues.
Changes proposed in this pull request
Add an endpoint that exposes the git commit SHA at /sha.
Checklist

[ ] Make sure all information from the Trello card is in here
[ ] Attach to Trello card
[ ] Rebased master
[ ] Cleaned commit history
[ ] Tested by running locally

 Add /sha endpoint to api


Avoid controller/service lock AB/BA situation

Currently there is an instance of controller and service lock being
obtained in different order which causes the AB/BA deadlock. Do not ever
wrap controller lock around service lock.
Signed-off-by: Jana Radhakrishnan<EMAIL_ADDRESS>

LGTM

LGTM


Roll Flutter Engine from 89d864552acd to a15b93d8d49c (16 revisions)

https://github.com/flutter/engine/compare/89d864552acd...a15b93d8d49c
2023-09-20<EMAIL_ADDRESS>Clang tidy-ify DlColor and friends. (flutter/engine#46122)
2023-09-20<EMAIL_ADDRESS>Apply the right tag for linux fuchsia cipd packages. (flutter/engine#46123)
2023-09-20<EMAIL_ADDRESS>Roll Skia from b78c91996051 to 86f48da2c812 (4 revisions) (flutter/engine#46121)
2023-09-20<EMAIL_ADDRESS>Roll Skia from d923bab3d5fa to b78c91996051 (4 revisions) (flutter/engine#46119)
2023-09-20<EMAIL_ADDRESS>Make dl_color_(filter|source) tidy. (flutter/engine#46111)
2023-09-20<EMAIL_ADDRESS>Make dl_(image|mask|path)_(filter|effect).h tidy! (flutter/engine#46110)
2023-09-20<EMAIL_ADDRESS>Migrate from LOG_X to kLogX. (flutter/engine#46107)
2023-09-20<EMAIL_ADDRESS>Roll Skia from b3c1f49821d8 to d923bab3d5fa (2 revisions) (flutter/engine#46118)
2023-09-20<EMAIL_ADDRESS>Make a variety of low-impact Clang tidy fixes in Impeller. (flutter/engine#46116)
2023-09-20<EMAIL_ADDRESS>[ios] fix asset url not found when loading app extension (flutter/engine#46073)
2023-09-20<EMAIL_ADDRESS>Make a variety of low-impact Clang tidy fixes. (flutter/engine#46114)
2023-09-20<EMAIL_ADDRESS>Roll Skia from c19115e8f712 to b3c1f49821d8 (3 revisions) (flutter/engine#46112)
2023-09-20<EMAIL_ADDRESS>Roll Skia from f4238844089f to c19115e8f712 (5 revisions) (flutter/engine#46108)
2023-09-20<EMAIL_ADDRESS>[web] Move context menu handling to its own class (flutter/engine#46042)
2023-09-20<EMAIL_ADDRESS>Delete ci/lint.sh, which is no longer used. (flutter/engine#46049)
2023-09-20<EMAIL_ADDRESS>[web] Make PlatformViewManager a clear singleton (flutter/engine#46044)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC jonahwilliams@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Kaggle filesystem

Fix gh-2.
Still missing actual download of files.

This was a bad idea, closing.


Add ExifTool

ExifTool by Phil Harvey
ExifTool is a platform-independent toolset for reading, writing and editing meta information in a wide variety of file types, as well as the maker notes of many digital cameras.

Thanks for the recommendation, if you would like it to be added to the list, please create an Pull Request Instead. Thanks for Understanding!

Thanks for the suggestion, but:

It's your list, not mine.
I'm not currently setup to do GitHub Pull Requests, which is too much hassle just for this.
Since you've not that interested in adding it, I'll just write it off as a waste of my time.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

As I said, that's too much hassle.


Service connection on error

Hi,
I have the following error which wasn't here yesterday.
Exception Message: The pipeline is not valid. Job Phase_1: Step PurgeAzureCDNEndpoint input ConnectedServiceName expects a service connection of type Azure but the provided service connection VSTS-Gyosa-DEV is of type azurerm. (type PipelineValidationException)

I updated from v1.* to v2.* but i get the same message. Idem after switching the Azure Connection Type.
Any idea why ?
Thanks

Some feedback, looks like its due to an update on AzureDevOps side.
https://developercommunity.visualstudio.com/content/problem/538728/release-failing-due-to-service-connection.html

Thanks for the link! I'll try to follow up on this issue.
Have you tried the different workaround provided?

Yes, i removed your add-on and added a new Azure PowerShell script with the Unpublish-AzureRm.... which works right now.

Microsoft seems to have fixed the problem on their side. Thanks for reporting!


Use Hamcrest assertions instead of JUnit in  microprofile/openapi - backport 2.x (#1749)

Part of #1749
I've already signed OCA. It's an automation problem.
Original PR

/oca-checked

/trigger


Livia Cabernard

Submitter
I am a postdoc from the Ecological Systems Design Group at ETH Zurich.
Description
Global supply chain analysis of the carbon footprint (GHG emissions) of global plastics production in 2015.
Figure 1 from:
Cabernard, L., Pfister, S., Oberschelp, C. et al. Growing environmental footprint of plastics driven by coal combustion. Nat Sustain 5, 139–148 (2022). 👉Link study
Detailed description:
a–e, The sum of each horizontal bar of the flow chart refers to the carbon footprint of global plastics production in 2015 (1.9 GtCO2e, 100%) and allocates it to the different perspectives in the global value chain: the sectors where GHG emissions are released (a); the processes that release GHG emissions, that is, combustion (88%) and non-combustion (7%) processes of fossil fuels (left side: 95% in total) and other processes (right side: 5 % in total): biogenic emissions (2.7%), hydrofluorocarbon emissions (1.7%) and cement production (0.8%) (b); the regions where GHG emissions are released (production perspective) (c); the regions where plastics are finally used (consumption perspective) (d); and the end products or sectors where plastics are finally used: plastics packaging material is allocated to the end product or sector where it is used (for example, food packaging in the food sector) (e). The flows show the linkages between the perspectives. To enhance clarity, linkages contributing to less than 0.1% of the plastics-related carbon footprint are not shown. The small graphs on the right show the temporal evolution of the carbon footprint of global plastics production for each perspective (a–e) over the past two decades (1995–2015) and in the future (2020–2030) if the world follows the IEA’s projection for a 2 °C or 6 °C scenario34,35 (but not accounting for the decrease in global GHG emissions in 2020 due to the COVID-19 pandemic, as in Liu et al.66). The colours of the graphs on the right correspond to the bars of the flow chart. The red line in each graph represents the global share.
Database(s) used

[ ] Used example data from the contest repository
[x] Used another database 👉 exiobase for 1995-2015. For the 2° and 6° scenarios we used the future exiobase database from Wiebe et al 2018

Links to the code and visualization
Code
👉 Matlab code to calculate results based on input data from exiobase and Wiebe et al 2018
👉 R and csv input files to create sankey in R with the network D3package
👉 tableau and excel input file to create plots on temporal trends
👉 Compilation and labelling of figures in pptx
Visualizations
👉 Visualization as pdf
Presentation video
👉 Video
License
Creative Commons Attribution 4.0 International

@liviacabernard Thanks for you entry, it was received in time and meets the criteria for a successful submission. We will have a final jury meeting in mid-March, and you can expect feedback by the end of March.


Old versioned documentation for Configuring Mail Providers

Hi, I'm trying to set up mail coach using the documentation and realize it's outdated. take for example https://spatie.be/docs/laravel-mailcoach/v5/configuring-mail-providers/mailgun. Mailcoach doesn't provide the Mail configuration page anymore and I keep running into this error when I try to create campaigns:
Exception: count(): Argument #1 ($value) must be of type Countable|array, null given (View: /var/www/html/vendor/spatie/laravel-mailcoach/resources/views/app/emailLists/layouts/emailList.blade.php) (View: /var/www/html/vendor/spatie/laravel-mailcoach/resources/views/app/emailLists/layouts/emailList.blade.php);

The mail configuration views are part of the spatie/mailcoach-ui package, make sure that package is installed as well if you're using the standalone version.
Otherwise you'll need to configure the mailers manually, for that you can reference https://laravel.com/docs/9.x/mail#configuration

I am using it within an existing project. I had to include mailers config in mail.php to solve the issue, something like:
 'mailers' => [
        'mailgun' => [
            'domain' => env('MAILGUN_DOMAIN'),
            'secret' => env('MAILGUN_SECRET'),
            'endpoint' => env('MAILGUN_ENDPOINT', 'api.mailgun.net'),
        ],
  ]


buggy NO_COLOR=true

Hi,
with aurutils-git, NO_COLOR=true is broken.
the package name is duplicate.
$ NO_COLOR=true aur search -n -k NumVotes arch-secure-boot
aur/arch-secure-bootaur/arch-secure-boot 1.3.2-1 (+2 0.00%)
    UEFI Secure Boot for Arch Linux + btrfs snapshot recovery

$ aur search -n -k NumVotes arch-secure-boot
aur/arch-secure-boot 1.3.2-1 (+2 0.00%)
    UEFI Secure Boot for Arch Linux + btrfs snapshot recovery

Regards

Seems like missing newline in no_color codepath

Hmm, i'm using kitty, with color enable, i can see the get the link when mouse over aur/arch-secure-boot and with NO_COLOR=true i can get the link on the second duplicate aur/arch-secure-boot.

Yes, you should it's always used.  Not quite sure why it's only doubling the name, as it's not missing elements that would otherwise interfere with printf(1)'s implicit looping feature.

The issue appears to be that the package name is printed a second time when ALL_OFF is undefined i.e. if colorize() was not run. The function should return in that case:
if [[ ! -v ALL_OFF ]]; then
    printf "%s" "$mesg"
    return  # <---
fi

That said, I'm not sure NO_COLOR should disable the link since it's only about "color" escapes (which are e.g. annoying when debugging with set -x).

Fixed with #996


README: update the status of the project

It's now outdated after version 0.4 ;)

self-ACK e1ee4c16a400e46339affaa97e097970a6af4117 -- trivial


Authentication via Oauth2 auth-provider using Basic Authorization is not working

While using Backstage's battery-included generic oauth2-auth-provider, the authentication fails to complete when using an authentication provider that expects clientId and clientSecret to be passed in Authorization header.
Expected Behavior
Successful authentication via all auth-providers that are based on oauth2.0
Current Behavior
Authentication is failing with the following error:
Authentication Failed, failed to obtain access token : authorization header required
Backstage is using passport-oauth2 strategy which is in turn using node-oauth's getOAuthAccessToken function
This function is making a post request to configured oauth-provider's token endpoint by setting clientId and clientSecret as request params.
(Refer: https://github.com/jaredhanson/passport-oauth2/blob/ee3fe9f17c0f3a90f2d9d938f267e9942b9fba49/lib/strategy.js#L130)
However, the expected method of authentication via most auth-providers is by using client id and secret in Authorization header as follows:
Authorization: Basic b65encodedclientId:clientSecret
This is the reason why authentication fails with the error message of missing authorization header.
However, node-oauth lib is no longer being maintained.
Possible Solution

To change passport strategy to one that also uses authorization header in post request to token endpoint.

Steps to Reproduce

Configure oauth2 config in app-config by setting the callback url, authorization url, tokenUrl, clientId, clientSecret

auth:
  environment:
  providers:
    oauth2:
      development:
        clientId:
        clientSecret:
        callbackUrl:
        authorizationUrl:
        tokenUrl:

Add the provider to sign-in page: https://backstage.io/docs/auth/#adding-the-provider-to-the-sign-in-page
Ensure that the provider configured uses Authorization header for authentication via clientID and clientSecret.

Context
To configure authentication via in-house generic oauth2 auth-provider using Backstage.
Existing issues with node-oauth and passport
https://github.com/ciaranj/node-oauth/issues/300
https://github.com/jaredhanson/passport-oauth2/issues/75

Hm. The constructor for the strategy here seems to be able to accept a customHeaders field, and that does get passed down and then used by node-oauth. Does that mean that we could encode this ourselves as a custom Authorization header in our own OAuth2AuthProvider?

OAuth 2.0 is unfortunately not a strict enough standard to have things work out of the box in a lot of situations.
Is it a well known OAuth provider that you're using @sonali-mishra-94? Perhaps there's a more specific passport provider that we can add instead.

Thanks @Rugvip for taking a look at this!
I work at Grab and I am trying to use an in-house oauth2 provider that is based upon oauth2.0 and is expecting the credentials to be passed in the Authorization header. So the passport strategy for oauth2 should have ideally been perfect.
We are keen on using in-house provider as it is an adopted company wide standard.
I think what @freben has suggested can be tried. I am sure a lot of others might be having the same problem given how many providers make use of auth headers...

@sonali-mishra-94 Would you be able to try making a change to the existing Backstage strategy for this, to supply such a header? It would be valuable to be able to see that the end result works for your particular installation.
In such a pull request, we might discuss further how this behavior should be enabled. For example, whether there should be some config parameter that turns it on, or if it should be done through a header generator callback given to the factory for the strategy, or something else.

Raised the following PR to supply custom authorization header:
https://github.com/backstage/backstage/pull/8292

Closing since the PR is merged.

Thanks alot for helping me with my first contribution to Backstage !
Looking forward to more PRs 😁