feature: Add invocation request hooks

Description

Fixes #

From talking to @Hazhzeng , we want to be able to execute custom logic in a customer's function, especially when it comes to reading/modifying the context before the request is made. This PR adds hooks for before and after the handling of the invocation request.
An example using OpenCensus AzureExporter which exports telemetry to App Insights. This is in a normal HTTPTrigger function.
import json
import logging
import requests

import azure.functions as func
from azure_functions_worker.extensions import register_before_invocation_request, register_after_invocation_request
from opencensus.ext.azure.trace_exporter import AzureExporter
from opencensus.trace import config_integration
from opencensus.trace.samplers import ProbabilitySampler
from opencensus.trace.tracer import Tracer
from opencensus.trace import execution_context
from opencensus.trace.propagation.trace_context_http_header_format import TraceContextPropagator

config_integration.trace_integrations(['requests'])

exporter = AzureExporter(instrumentation_key='<your-ikey-here>')

def callback(context):
    print("TRACEPARENT: " + context.trace_context.Traceparent)
    span_context = TraceContextPropagator().from_headers({"traceparent": context.trace_context.Traceparent, "tracestate": context.trace_context.Tracestate})
    # Create a tracer using the current traceparent passed in from the context in the callback
    tracer = Tracer(span_context=span_context, exporter=exporter, sampler=ProbabilitySampler(1.0))
    context.tracer = tracer #  < -- modify the context by passing in a tracer

def callback2(context):
    print("DONE")

register_before_invocation_request(callback)
register_after_invocation_request(callback2)

def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    tracer = context.tracer
    execution_context.set_opencensus_tracer(tracer)  # < -- sets the passed in tracer as the current tracer
    with tracer.span("parent"):
        response = requests.get(url='http://example.com')
    return json.dumps({
        'method': req.method,
        'ctx_func_name': context.function_name,
        'ctx_func_dir': context.function_directory,
        'ctx_invocation_id': context.invocation_id,
        'ctx_trace_context_Traceparent': context.trace_context.Traceparent,
        'ctx_trace_context_Tracestate': context.trace_context.Tracestate,
    })

In the above code, the context is passed through these hooks and made available to the callback functions. Within the callbacks, OpenCensus SDK treats the context as the parent context, creates a new tracer with this context as the parent, and sets this tracer to be the one that is used currently (execution_context.set_opencensus_tracer(tracer)). Then any custom dependencies that will be tracked using Opencensus SDK (like response = requests.get(url='http://example.com'), will be connected with the current context within the functions environment.
PR information

[x] The title of the PR is clear and informative.
[ ] There are a small number of commits, each of which has an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[ ] If applicable, the PR references the bug/issue that it fixes in the description.
[ ] New Unit tests were added for the changes made and CI is passing.

Quality of Code and Contribution Guidelines

[x] I have read the contribution guidelines.

/azp run

This is a good idea, but wondering if there should be a discussion in this repo about how this is implemented. (if the issue exists, please link to it, would love to take a look)
Specifically interested in examples of how these callbacks are registered, and whether there was any discussions on pros/cons compared to a middleware/pipeline model.
We also had asks for this in other languages, so it feels like there might be a broader discussion here to ensure the right scenarios are covered and see if there's a possibility to create a consistent approach.

@anthonychu
There isn't an issue that currently exists for this. There is an internal email thread that proposes the idea (but not any implementation details) by @Hazhzeng and I. We need this feature on the monitoring team to be able to hook on with our SDKs and I took a stab at it to expedite the discussions.
@markwolff is implementing this on the Node side and we are essentially doing the same thing. What would you recommend for next steps if we want to get this feature in ASAP?

@lzchen OK so this is internal to the workers and it's specific to supporting App Insights correlation and is not something that's exposed to the customer? If that's the case, I don't have any concerns.

Hi @lzchen, thanks for making the PR, overall it looks good.
Could you update the PR description and give us a code example on how does AppInsight integrate with the extension interface? (e.g. register_before_invocation_request and clear_before_invocation_request). It would be great if you can explain it in a few sentences.

@Hazhzeng
Updated the PR description with an example and info on how this lights up our correlation scenario.

@Hazhzeng - In the example, I can see the following -
from azure_functions_worker.extensions import register_before_invocation_request, register_after_invocation_request

Would this mean that the customer needs to add azure_functions_worker to the requirements.txt?

/cc @lzchen

@vrdmr - Well spotted
The customer writing the Azure Functions code shouldn't need to import the azure_functions_worker, but the library writing the extension will need to do it. I would expect the appinsight will publish a separate library and wrapping the opencensus (e.g. azure-functions-appinsight) and place the initialization code into it. So that the customer only need to import azure-functions-appinsight.
But this still introduces an issue since Linux Consumption SKU prioritizes the customer sys.path over the Azure Functions Worker sys.path. This has a potential to trigger the backward-compatibility issue like https://github.com/Azure/azure-functions-python-worker/issues/729 since azure-functions-appinsight will also introduce an installation of azure_functions_worker -> grpcio, grpcio-tools -> protobuf.

I think to resolve this issue from the root, we do need to fork the grpcio (Apache-2.0), grpcio-tools (Apache-2.0) and protobuf (license) library from google. I consider other alternatives but none of them actually solve the issue.

If we try to remove the azure_functions_worker library in remote build, we still don't have control on grpcio, grpcio-tools and protobuf dependent libraries (e.g. say if we remove grpcio, grpcio-tools and protobuf if customer define azure_functions_worker in requirements.txt, this will break customers using tensorflow since it also depends on grpcio and grpcio-tools).
Use pipdeptree to resolve grpcio-tools dependent libraries and remove them only if used by azure_functions_worker. This is a generally safer solution, but the oryx build does not generate virtual environment and thus the packages cannot be analyze correctly.

Load azure_functions_worker from customer packages. This is dangerous since we have no idea which customer brought old worker into function app. Enabling this will break the current behavior in their function app and they will lose all the bug fixes and update features.

@lzchen - Sorry, I should've caught this much earlier when planning the change. We need to solve this potential backward-compatibility issue before releasing this change, will reset my review.

@Hazhzeng
Any updates on this?

@Hazhzeng
Any updates on this?

Hey @vrdmr,
I think we should continue merge this PR since we introduces PYTHON_ISOLATE_WORKER_DEPENDENCIES flag.
If the customer runs into the protobuf issue. We can just simply ask them to turn on the feature flag for mitigation.

Hey @vrdmr,
I think we should continue merge this PR since we introduces PYTHON_ISOLATE_WORKER_DEPENDENCIES flag.
If the customer runs into the protobuf issue. We can just simply ask them to turn on the feature flag for mitigation.

Sounds good @Hazhzeng.
@Hazhzeng/@lzchen - Is there a way to test this end to end? We don't need to block the PR for the E2E, but can be added post this PR. Your thoughts?

Sounds good @Hazhzeng.
@Hazhzeng/@lzchen - Is there a way to test this end to end? We don't need to block the PR for the E2E, but can be added post this PR. Your thoughts?

@anthonychu

@lzchen Leighton Chen FTE OK so this is internal to the workers and it's specific to supporting App Insights correlation and is not something that's exposed to the customer? If that's the case, I don't have any concerns.

I'm a bit confused by this. From the description and the usage sample, my impression is that this is indeed a feature exposed to the customer. Whether we document/advertise this or not, we are going to expose the register_before_invocation_request and register_after_invocation_request functions, so customers can and will start depending on them. Is this correct?
I'm not saying we shouldn't be doing this. But we need to be aware of this, and also consider consistency with other languages.

@anthonychu

@lzchen Leighton Chen FTE OK so this is internal to the workers and it's specific to supporting App Insights correlation and is not something that's exposed to the customer? If that's the case, I don't have any concerns.

I'm a bit confused by this. From the description and the usage sample, my impression is that this is indeed a feature exposed to the customer. Whether we document/advertise this or not, we are going to expose the register_before_invocation_request and register_after_invocation_request functions, so customers can and will start depending on them. Is this correct?
I'm not saying we shouldn't be doing this. But we need to be aware of this, and also consider consistency with other languages.

@vrdmr

@Hazhzeng/@lzchen - Is there a way to test this end to end? We don't need to block the PR for the E2E, but can be added post this PR. Your thoughts?

An e2e test could be created to light up the scenario of connecting custom dependencies via Opencensus with other services yes. I agree this can be added after this PR.

@vrdmr

@Hazhzeng/@lzchen - Is there a way to test this end to end? We don't need to block the PR for the E2E, but can be added post this PR. Your thoughts?

An e2e test could be created to light up the scenario of connecting custom dependencies via Opencensus with other services yes. I agree this can be added after this PR.

@AnatoliB

Whether we document/advertise this or not, we are going to expose the register_before_invocation_request and register_after_invocation_request functions, so customers can and will start depending on them. Is this correct?

Good question. I'm not sure about the functions worker guidelines for not exposing methods to the user (in Azure monitor, we simply make the methods protected and it implies to users that methods are not exposed in our API). Would this be sufficient enough? It is difficult (especially a language like Python) to completely prevent users from using this.

also consider consistency with other languages.

This feature has already been implemented in nodejs, however the methods are marked as private.

Also, are we going to document this?

If this functionality is not being exposed to customers, do we have to document this?

@AnatoliB

Whether we document/advertise this or not, we are going to expose the register_before_invocation_request and register_after_invocation_request functions, so customers can and will start depending on them. Is this correct?

Good question. I'm not sure about the functions worker guidelines for not exposing methods to the user (in Azure monitor, we simply make the methods protected and it implies to users that methods are not exposed in our API). Would this be sufficient enough? It is difficult (especially a language like Python) to completely prevent users from using this.

also consider consistency with other languages.

This feature has already been implemented in nodejs, however the methods are marked as private.

Also, are we going to document this?

If this functionality is not being exposed to customers, do we have to document this?

@lzchen Please help me understand this, I might be missing some context. I'm still not clear on how it is not exposed to customers. I thought the sample code in the description demonstrated intended usage. So, after this PR, any Python function app can take advantage of this feature by invoking register_before_invocation_request and register_after_invocation_request. Furthermore, preventing the customer from invoking these would defeat the purpose, as your sample in the description will stop working. Is this correct?
In the nodejs implementation you are referring to, registerBeforeInvocationRequest and registerAfterInvocationRequest seem to be public, probably for exactly the same reason.

@lzchen Please help me understand this, I might be missing some context. I'm still not clear on how it is not exposed to customers. I thought the sample code in the description demonstrated intended usage. So, after this PR, any Python function app can take advantage of this feature by invoking register_before_invocation_request and register_after_invocation_request. Furthermore, preventing the customer from invoking these would defeat the purpose, as your sample in the description will stop working. Is this correct?
In the nodejs implementation you are referring to, registerBeforeInvocationRequest and registerAfterInvocationRequest seem to be public, probably for exactly the same reason.

@lzchen I like the feature, I believe it is useful, and the implementation looks good to me. But I keep reading two things that look mutually-exclusive to me:

"This is just a worker implementation detail." In this case, Functions users cannot take (or not take) a dependency on this, no documentation is required, consistency with other languages is irrelevant.
"This is a useful and simple feature." In this case, we should review it from this perspective: consider consistency with other languages, documentation, etc., and make sure we keep supporting this feature moving forward. I'm not saying this PR is not good enough for these purposes, and chances are we'll just take it as is. But we need to know.

I think we need to choose one, it cannot be both :-)

@lzchen I like the feature, I believe it is useful, and the implementation looks good to me. But I keep reading two things that look mutually-exclusive to me:

"This is just a worker implementation detail." In this case, Functions users cannot take (or not take) a dependency on this, no documentation is required, consistency with other languages is irrelevant.
"This is a useful and simple feature." In this case, we should review it from this perspective: consider consistency with other languages, documentation, etc., and make sure we keep supporting this feature moving forward. I'm not saying this PR is not good enough for these purposes, and chances are we'll just take it as is. But we need to know.

I think we need to choose one, it cannot be both :-)

@AnatoliB
Got it. I think we would like to take the (2) approach, in which case I will fix the PR to adhere to the feature guidelines.

@AnatoliB
Got it. I think we would like to take the (2) approach, in which case I will fix the PR to adhere to the feature guidelines.

@anthonychu Since we agreed that this is actually a customer-facing feature, do you want to review it from this perspective?
I have a question on consistency with the Node.js implementation. While comparing this PR with Node.js implementation, I noticed that the callback functions in the Node.js case look like this:
type InvocationRequestBefore = (context: Context, userFn: Function) => Function;
type InvocationRequestAfter = (context: Context) => void;

Does anybody know what is the story behind userFn? In the Python case, only context is passed in both cases. Do we have a reason to follow the same pattern?

@anthonychu Since we agreed that this is actually a customer-facing feature, do you want to review it from this perspective?
I have a question on consistency with the Node.js implementation. While comparing this PR with Node.js implementation, I noticed that the callback functions in the Node.js case look like this:
type InvocationRequestBefore = (context: Context, userFn: Function) => Function;
type InvocationRequestAfter = (context: Context) => void;

Does anybody know what is the story behind userFn? In the Python case, only context is passed in both cases. Do we have a reason to follow the same pattern?

Closing in favor of https://github.com/Azure/azure-functions-python-worker/pull/815


Authorize endpoint status code 301

Hi, I've a question about redirection of /connect/authorize. This endpoint returning HTTP 302 status code but I want to change with 301. Is there any parameter for this?

Why 301?


Remove version: CrystalDewWorld.CrystalDiskInfo.KureiKeiEdition version 8.12.2

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

URL: https://osdn.net/frs/redir.php?m=ipconnect&f=crystaldiskinfo%2F75303%2FCrystalDiskInfo8_12_2KureiKei.exe
Response Error: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
(Automated message - build 923)


Failed syncing database:  ConnectionError [SequelizeConnectionError]: database "torrentio" does not exist

error.txt
Describe the bug
Getting error when running docker compose up -d, not starting the scrape.
To Reproduce
Running
Just started giving this error out of nowhere. What i tried so far
docker kill $(docker ps -aq)
docker rm $(docker ps -aq)
docker rmi -f $(docker images -a -q)
docker system prune
removing the whole folder from server
git clone https://github.com/Gabisonfire/torrentio-scraper-sh.git
(running docker compose up -d en watching the logs of the scraper)
Expected behavior
Starting the scrape:
scraper-1    | Scraper started
scraper-1    | [Fri Feb 02 2024 15:20:51 GMT+0000] starting YTS scrape...
scraper-1    | Scrapping YTS page 1
Logs
See error.txt
Hardware:

OS ubuntu 22.04
Server: OpenVZ with AMD EPYC™ 7443P - 4.00 GHZ

Additional context
Worked perfectly in the morning (and all the other days i had it running) but now its not working anymore.
Cloned https://github.com/kubo6472/torrentio-ghcr and with this built it is working perfectly:
scraper-1    | Scraper started
scraper-1    | [Fri Feb 02 2024 15:20:51 GMT+0000] starting YTS scrape...
scraper-1    | Scrapping YTS page 1
etc etc etc
Thank you already!

Please post your Postgres logs as well

Thanks for responding so fast! Can you tell where to find them? (Sorry :) I am kinda new in this)

type docker compose logs postgres into the terminal when you are inside the folder for the project.

Sounds like you could be missing an env var
ENABLE_SYNC=true on the consumer will create the database if it doesn't exist as long as the connection string is correct, as it validates schema on start

If this is true can we remove this var and assume its always true?

It's a slightly faster startup of the container but I guess it doesn't matter. Just the consumer. Yeah we can remove it

I don't have a env folder. Could that be the problem?

You are on the pre-rewrite version

@purple-emily
Log:
postgres-1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres-1  |
postgres-1  | 2024-02-02 15:19:44.319 UTC [1] LOG:  starting PostgreSQL 14.10 on x86_64-pc-linux-musl, compiled by gcc (Alpine 13.2.1_git20231014) 13.2.1 20231014, 64-bit
postgres-1  | 2024-02-02 15:19:44.319 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
postgres-1  | 2024-02-02 15:19:44.319 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres-1  | 2024-02-02 15:19:44.323 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres-1  | 2024-02-02 15:19:44.329 UTC [21] LOG:  database system was shut down at 2024-02-02 15:05:34 UTC
postgres-1  | 2024-02-02 15:19:44.341 UTC [1] LOG:  database system is ready to accept connections
postgres-1  | 2024-02-02 15:19:46.793 UTC [28] FATAL:  database "torrentio" does not exist
postgres-1  | 2024-02-02 15:19:49.084 UTC [29] FATAL:  database "torrentio" does not exist
postgres-1  | 2024-02-02 15:19:51.402 UTC [30] FATAL:  database "torrentio" does not exist
postgres-1  | 2024-02-02 15:19:53.802 UTC [31] FATAL:  database "torrentio" does not exist
postgres-1  | 2024-02-02 15:19:56.551 UTC [32] FATAL:  database "torrentio" does not exist
postgres-1  | 2024-02-02 15:19:59.929 UTC [33] FATAL:  database "torrentio" does not exist
postgres-1  | 2024-02-02 15:20:05.335 UTC [34] FATAL:  database "torrentio" does not exist
postgres-1  | 2024-02-02 15:20:14.481 UTC [1] LOG:  received fast shutdown request
postgres-1  | 2024-02-02 15:20:14.482 UTC [1] LOG:  aborting any active transactions
postgres-1  | 2024-02-02 15:20:14.488 UTC [1] LOG:  background worker "logical replication launcher" (PID 27) exited with exit code 1
postgres-1  | 2024-02-02 15:20:14.491 UTC [22] LOG:  shutting down
postgres-1  | 2024-02-02 15:20:14.514 UTC [1] LOG:  database system is shut down

If you haven’t used the project at all. Delete the folder and download/clone again

@purple-emily I deleted the folder like twice today, but it didn't change a thing (I even showed my co-worker). I git cloned it again and now i am seeing the env folder XD. Now i know how users feel when they call me and their problem fixes itself while calling me :)
Is it normal that this addon now uses 4GB of ram?

@purple-emily I deleted the folder like twice today, but it didn't change a thing (I even showed my co-worker). I git cloned it again and now i am seeing the env folder XD. Now i know how users feel when they call me and their problem fixes itself while calling me :)
Is it normal that this addon now uses ~4GB of ram

It may use more whilst the producer is filling up the message queue and less in between.
4gb sounds like a lot. I was running 60 consumers and using around 6gb of ram. The producer uses the most ram during operation but it’s throttled to not go crazy and fill the queue up with hundreds of thousands of torrents to process. It should fill up a chunk then wait for them to be processed. Rinse and repeat.

Can you use top or htop and verify what’s using all the ram?

Stopped al the containers and the mem usage is 2.9Gb... I had this problem before. Seems like docker bugging out?

I am restarting my server and then rerunning the docker compose -d with the now working docker ;) Will drop feedback asap. Can i close this bug report, while we can keep chatting?
I also want to say @Gabisonfire @purple-emily @iPromKnight Thank you for helping me out!

Mem usage is now at ~1800MB ~3 min running

That sounds much more like what I’d expect. Classic turn it off and on again

Might be worth updating everything just in case if you’ve had that issue before
sudo apt update && sudo apt upgrade never hurts

btw tried googling the  problem for a couple min but couldn't find something to fix it. So offcourse i wil try to google first ;)

Sounds good, hope everything runs smoothly!

Have a great night / day / weekend!

@thoohtje what version of docker you using?
did you install it with snap/flatpack or from package manager?

To be fair idk, i think through apt-get. After reinstalling for i third time i didn't even install snapd i see :) Is it better to install docker with snap?

No I’d follow the guide on the docker website.
https://docs.docker.com/desktop/install/linux-install/

Completly reinstalled docker from scratch using the docker manual:
docker engine:

Docker desktop:

Reinstall freed up about 5GB space ;0
I hope this solves the mem problem but after the install my mem usage is already at 3.7GB:

Rebooting my server and am going to let it run for a couple hours / night. If i still have mem issues and i can't seem to fix them i will comment back here.

@purple-emily
Sadly i still have memory problems when using docker ;(
Reinstalled the whole server and set everything up from scratch. So only the really needed packages (nano, docker, nginx, ufw, openjdk (for my discord bot, which isn't running) are installed).

i am kinda giving up on this sadly (as for now). To many problems (NOT blaming Gabi for this!!) for something i wanted to use as a cheap backup for torrentio. Last thing i am going to try is to install an older version of docker.
Thank you all for the help.

Downgraded to docker v24 and the memory issue is gone! But got another problem; database seems te reset after ~12 hours. I also tried to clone the newest release, but the consumer containers are restarting every second. logs says:
{"log":"(node:1) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.\n","stream":"stderr","time":"2024-02-04T20:10:20.0638494Z"}
{"log":"{"level":50,"time":1707077420300,"pid":1,"hostname":"78720f3ceb23","msg":"Failed syncing database: "}\n","stream":"stdout","time":"2024-02-04T>
{"log":"/app/dist/index.cjs:199\n","stream":"stderr","time":"2024-02-04T20:10:20.313934329Z"}
I will hopefully retry this project in a couple of months and see if my/the problems still exist.

Can you add these logs to a new issue for someone to look into?

@purple-emily
Thank you for replying, but i got jackett running through elfhosted and using a free version of torrentio / jackett that someone is publicly hosting on elfhosted.
I probs wil try this project out somewhere in the future. But for now im done trying to :)
Thank you again for the help so far.


Annoying sound I can not find a way to turn off.

Mod Version:
DynamicSurroundings-1.11.2-<IP_ADDRESS>.jar
Forge Version:
forge1.11.2-<IP_ADDRESS>8
Description:
Can't find a way to disable an annoying sound that's only played near my base. Already found out it is
the part of biome sounds by turning on the sounds' settings one by one. It sounds like some kind of a bell or something. I've got a little video showing it off. It is driving me completely crazy!
The video. The sound can be heard at 0:07 - 0:10 and 0:34 - 0:37, and also 0:43.

The video link is broke.  Are you near a village or have a villager farm nearby?

Sorry, completely forgot to make the video public. And yes, I do have a villager that I cured and locked down in a room. It might be it.

That is probably the sound, then.  Internally in Minecraft if you have a villager and a valid door configuration you have a Village.

@serkas001 Have you had a chance to block the sound?  Also, I would like to know what you find annoying about the sound - too high pitch, too "clinky", etc.  Would it be better if it were lower in tone?

@serkas001 I watched the video and yes that is the village anvil sound effect.

Yeah, I did find a way to turn it off, not a big deal. It was annoying for me, first of all, because I could not find the source of the sound and it wasn't obvious at all. And also because I think it is too frequent, like 5-6 times a minute is too much.
Thanks! Nevertheless I'm loving the mod, it is really great!

Glad to hear you are liking the mod.  I am looking to change how the anvil sound gets played and the feedback helps.  I am going to close out this issue for now.  If you have any further issues let me know.  Thanks!


[ORCO-26] Fix various TSan issues.

Scope & Purpose
This PR fixes various issues I found while running integration tests locally with a TSan build.

[x] :hankey: Bugfix

Backports:

[x] Backport for 3.9: #15515

Related Information

[x] Jira ticket number: https://arangodb.atlassian.net/browse/ORCO-26

Testing & Verification

[x] The behavior in this PR was manually tested
[x] This change is already covered by existing tests
[x] I ensured this code runs with TSan

https://jenkins.arangodb.biz:3456/job/arangodb-ANY-linux-san.x86-64/952/

@MBkkt Cursor::_isUsed and Cursor::_expires are purely informative and not used for any kind synchronization, that's why all operations can be relaxed. The calls to use() are either performed on a freshly created cursor which is not shared yet, or protected by a mutex.

https://jenkins.arangodb.biz:3456/job/arangodb-ANY-linux-san.x86-64/955/

https://jenkins.arangodb.biz:3456/job/arangodb-ANY-linux-san.x86-64/974/

@mpoeter needs new backport


stop for loop

I think in Bourne shell the loop for i in {1..10} is not working. At least for me it only goes through once and causes the stop to not wait long enough. Other scripts I looked at tend to use other type of loops.

Same problem here. It's got #!/bin/sh at the top of the script but init v or whatever is running it using bash. Any idea what type of loop we should use instead?

I replaced line 47 to line 57 with:
kill `get_pid`
i=0
while is_running; do
  if [ $i = '60' ]; then
    break;
  else
    echo -n "."
    i=$(($i+1))
    sleep 1
  fi
done
echo

Also I raised the max waiting time to 60 seconds

hehe, I did something similar

(and then I call kill -9 if it still doesn't die)

It is Bourne shell, so I used for i in 1 2 3 4 5 6 7 8 9 instead of for i in {1..10}
http://steve-parker.org/sh/loops.shtml


UI issues

https://docs.google.com/document/d/190KWDWjcVqVXuA217XRDsvPJ3Vfo_4EnX3UYtsHi3Zs/edit

Fixed.


Blank when edit file from terminal

VSCode Version: Code 1.3.1 (e6b4afa53e9c0f54edef1673de9001e9f0f547ae, 2016-07-12T13:28:46.761Z)
OS Version: Linux x64 4.4.0-28-generic
running in virtual box
-- host pc windows 10

Steps to Reproduce:

$ code /some/path/to/a.sh
code launches showing either black background (first time) or same content as desktop (second time)

seems like this was caused by "Enable 3D Acceleration"
after turning off that option this seems to be resolved

Yes this is due to limitations in Chromium.


05 how to create configuration page 05

create a configuration page article.

Docs Build status updates of commit 6ec6b5e:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/tabs/how-to/create-tab-pages/configuration-page.md
:warning:Warning

Details

msteams-platform/includes/prepare-environment.md
:warning:Warning

Details

msteams-platform/tabs/how-to/create-tab-pages/configuration-page.md

Line 15: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 15: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 84: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.settings.settings?view=msteams-client-js-latest.md).
Line 89: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 95: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.context?view=msteams-client-js-latest.md).
Line 99: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 139: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.context?view=msteams-client-js-latest.md).
Line 158: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 158: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 162: [Warning] Invalid file link:(~/platform/tabs/how-t-/create-tab-pages/removal-page.md).
Line 166: [Warning] Invalid file link:(~/resources/design/framework/tabs-mobile.md).
Line 184: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 185: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 189: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 190: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 191: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 192: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).

msteams-platform/includes/prepare-environment.md

Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit c20c2c7:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/tabs/how-to/create-tab-pages/configuration-page.md
:warning:Warning

Details

msteams-platform/includes/prepare-environment.md
:warning:Warning

Details

msteams-platform/tabs/how-to/create-tab-pages/configuration-page.md

Line 15: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 15: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 84: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.settings.settings?view=msteams-client-js-latest.md).
Line 89: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 95: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.context?view=msteams-client-js-latest.md).
Line 99: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 139: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.context?view=msteams-client-js-latest.md).
Line 158: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 158: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 163: [Warning] Invalid file link:(~/platform/tabs/how-t-/create-tab-pages/removal-page.md).
Line 167: [Warning] Invalid file link:(~/resources/design/framework/tabs-mobile.md).
Line 185: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 186: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 190: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 191: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 192: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 193: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).

msteams-platform/includes/prepare-environment.md

Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit d0be372:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/tabs/how-to/create-tab-pages/configuration-page.md
:warning:Warning

Details

msteams-platform/includes/prepare-environment.md
:warning:Warning

Details

msteams-platform/tabs/how-to/create-tab-pages/configuration-page.md

Line 15: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 15: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 84: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.settings.settings?view=msteams-client-js-latest.md).
Line 89: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 95: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.context?view=msteams-client-js-latest.md).
Line 99: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 139: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.context?view=msteams-client-js-latest.md).
Line 158: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 158: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 163: [Warning] Invalid file link:(~/platform/tabs/how-t-/create-tab-pages/removal-page.md).
Line 167: [Warning] Invalid file link:(~/resources/design/framework/tabs-mobile.md).
Line 185: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 186: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 190: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 191: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 192: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 193: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).

msteams-platform/includes/prepare-environment.md

Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit c77e676:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/tabs/how-to/create-tab-pages/configuration-page.md
:warning:Warning

Details

msteams-platform/includes/prepare-environment.md
:warning:Warning

Details

msteams-platform/tabs/how-to/create-tab-pages/configuration-page.md

Line 23: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 23: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 92: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.settings.settings?view=msteams-client-js-latest.md).
Line 97: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 103: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.context?view=msteams-client-js-latest.md).
Line 107: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 147: [Warning] Invalid file link:(~/javascript/api/@microsoft/teams-js/microsoftteams.context?view=msteams-client-js-latest.md).
Line 174: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 174: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 179: [Warning] Invalid file link:(~/platform/tabs/how-t-/create-tab-pages/removal-page.md).
Line 183: [Warning] Invalid file link:(~/resources/design/framework/tabs-mobile.md).
Line 201: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 202: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 206: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 207: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 208: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).
Line 209: [Warning] Invalid file link:(~/tabs/how-to/create-tab-pages/foo.md).

msteams-platform/includes/prepare-environment.md

Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).
Line 3: [Warning] Invalid file link:(~/concepts/build-and-test-your-app/foo.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Improve storing notebook.

This PR relates to: https://issues.apache.org/jira/browse/ZEPPELIN-454
Storing the notebook process changed. (store".note.json" temporary file first and if  it is succeed ".note.json" move to "note.json")

LGTM

Shall not we have an error handling in case if .note.json -> note.json fails? I.e next time when zeppelin starts and have .note.json it could recover privious state, what do you think?
I'm not sure if I understand so could you let me know if this change also impact sync between notebook repos?

LGTM, @bzz it doesn't affect sync between repos

@khalidhuseynov it's good to know, but could you add a couple of words explaining why is that? :)
Nobody thinks that we also should have a recovery from .note.json state? Sorry if I'm missing something obvious here.

To recover from .note.json, it should validate .note.json first, while it's possible to have broken json string.

sure @bzz so it doesn't affect sync since it just changes the way note.json saves to local fs (since in VFSNotebookRepo), making note.json either valid or non-saved (excluding the case of broken note.json). So in case of exception with local saving .note.json,  the real note.json isn't saved as expected and exception is thrown.

I think it adds value (prevent corruption of note.json when saving).
I think restoring .note.json (when possible) could be a future work.
Looks good to me.

Sounds good

Thanks, then i'm merging it into the master.


MoleculeTurbine emits duplicate values

MoleculeTurbine can emit duplicate output values, which results in tests that assert non-stateful behavior.

Is the use case here that the backing state changes which triggers recomposition, but that state doesn't affect the logic so it re-emits the previous value?

Right: the backing state changes, but the resulting ViewModel is identical.

I think I need an example of when this is undesirable and what API/behavior changes you'd like to see?
MoleculeTurbine will only attempt to recompose if you await an event, so it will only emit the duplicate value if you ask it to. If the goal is to not have to assert against state changes that don't affect the value, then you can remove the await.
If we de-dupe the value behind the scenes, then you lose the ability to assert because the only way to assert a lack of value is to await the timeout. Then your tests will be slow. We could add an awaitUnchanged(), but if that's what you're after then with the current APIs you can hold onto the previous value and compare it to the next one from awaitItem().

MoleculeTurbine will only attempt to recompose if you await an event, so it will only emit the duplicate value if you ask it to. If the goal is to not have to assert against state changes that don't affect the value, then you can remove the await.

The big question for me is — what am I awaiting on? Am I awaiting on a StateFlow? As a consumer of the API, that's what I expect: Molecule is a way of defining a StateFlow.
Or am I triggering a recomposition? If that's the case, then I should always get a fresh value, regardless of whether the internal state has been updated or not.
The current state of affairs is that I get a new output every time the internal state changes. This might make sense when writing validations for Molecule itself, because those internal details are important to validate, but as a consumer of Molecule it leads to engineers writing meaningless validations: "This test failed because it emitted 5 LoanViewModel(loanAmount = 5)s instead of 4." That won't make a difference in prod, so I don't understand why it should be in the test.
It might even lead engineers to believe that Molecule behaves the way the tests behave. Which is concerning, because.... well, it should behave the way the tests behave, right?

If we de-dupe the value behind the scenes, then you lose the ability to assert because the only way to assert a lack of value is to await the timeout.

My perspective is that this is a fact of life in testing: you can't assert on things that aren't visible. If you want to assert on them.. make them visible.

If that's the case, then I should always get a fresh value, regardless of whether the internal state has been updated or not.

Not sure if this is possible! I'll have to take a closer look at the APIs. We're currently at the whim of Compose for how it determines that recomposition is needed.

The current state of affairs is that I get a new output every time the internal state changes. [...] "This test failed because it emitted 5 LoanViewModel(loanAmount = 5)s instead of 4."

This isn't entirely true, though. You can change as much state as you want between awaits and you'll only get one value out when you awaitItem().
val events = MutableSharedFlow<Unit>()

testMolecule({
  var count by remember { mutableStateOf(0) }

  LaunchedEffect(Unit) {
    events.collect {
      count++
    }
  }

  count
}) {
  assertEquals(0, awaitItem())

  events.emit(Unit)
  events.emit(Unit)
  events.emit(Unit)
  events.emit(Unit)
  events.emit(Unit)

  assertEquals(5, awaitItem())

  assertFailsWith<TimeoutCancellationException> { awaitItem() }
}

However, yes, if you choose you can also write overly verbose tests:
  assertEquals(0, awaitItem())

  events.emit(Unit)
  assertEquals(1, awaitItem())

  events.emit(Unit)
  assertEquals(2, awaitItem())

  events.emit(Unit)
  assertEquals(3, awaitItem())

  events.emit(Unit)
  assertEquals(4, awaitItem())

  events.emit(Unit)
  assertEquals(5, awaitItem())

So the granularity at which MoleculeTurbine emits items is entirely up to the test writer.

It might even lead engineers to believe that Molecule behaves the way the tests behave. Which is concerning, because.... well, it should behave the way the tests behave, right?

The thing about this is that we can't reliably match production because production behavior is entirely dependent on the frame clock and the timing at which state gets updated. If two states are being updated from two different coroutines, then in production, a particular recomposition could either see one or both of the updates depending on where the frame sync landed. If we try to match production for testing, then that will likely lead to flaky tests.

So the granularity at which MoleculeTurbine emits items is entirely up to the test writer.

My example was wrong. But even then — why should the test author care about whether the internal state was updated or not? This does not match the StateFlow API, which implicitly deduplicates.

The thing about this is that we can't reliably match production because production behavior is entirely dependent on the frame clock and the timing at which state gets updated. If two states are being updated from two different coroutines, then in production, a particular recomposition could either see one or both of the updates depending on where the frame sync landed. If we try to match production for testing, then that will likely lead to flaky tests.

But from the perspective of production code, the value is a state consumed in a pull fashion, right? (I'm really winging it, so... I'm sure I'm whiffing the implementation details a second time.) It can receive the same value 5 times in 25ms, and it means the same as if it had received the value one time. They're states, and so the meaning of the flow is in the distinct values in the stream. If a production implementation were to behave differently upon receiving 2 events versus receiving 1 event, it would be an error.
There's another point above I want to revisit:

If we de-dupe the value behind the scenes, then you lose the ability to assert because the only way to assert a lack of value is to await the timeout.

The benefit of this is narrow. The code under test will only emit a new value if the interaction under test triggers a write to the composed state. This often won't happen: a lot of event handling code doesn't touch state, for example, and the TDD flow of "let's write a failing test" never will. So this ends up adding another ineffective half measure to the pile of incomplete coroutines testing solutions.

Molecule is a way of defining a StateFlow.

Molecule has a way of defining a StateFlow. It also has a way of defining a Flow and we'll probably expose State at some point. We also originally had modules for defining Observable.

Molecule has a way of defining a StateFlow. It also has a way of defining a Flow and we'll probably expose State at some point. We also originally had modules for defining Observable.

That is really really awesome. I hate testing StateFlow, but I usually want StateFlow in prod.
The StateFlow is just a transformation on what a molecule implementation produces, though. That's at the heart of my issue here: Is the sequence of values produced by a molecule implementation stateful? Or is it eventful? Because if it's stateful (which I assume that it is), the current behavior doesn't appear to make much sense to me, and I'd find it confusing to write tests against.

No longer relevant


Currently don't see options for turning off logging output in azure table execute batch.

Where do you suppress logging? I looked in the source and thought if the log crate is used I'd set logging to release_max_error in compiler dependency config, but that doesn't seem to help. If someone could point in the right direction that would be nice. Thank you.

Hello, appreciate the response; however, setting the var doesn't work I double checked debug vs release as well and still get output.  I even tried to use gag on std out and that also doesn't work, but gives me some more insight. I'll dig around when I get back to it see if I can't figure out something.


Link errors in Navbar

The Contributors and Contact from Navbar is not working.

@susanhsrestha ,
It's WIP.
In the next commit it will be updated.
But thanks for mentioning the issue.
👍🏽


ssdt-vs2017-15.9.10

@WendyYes : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#label:"aq-pr-triaged"
@dzsquared - assuming this content is approved, it should ideally be added via the private repo (sql-docs-pr).

this content went through the correct repo (sql-docs-pr) and can be closed
#please-close


Download in TF1.fr / WAT.tv is no more available

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.01.16
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

$ sudo youtube-dl -U
[sudo] Mot de passe de me :
youtube-dl is up-to-date (2021.01.16)
$ youtube-dl -F https://www.tf1.fr/tf1/barbapapa/videos/barbapapa-lempire-orange-18906322.html
[TF1] barbapapa-lempire-orange-18906322: Downloading webpage
[wat.tv] 13762382: Downloading JSON metadata
[wat.tv] 13762382: Downloading JSON metadata
[wat.tv] 13762382: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[wat.tv] 13762382: Downloading f4m manifest
WARNING: Unable to download f4m manifest: HTTP Error 403: Forbidden
[wat.tv] 13762382: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[wat.tv] 13762382: Downloading f4m manifest
WARNING: Unable to download f4m manifest: HTTP Error 403: Forbidden
[wat.tv] 13762382: Downloading MPD manifest
WARNING: Failed to download MPD manifest: <urlopen error [Errno -2] Name or service not known>
[wat.tv] 13762382: Downloading MPD manifest
WARNING: Failed to download MPD manifest: <urlopen error [Errno -2] Name or service not known>
[wat.tv] 13762382: Checking http-264 URL
[wat.tv] 13762382: http-264 URL is invalid, skipping: <urlopen error [Errno -2] Name or service not known>
[wat.tv] 13762382: Checking http-464 URL
[wat.tv] 13762382: http-464 URL is invalid, skipping: <urlopen error [Errno -2] Name or service not known>
[wat.tv] 13762382: Checking http-664 URL
[wat.tv] 13762382: http-664 URL is invalid, skipping: <urlopen error [Errno -2] Name or service not known>
[wat.tv] 13762382: Checking http-1264 URL
[wat.tv] 13762382: http-1264 URL is invalid, skipping: <urlopen error [Errno -2] Name or service not known>
[wat.tv] 13762382: Checking http-1864 URL
[wat.tv] 13762382: http-1864 URL is invalid, skipping: <urlopen error [Errno -2] Name or service not known>
[wat.tv] 13762382: Checking http-2564 URL
[wat.tv] 13762382: http-2564 URL is invalid, skipping: <urlopen error [Errno -2] Name or service not known>
ERROR: No video formats found!; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Description

youtube-dl worked fine to download videos on TF1.fr (french TV channel), until january 21st.
Since this date, download is no more possible, with the logs posted before.
I didn't find any m3u8 file in the F12 tool > Network tab.

is an account needed in order to watch the videos ??

is an account needed in order to watch the videos ??

Yes an account in needed. I can passe my logins, how to do this in private?

Yes an account in needed. I can passe my logins, how to do this in private?

Sorry I didn't understand why it's closed, and fixed ? I have searched but didn't find the same issue, so duplicate of what ?

Sorry I didn't understand why it's closed, and fixed ? I have searched but didn't find the same issue, so duplicate of what ?


cancel "Merge split views" leaves the purple boxes in the viewer

When pressing Cancel in the Merge Split Views function, the purple boxes stay visible in the viewer.

Forgot the screenshot :)

Thanks! This should be fixed now!


Not working [Bug: Current]

Describe the bug
I changed location but not work skin
Screenshots

Windows 11 Latest version

Strange. Mine is loop normal.

It is happening to me too in windows 10


Don't fail on unsupported tier > 1 records in populate

Not all records have short_answers so we need to avoid the KeyError on those that don't. This also adds a test case from the problematic record provided in https://github.com/octodns/octodns-ns1/issues/17 and fixes some ~bugs in the existing test code where answers weren't what they were expected to be (likely copy-n-paste errors.)
/cc https://github.com/octodns/octodns-ns1/issues/17 @flz

A "safe" change, avoids an exception on the (rare) case there's no short_answers in the record data.


MHA not working -  Access to ODATA is disabled

First and foremost this is a great product.  Thank you for working tirelessly on it.
We are using O365 Gov cloud and I know REST has been disabled there.  Also we do EWS filtering and our allow list consist of the following agents.
-EwsAllowList "Outlook-Android/","Outlook-iOS/","ExchangeService
sClient/","outlook","MAPI","skype","Microsoft+Office/","UCCAPI/","OC/","OCPhone/","Microsoft Office/","MacOu
tlook/*"
Could either of these issues be the cause of why MHA is not working for us.   Any ideas?
We get the following when we click the MHA icon on an email within Outlook.
textStatus: error errorThrown: State: rejected jqXHR: { "readyState": 4, "responseText": "{"error":{"code":"ErrorAccessDenied","message":"Access to OData is disabled."}}", "responseJSON": { "error": { "code": "ErrorAccessDenied", "message": "Access to OData is disabled." } }, "status": 403, "statusText": "error" }

This might be the same issue as #38, which I now think was caused by moving the rest/ews calls into a frame. I'm working on pulling them back out and leaving the frames to just to rendering, which should fix #38, and then we'll see if it fixes this.
BTW - this didn't happen to be working at the start of the year did it?

Stephen,
We found the solution.  If you are doing EWS filtering as we are you need to include "Mozilla/*" in the list.  I basically kicked off fiddler and attempted to use MHA and noticed that the user agent was "Mozilla/...".  After adding that by performing the following powershell statement it all started working.  Thanks for your help.
set-organizationconfig -ewsallowlist "Outlook-Android/","Outlook-iOS/","ExchangeServicesClient
/","outlook","MAPI","skype","Microsoft+Office/","UCCAPI/","OC/","OCPhone/","Microsoft Office/","MacOutlook/",**"Mozilla/"**

Good find - I'm reporting this to the devs - this is non-obvious...

Hey - can you give me more details on the frame that failed for you? Perhaps send me the fiddle? You can DM me @ my twitter: https://twitter.com/sgriffin


Pop the free ID when assigning to a process

Fixes #2128.

Yes, I believe this has been causing us some trouble as well. Thanks for tracking it down. I believe the build is failing because it needs to have gofmt run on the changed file.

@captncraig I did this quickly just using the github website (seems like I left whitespace on the empty lines). I don't have any Go toolchain setup locally.
Would you be able to take it from here? And check the windows and osx implementations don't have the same issue? Otherwise I may have some time next week to set things up.

@captncraig Managed to make Travis green. Is this ok to merge?

Yeah, let's do it.


NO-ISSUE: Change the name of DMN legacy editor to DMN classic editor

@LightGuard

fine for me, however, it is slightly unexpected, maybe it is just me, that sometimes we use Classic and sometimes we use classic.

I think it depends on the context.

Closed because the PR needs to target to another branch, not this one, as discussed in private chat....


Type mapping unexpectedly leads to missing keys

Bug Report

🔎 Search Terms

mapped types missing keys
🕗 Version & Regression Information

I was unable to test this on prior versions because we didn't use the --exactOptionalPropertyTypes previously

⏯ Playground Link

Playground link with relevant code
💻 Code

type Flatten<T> = { [K in keyof T]: T[K] };

type OptionalKeys<T> = { [K in keyof T]: {} extends Pick<T, K> ? K : never }[keyof T];
type OptionalProperties<T> = Pick<T, OptionalKeys<T>>;

type UndefinedPropertyKeys<T> = { [K in keyof T]: undefined extends T[K] ? K : never }[keyof T];
type DefinedPropertyKeys<T> = { [K in keyof T]: undefined extends T[K] ? never : K }[keyof T];

type UndefinedToOptional<T> = Flatten<{
    [K in UndefinedPropertyKeys<T>]+?: T[K]
} & {
    [K in DefinedPropertyKeys<T>]: T[K]
}>;

type Convert<T> = Flatten<{ [K in keyof OptionalProperties<T>]-?: T[K] }>;

type input = {
    remove: number | undefined;
    makeMandatory?: number;
    retain?: number | undefined;
}

//Equals Convert<input>
type intermediateInput = {
    makeMandatory: number;
    retain: number | undefined;
}

//Type is { makeMandatory: number; retain?: number | undefined }
const x: UndefinedToOptional<intermediateInput> = {
    makeMandatory: 0,
};

//Type is { }
const y: UndefinedToOptional<Convert<input>> = {
    makeMandatory: 0,
};

🙁 Actual behavior
The keys of the mapped type are stripped when applying another type mapping.
🙂 Expected behavior
The result of mapping a mapped type should show the same behaviour as mapping a recreation of the mapped type.

//Type is { }
const y: UndefinedToOptional<Convert<T>> = {
    makeMandatory: 0,
};

What type is T? It seems you forgot something. The playground code and the code in this issue differ.

I've updated the TS playground link and example code

This is likely a difficult bug but we could use some help on a shorter repro

The problem seems to resolve itself when converting
type OptionalKeys<T> = { [K in keyof T]: {} extends Pick<T, K> ? K : never }[keyof T];
type OptionalProperties<T> = Pick<T, OptionalKeys<T>>;

to
type OptionalProperties<T> = { [K in keyof T]: {} extends Pick<T, K> ? K : never };

which might help in finding a solution to this bug

This is likely a difficult bug but we could use some help on a shorter repro

It can be reduced to
type Flatten<T> = { [K in keyof T]: T[K] };

type OptionalKeys<T> = { [K in keyof T]: {} extends Pick<T, K> ? K : never }[keyof T];
type OptionalProperties<T> = Pick<T, OptionalKeys<T>>;

type UndefinedPropertyKeys<T> = { [K in keyof T]: undefined extends T[K] ? K : never }[keyof T];

type UndefinedToOptional<T> = Flatten<{
    [K in UndefinedPropertyKeys<T>]+?: T[K]
}>;

type Convert<T> = Flatten<{ [K in keyof OptionalProperties<T>]-?: T[K] }>;

type input = {
    retain?: number | undefined;
}

//Equals Convert<input>
type intermediateInput = {
    retain: number | undefined;
}

//Type is { retain?: number | undefined }
const x: UndefinedToOptional<intermediateInput> = { };

//Unexpected: Type is { }
const y: UndefinedToOptional<Convert<input>> = { };

The Flatten does help to show what the resulting type is using the tooltip. This comes in handy, since there are no errors to indicate that there is a problem.

since there are no errors to indicate that there is a problem.

That's why I used type Test2 = UndefinedToOptional<Convert<input>>["retain"] instead. It shows that the resulting type currently has no such property.

@RyanCavanaugh Yeah this seems very tricky, I don't know how to fix it but after playing with the example I think I've at least managed to trim it down so that the root problem is more clear:
type Original = {p1?: string}

// seems equivalent to keyof but isn't under strictNullChecks because `undefined` will be included if T has any optional properties
type CustomKeyOf<T> = { [K in keyof T]: K }[keyof T]

type CorrectlyErrors = Pick<Original, CustomKeyOf<Original>> // Type '"p1" | undefined' does not satisfy the constraint '"p1"'

// This should error like CorrectlyErrors, but making it generic currently allows it to slip by
// I believe this is ultimately the root problem because it results in `undefined` being considered a key below
type Identity<T> = Pick<T, CustomKeyOf<T>>

// appears to be the same type as Original when hovering but isn't
type Bugged = Identity<Original>

type UnexpectedError = Bugged[keyof Bugged] // Type 'undefined' cannot be used as an index type.

type KeyofOriginal = keyof Original // "p1" as expected
type KeyofBugged = keyof Bugged // "p1" | undefined, which should never happen

// running Bugged through Identity again results in a {} type this time, which is what the OP ran into
type Bugged2 = Identity<Bugged>
type Missing = Bugged2["p1"]  // Property 'p1' does not exist on type 'Bugged2'

Playground Link


Sniper not attempting snipe

I ran into a bug today, where the MCsniperPY rewrite would just not attempt a snipe. I believe this is caused by a bug in the calculation of droptime - 20 seconds here that's triggered when a name is dropping in > 20 seconds. I could be wrong. I'll try to debug it later today.

I'm nearly positive this has been fixed now. I worked on it the day I submitted and, although it hasn't been completely tested, I believe it no longer will occur.


WSL freezing on startup with networkingMode=mirrored after installing <IP_ADDRESS> even after downgrade

Windows Version
Microsoft Windows [Version 10.0.22631.3007]
WSL Version
<IP_ADDRESS> - <IP_ADDRESS>
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
<IP_ADDRESS>-1
Distro Version
Debian 12
Other Software
No response
Repro Steps
Set the following in .wslconfig
[experimental]
networkingMode=mirrored

Then update to version <IP_ADDRESS> using wsl --update --pre-release
Expected Behavior
WSL should work.
Actual Behavior
WSL freezes seemingly forever during startup of a distribution, I can't even list distributions using wsl --list when it's frozen.
I have uninstalled the updates and see the same problem with <IP_ADDRESS> and <IP_ADDRESS> even though it worked fine before installing <IP_ADDRESS>.
Diagnostic Logs
No response

vim .wslconfig
networking=NAT

Please collect traces : https://github.com/microsoft/WSL/blob/master/diagnostics/collect-networking-logs.ps1

@keith-horton I did my best, the script runs wsl.exe several times which freezes each time so I had to forcefully terminate the WSL service after a while each time it happened to allow the script to continue. So I don't know if these logs are useful but I uploaded them for you to Azure Blob Storage so I can delete them later.
https://blob.steamcore.se/github/WslNetworkingLogs-2024-01-17_23-49-23.zip

Thanks! There are 2 different issues happening.

vSwitch is in a bad state and is failing to create new endpoints to mirror networks into the WSL container.
eventually our WSL service crashes.

Could you follow the instructions here https://github.com/Microsoft/WSL/blob/master/CONTRIBUTING.md#11-reporting-a-wsl-process-crash to capture the memory dmp of WSL crashing?
Thanks!

@keith-horton It doesn't crash, it becomes unresponsive. I collected this dump shortly after it seems to halt, I hope it helps.
https://blob.steamcore.se/github/WslLogs-2024-01-18_22-44-58.zip

Hi there. in the previous traces submitted we could clearly see at some stage our service no longer responding to calls - and eventually crash and completely restart.

I killed the process tree using Process Explorer to get the script moving along, isn't that what you were seeing?

I just tried v2.1.1 and while it didn't solve the issue I noticed that mirrored mode worked once at first after a fresh boot but if I shut down WSL using wsl --shutdown and then try to open a new session it is back to being frozen again.

Thanks for sticking with this David.
Can you run the script https://github.com/microsoft/WSL/blob/master/diagnostics/collect-networking-logs.ps1, then stop it once you have a repro?
Just so it's clear: when you say it's frozen, does that mean:

the WSL Linux shell goes away?
or it stays but doesn't respond to keyboard input?
or it responds to keyboard input but has no networking?
does "wsl --version" or "wsl --status" complete, or fail with an error?

(this time please don't kill any processes)
if running "wsl" commands hangs or fails, can you get a process dmp from Task Manager? Right-click on wslservice.exe and click "Create memory dump file".
Thanks!

@keith-horton I'm not sure what you mean by stop it, that script doesn't work without killing the processes because it can't even collect the before data.
I'm talking about this line here:
https://github.com/microsoft/WSL/blob/488082b8bdbf4adb5707326eda882c7bfff2e541/diagnostics/collect-networking-logs.ps1#L28
I left it running for 10 minutes and the log file linux_network_configuration_before.log has 0 size.
the WSL Linux shell goes away?
Windows Terminal seems to time out waiting for a shell if I try to open a new tab.
If I run wsl.exe in an existing tab it stays frozen forever, no prompt appears.
or it stays but doesn't respond to keyboard input?
or it responds to keyboard input but has no networking?
No, it doesn't even start.
does "wsl --version" or "wsl --status" complete, or fail with an error?
wsl --version works
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-2
WSLg version: 1.0.60
MSRDC version: 1.2.5105
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22631.3085

But wsl --status never completes, the prompt never returns (unless I press ctrl+c), nothing happens.
Here is a full dump file of wslservice.exe when it's frozen.
https://blob.steamcore.se/github/wslservice-dmp.zip

Thanks... this makes sense now. Yeah, the script is stuck trying to make a call through wsl.exe, but since the service is hung, that is also blocked. We're trying to see what is going on within Linux as to why our init instance doesn't seem to run. Can you take a manual trace?
It uses the wpr profile from the same WSL/diagnostics.
wpr -start wsl_networking.wprp -filemode

wpr -stop .\wsl_trace.etl
Thanks David. We cannot repro this, and have not heard this from others.
Can you also share what distro you have installed, and any other software you have installed in the Linux partition?
Thanks!

Interesting, I didn't even consider my distribution being a problem since wsl --status isn't even working.
Your question made me try some stuff, I installed a new blank Ubuntu distro and that one seems to work fine. Then when I was listing my software that's installed I realized I have a network mount in /etc/fstab (ceph-fuse) that's mounted using x-systemd.automount, so I tried disabling that and now all of a sudden my distro starts but now I have no network device! Just lo and loopback0. I tried running ifup eth0 but it just tells me "unknown interface eth0".
So it seems that my WSL freezes because systemd locks up because I don't have a network device when running in mirrored more, are there some other steps I can take to debug this now that we have gotten this far?
Here's a trace when it froze
https://blob.steamcore.se/github/wsl_trace-frozen.zip
And here's a trace when it started but I have no eth0
https://blob.steamcore.se/github/wsl_trace-no-eth0.zip

Sorry for the spamming but I just double checked and the fresh Ubuntu install doesn't have working networking either, it has both an eth0 and an eth1 but neither has an IP-adress.

I just stumbled across issue #11263 so I tried removing the internal Hyper-V switch that I had configured but wasn't even using and suddenly mirrored seems to be working again.

@DavidZidar , thanks for the follow-up - glad it's working now!
Sorry - I don't know what was going so badly deep in vswitch

@keith-horton It's complicated stuff I can imagine, thanks for investigating anyway! Unless you would like me to keep this issue open I'll close it in favor of the other one.

Sure, feel free to close this one.

I just stumbled across issue #11263 so I tried removing the internal Hyper-V switch that I had configured but wasn't even using and suddenly mirrored seems to be working again.

Exactly the issue I was running into. I had a blank test vSwitch and forgot to remove this after I was done tinkering.


SDK SpeechStartDetected not being called on Unity 2021.3.3f1

Describe the bug
Using the speechrecognizer sample application works correctly on the version of Unity the sample was published at 2018.3.14f1.
When i upgrade a copy of the sample project to Unity 2021.3.3f1 I noticed that SpeechStartDetected is never called and no successful voice recording seem to be made. There are no errors logged in the Unity console.
To Reproduce
Steps to reproduce the behavior:

Setup the speechRecognizer app in 2018.3.14f1 and confirm correct operation
Make a copy of the project and upgrade it to 2021.3.3f1
Open and run the upgraded project. Observe issue, Application does not function any more.

Expected behavior
I expect the sample application to continue to work correctly.
Version of the Cognitive Services Speech SDK
Microsoft.CognitiveServices.Speech.1.22.0 - Unity package
Sample code latest from github on main -https://github.com/Azure-Samples/cognitive-services-speech-sdk/commit/96cb39806823a649942c8c0cb481254f78b0cefd
Platform, Operating System, and Programming Language

OS: Windows 10
Hardware - x64
Programming language: C# and Unity
Browser: None

Additional context

Error messages, stack trace, sample code to reproduce the issue,

Use Unity spechrecognizer sample code from repo - https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/samples/unity/speechrecognizer

Please provide full Speech SDK Log,
LuisAiNotWorking.txt
LuisAiWorking.txt

Please  let me know if you need any Unity logs.

Hi, your logs show different microphone devices in working vs. non-working cases, was that intentional?
LuisAiWorking.txt : "Microphone (HD Pro Webcam C920)"
[440145]: 78ms SPX_DBG_TRACE_VERBOSE:  named_properties.h:468 ISpxNamedProperties::GetStringValue: this=0x0000000017FFA6D0; name='SPEECH-MicrophoneNiceName'; value='Microphone (HD Pro Webcam C920)'

LuisAiNotWorking.txt : "Headset Microphone (Oculus Virtual Audio Device)"
[148361]: 1573134ms SPX_DBG_TRACE_VERBOSE:  named_properties.h:468 ISpxNamedProperties::GetStringValue: this=0x0000014DF779EEA0; name='SPEECH-MicrophoneNiceName'; value='Headset Microphone (Oculus Virtual Audio Device)'

In the latter case (with "Headset Microphone (Oculus Virtual Audio Device)") it seems the input audio contains only silence thus no speech/intent recognized, just a silence timeout at regular intervals (every 5 seconds of input).
[148361]: 1578488ms SPX_DBG_TRACE_VERBOSE:  usp_reco_engine_adapter.cpp:1695 Response: Speech.Phrase message. Status: 2, Text: , starts at 0, with duration 50400000 (100ns).
[148361]: 1578488ms SPX_DBG_TRACE_VERBOSE:  named_properties.h:468 ISpxPropertyBagImpl::SetStringValue: this=0x0000014D70003730; name='RESULT-Json'; value='{"Id":"ff2922b7274d4105807152002ca7048f","RecognitionStatus":"InitialSilenceTimeout","Offset":0,"Duration":50400000}'
[148361]: 1578488ms SPX_DBG_TRACE_VERBOSE:  lu_engine_adapter.cpp:250 CSpxLUEngineAdapter::ProcessResult: text=''

[148361]: 1583503ms SPX_DBG_TRACE_VERBOSE:  usp_reco_engine_adapter.cpp:1695 Response: Speech.Phrase message. Status: 2, Text: , starts at 50400000, with duration 50400000 (100ns).
[148361]: 1583503ms SPX_DBG_TRACE_VERBOSE:  named_properties.h:468 ISpxPropertyBagImpl::SetStringValue: this=0x0000014D70003730; name='RESULT-Json'; value='{"Id":"d0ceb14719ef4aa091bf6ead0c8b9bd5","RecognitionStatus":"InitialSilenceTimeout","Offset":50400000,"Duration":50400000}'
[148361]: 1583503ms SPX_DBG_TRACE_VERBOSE:  lu_engine_adapter.cpp:250 CSpxLUEngineAdapter::ProcessResult: text=''

[148361]: 1588596ms SPX_DBG_TRACE_VERBOSE:  usp_reco_engine_adapter.cpp:1695 Response: Speech.Phrase message. Status: 2, Text: , starts at 100800000, with duration 50400000 (100ns).
[148361]: 1588596ms SPX_DBG_TRACE_VERBOSE:  named_properties.h:468 ISpxPropertyBagImpl::SetStringValue: this=0x0000014D70003730; name='RESULT-Json'; value='{"Id":"a2873d4a9636461d9d47550d06ffc721","RecognitionStatus":"InitialSilenceTimeout","Offset":100800000,"Duration":50400000}'
[148361]: 1588596ms SPX_DBG_TRACE_VERBOSE:  lu_engine_adapter.cpp:250 CSpxLUEngineAdapter::ProcessResult: text=''

[148361]: 1593589ms SPX_DBG_TRACE_VERBOSE:  usp_reco_engine_adapter.cpp:1695 Response: Speech.Phrase message. Status: 2, Text: , starts at 151200000, with duration 50400000 (100ns).
[148361]: 1593590ms SPX_DBG_TRACE_VERBOSE:  named_properties.h:468 ISpxPropertyBagImpl::SetStringValue: this=0x0000014D70003730; name='RESULT-Json'; value='{"Id":"4d8879ec5cf743fcb037732881ddf231","RecognitionStatus":"InitialSilenceTimeout","Offset":151200000,"Duration":50400000}'
[148361]: 1593590ms SPX_DBG_TRACE_VERBOSE:  lu_engine_adapter.cpp:250 CSpxLUEngineAdapter::ProcessResult: text=''

I opened the sample directly in 2021.3.3f1 and tested Intent Recognition as in your logs, using a USB headset, and it worked fine.

Hello, thank you for following up. I do have several microphone devices on my PC. I was not aware of a way to select a particular recording device? My intention is to use the system with the Oculus Quest 2. I suppose it is possible that Unity 2018 handles enumerating microphone devices differently than does 2021.3 and attempts to use the Oculus vs the Web cam microphone.
When testing I use the Quest2 over the PC-Link cable. I will try a build and see if the issue is still there.

Hi I can a build to the Quest 2 works fine where testing in the editor does not. I think it may be related to this https://store.facebook.com/help/quest/articles/fix-a-problem/troubleshooting-oculus-link/mic-issues-with-oculus-link/
Thanks for the pointer. I think this issue can be closed.


azure-cli 2.24.0 writes to STDERR

Describe the bug
We have been deploying to MS-hosted agents without issue. It looks like some of the agents started picking up azure-cli v2.24.0 recently and our deployments now fail only on those agents with: 2021-05-26T21:13:12.9875423Z ##[error]Script failed with error: Error: The process 'D:\a\_temp\azureclitaskscript1622063412668.bat' failed because one or more lines were written to the STDERR stream 2021-05-26T21:13:12.9964337Z [command]C:\Windows\system32\cmd.exe /D /S /C ""C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" account clear" The lines immediately before the error statement are the logs from a call to az container create There are no other errors reported in the tasklog. This error does not occur when we are assigned an agent with azure-cli v2.23.0. The error is not fixed by unchecking "fail on standard error" in the Azure CLI task.
To Reproduce
Be assigned an agent with v2.24.0 and run the Azure CLI task to create a container
Expected behavior
The Azure CLI task should succeed on both v2.23.0 and v2.24.0
Environment summary
Microsoft-hosted agent w/ vs2017-win2016 image
Azure pipelines Azure CLI task v1.164.0
azure-cli 2.24.0
core 2.24.0
telemetry 1.0.6
azure-devops 0.18.0

I ended up piping the errors to a file and this definitely happens on a call to az container create. I'm attaching the output of the error pipe and it looks like just a single empty line logged to STDERR
container_create - Example.log

Reverted my local version to azure-cli 2.23 and re-ran the exact same command and nothing was written to STDERR. Cannot attach the log file produced by the STDERR pipe on the container create command because it is just an empty file. This is different than v2.24 where there is a single empty line in the file.

Enabled --debug on the create command and did the same test with the two versions. Interestingly enough, there is an extra empty line in the error log for v2.24 that is not in v2.23. The line appears after the response content from a PUT with request url: 'https://management.azure.com/subscriptions/{SUBSCRIPTION_ID}/resourceGroups/{RESOURCE_GROUP}/providers/Microsoft.ContainerInstance/containerGroups/{CONTAINER_INSTANCE}?api-version=2018-10-01' and immediately before these lines:
DEBUG: cli.knack.cli: Event: CommandInvoker.OnTransformResult [<function _resource_group_transform at 0x04357B20>, <function _x509_from_base64_to_hex_transform at 0x04357B68>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnFilterResult []
DEBUG: cli.knack.cli: Event: Cli.SuccessfulExecute []
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x04321190>]
INFO: az_command_data_logger: exit code: 0
INFO: cli.__main__: Command ran in 2.919 seconds (init: 1.203, invoke: 1.715)
INFO: telemetry.save: Save telemetry record of length 3092 in cache

route to service team

@msftradford, the log file you shared is empty: https://github.com/Azure/azure-cli/files/6556807/container_create.-.v2.24.Example.log
Could you share the actual content in STDERR? Is it the same as #18276?

hi @jiasli! that's right. It appears to be a completely empty file but it actually has one single line composed of 70 space characters and I believe that single line is causing the task to fail (only with v2.24). This is different from the actually totally empty file that was created with v2.23.0. The issue that you mentioned appears unrelated - we don't receive any useful error messages beyond the single "empty" line with 70 space characters. This could still be an incompatibility issue between azure-cli and Python, but I'm not sure if it is the exact same issue.

@msftradford, I probably needs to route this issue to service team to investigate... Could you repro this issue locally?

yes! the piping to log files was done by running the commands locally, that is when I noticed the empty line in the STDERR pipe.

I am able to reproduce with
az container create --resource-group myResourceGroup --name mycontainer2 --image mcr.microsoft.com/azuredocs/aci-helloworld --dns-name-label aci-demo060702 --ports 80 2>stderr.txt

Looks like it has something to do with progress bar.

+ @Juliehzl to help take a look.


Custom width/height ratio & field background drawables for SquarePinField

Hi, i wonder if it's possible to add a custom background using drawables for SquarePinField, or at the very least custom width/height ratio instead of the default 1:1.
It is intended to fill a ui requirement such as this:

I can implement a custom drawable. The one with custom width/height ratio should be much easier to implement. Can you give me a week's time?

Amazing

I can implement a custom drawable. The one with custom width/height ratio should be much easier to implement. Can you give me a week's time?

Awesome!
Thank you so much for this great library

Is this issue fixed ?

Line text field text is very small and can't seem to increase it with text size


Test desktop.auth.view_test.TestRemoteUserLogin fails when hue is built with python3

Is the issue already present in https://github.com/cloudera/hue/issues or discussed in the forum https://discourse.gethue.com?
No
Describe the bug:
$ hue test specific desktop.auth.views_test.TestRemoteUserLogin fails when hue is built from python3.
Below is one of error logs from the test. Note the test fails on every test in TestRemoteUserLogin.
Error log:
======================================================================
FAIL: desktop.auth.views_test.TestRemoteUserLogin.test_force_lower_case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/deploy/brandon/khp-package-clean/repositories/hadoopeng/hue/build/env/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/deploy/brandon/khp-package-clean/repositories/hadoopeng/hue/desktop/core/src/desktop/auth/views_test.py", line 474, in test_force_lower_case
    assert_equal(1, len(User.objects.all()), User.objects.all())
AssertionError: 1 != 2 : <QuerySet [<User: hue>, <User: test_remote_user_login>]>
-------------------- >> begin captured logging << --------------------
desktop.middleware: INFO: Unloading MimeTypeJSFileFixStreamingMiddleware
desktop.middleware: INFO: Unloading SpnegoMiddleware
desktop.middleware: INFO: Unloading ProxyMiddleware
access: DEBUG: <IP_ADDRESS> -anon- - "GET /hue/accounts/login/ HTTP/1.1" - (mem: 143mb)
desktop.middleware: WARNING: request.fs was not set
access: INFO: <IP_ADDRESS> -anon- - "GET /hue/accounts/login/ HTTP/1.1" returned in 15ms 200 117664 (mem: 143mb)
useradmin.models: INFO: Installed a user "{'username': 'hue'}"
root: DEBUG: GET None http://infra-hadoop-build1.dakao.io:43052/webhdfs/v1?user.name=hue&doas=hue&op=GETHOMEDIRECTORY  returned in 6ms 200 20 {"Path":"/user/hue"}
useradmin.models: INFO: Home directory already exists for user: hue
desktop.auth.backend: INFO: User test_remote_user_login was created.
axes.attempts: DEBUG: AXES: Skipping clean for expired access attempts because no AXES_COOLOFF_TIME is configured
axes.handlers.database: INFO: AXES: Successful login by {username: "test_remote_user_login", ip_address: "<IP_ADDRESS>", user_agent: "<unknown>", path_info: "/hue/accounts/login/"}.
access: DEBUG: <IP_ADDRESS> test_remote_user_login - "POST /hue/accounts/login/ HTTP/1.1" - (mem: 143mb)
hadoop.fs.upload: DEBUG: Chunk size = 67108864
access: WARNING: <IP_ADDRESS> test_remote_user_login - "POST /hue/accounts/login/ HTTP/1.1" - (mem: 143mb)-- Failed login for user: None
access: INFO: <IP_ADDRESS> test_remote_user_login - "POST /hue/accounts/login/ HTTP/1.1" returned in 50ms 200 152592 (mem: 143mb)
desktop.middleware: ERROR: Could not audit the request: [Errno 2] No such file or directory: '/var/log/hue/audit.log'
root: DEBUG: GET None http://infra-hadoop-build1.dakao.io:43052/webhdfs/v1//user/test_remote_user_login?user.name=hue&doas=test_remote_user_login&op=GETFILESTATUS  returned in 8ms 0 0
root: DEBUG: GET None http://infra-hadoop-build1.dakao.io:43052/webhdfs/v1//user/test_remote_user_login_2?user.name=hue&doas=test_remote_user_login&op=GETFILESTATUS  returned in 5ms 0 0
--------------------- >> end captured logging << ---------------------

I added debug message to assertion to check why User.objects.all() has 2 items instead 1. As you can see, it has two users [<User: hue>, <User: test_remote_user_login>], where the test expects <User: test_remote_user_login> only. Maybe remote login logic has been changed in python3 (or in django 3)?
Steps to reproduce it?

build hue with python3
run hue test specific desktop.auth.views_test.TestRemoteUserLogin

Hue version or source? (e.g. open source 4.5, CDH 5.16, CDP 1.0...). System info (e.g. OS, Browser...).
open source latest master branch

'hue' is a system user, there is probably some Py3 in the login view calling the install of it from install_sample_user.
(searching for Installed a user)
This might be a legit thing, we could either dig a bit more or update the test for py3.
useradmin.models: INFO: Installed a user "{'username': 'hue'}"

https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/auth/views.py#L105


chrome does not trust custom certificates for quic

I have written an HTP3 service using Quiche, which loads a custom certificate that has been added to the system certificate repository. However, when accessing this service using Chrome and using Wireshark to capture packets, it will display an issue of certificate distrust. Has anyone encountered this issue before and how can we solve it

This is a chrome issue; see https://www.chromium.org/quic/playing-with-quic/ for more information.


Interlock fix

Moved service-labels info and deleted three options.

Deploy preview for docsdocker ready!
Built with commit d445de505d4d773ada82cdce8dfb124e298d51ff
https://deploy-preview-8902--docsdocker.netlify.com


Invalid request is sent for service/local/staging/profiles/{id}/start

The plugin sends {"data":{"description":{"data":"publishing"}}} message, however it does not match to the specification.
Here's API: https://oss.sonatype.org/nexus-staging-plugin/default/docs/path__staging_profiles_-profileIdKey-_start.html
Note that extra <data> is present only right after top-level element, however it looks like WrappingTypeAdapterFactory wraps all the objects with extra data which causes invalid description.
Apparently, Nexus ignores that and just uses default Explicitly created description.

Note: comment is visible in Nexus UI, so it makes sense to put project name + version.
The first two were created with nexus-publish-plugin while the last one was created with nexus-staging-maven-plugin:

Good catch!

By the way, I figured that out when building https://github.com/vlsi/nexus-stub
It caused parse failures, so I just completely ignore request body for now: https://github.com/vlsi/nexus-stub/blob/57dbb30dd34187c87f1825d07a1f49e23106bcd3/src/main/kotlin/com/github/vlsi/nexusstub/staging/StagingProfilesController.kt#L44-L48

I've fixed this in https://github.com/marcphilipp/nexus-publish-plugin/commit/cc61011d31a195b46b48f2bbe14a3d16f54b184a.
@vlsi Could you please give it another try with your nexus-stub project?


nitrpg: roadmap

This issue is used to submit ideas about the development of nitrpg.

[x] add a README
[x] add a homepage that list opened games
[x] players can unlock achievements
[ ] make stats by period (week, month, year)
[ ] add some graphs
[ ] touched by god (the owner @ yourself)
[ ] rpg-abuser (unlock 5+ achievements in a 24h time span)

Add the others tasks from dashboard (assigned things and needwork things)
feed information with past events

Achievement idea: After the battle. Give a +1 on a merged PR

Achievement idea: more review comments that lines of code (wink wink @MehdiAit)

Achievement idea: COMBO, bonus points if 2 or more achievements are done in a short amount of time

Achievement idea: "meta-achievement" propose an achievement idea in a discussion :)

Achievement idea : "Touched by god", if @privat pings you


[BEAM-775] Remove Aggregators from the Java SDK

This is #2718 rebased on master to see if build is fixed.

Coverage decreased (-0.1%) to 69.866% when pulling 4a124f6c199a3c61731d3b4e3ac6232a81e7583f on dhalperi:fixup-2718 into 7fa00647d5506281f456b4b9974ee9efc7484dba on apache:master.

I merged in #2816 upon tests passing giving credit to #2718.
Please close.


Enable use of Custom ceph repository on SuSE/SLES

This commit allows using custom ceph repository with ceph-ansible deployments on Suse/SLES distributions.
Signed-off-by: Nitin Kamble<EMAIL_ADDRESS>

jenkins test dev-centos-container-lvm_auto_discovery

jenkins test dev-centos-non_container-lvm_auto_discovery

These changes backport to the current stable branch v3.2 cleanly.

These changes backport to current stable v3.2 branch cleanly.

@leseb why should we backport this into stable-3.2 ? Is there a BZ for this ?

we only backport bugfixes into stable branches


getting prop-types warning...

Any idea about this?

Did you upgrade to the latest release? 0.6.1 should have this resolved

Sorry my bad. I have updated the dependency.


Database file keep locked even after the connection is closed

I'm using Microsoft.Data.Sqlite (2.1.0-preview2-final) in netcoreapp2.0 application.
My code is pretty straightforward, I open the connection to new db-file , create some tables, insert some data (not much) and then close the connection and dispose everything just to be on the safe side. But still, I get the error "The process cannot access the file 'test.db' because it is being used by another process."  when I try to delete the file after it's being populated with the data.
A workaround was found: call GC.Collect(); with GC.WaitForPendingFinalizers(); before delete db-file.
Sorry, I could not reproduce this error on a simple example. 😞
And, on previous version Microsoft.Data.Sqlite (2.0.0) this error was not noticed.

also #271
a generally recommended workaround that works is to call GC.Collect - https://stackoverflow.com/questions/2779599/sqlite-locking-file-even-though-connection-is-closed

I dug into this. The only way I was able to hit it was by not disposing SqliteCommand objects. This causes some sqlite3_stmt objects to be finalized after the call to SqliteConnection.Dispose().
We could update SqliteCommand call sqlite3_stmt.Dispose() during finalization so everything is freed in the same generation.


Prepare spring cloud azure 5.1.0 release

As the title to prepare Spring Cloud Azure 5.1.0 release

API change check
APIView has identified API level changes in this PR and created following API reviews.
spring-cloud-azure-core


Fix strange ref warning

I hit a strange warning with refs, and we need to update the way the ref is assigned in ActionForm for React 16 anyway.

Codecov Report

Merging #400 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #400      +/-   ##
==========================================
+ Coverage   99.77%   99.77%   +<.01%
==========================================
  Files          25       25
  Lines         894      896       +2
==========================================
+ Hits          892      894       +2
  Misses          2        2

Impacted Files
Coverage Δ

src/addons/action-form.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7e1c91...f027964. Read the comment docs.


change with the currency selection

-It should change with the currency selection.

This is already resolved and merge to development


Made all labels have the message_line color

Now all messages that appear in LabeledLine will have the yellowish color. This color differs from terminal to terminal but is all sort of yellow (from what I found). I could use the default color (black text on white background), but this seems to catch the eye more so the user can easily see that some text just appeared on the screen. Resolves #54

My version of that color must be different I'm guessing. I see dull yellow text on a white background (which doesn't pop, it's very muted).

Hmm yeah that's true. On my Ubuntu terminal, it was too bright and didn't have much contrast with the background. Maybe something darker like blue or brown may work better. I'll send another PR and you can see if it works well on your terminals.


Issue #1254: Clean up config functional tests, enhance merge syntax

Files Changed

tm/sys/config.py
tm/sys/test/functional/test_config.py

ready for review @caphrim007


Add more testing and fix FirstLocationOfMinimum, FirstLocationOfMaxmimum, AggLinearTrend

Hello,
First of all, thanks for this extension of featuretools :)
I'd like to contribute by adding tests that compare features produced by featuretools-tsfresh-primitives with the ones calculated by tsfresh. The tests can take some time because they create many combinations of features based on ComprehensiveFCParameters.
In this context, I wrote the function primitives_from_fc_settings that returns a list of primitives from a dict of tsfresh settings.
I also fixed the features that didn't pass the tests:

FirstLocationOfMinimum and  FirstLocationOfMaxmimum failed if passed a pandas.Series, which is the case within featuretools.dfs. This is because, if passed a Series, the result depends on the index. For example:

from tsfresh.feature_extraction.feature_calculators import first_location_of_minimum

import numpy as np
import pandas as pd

x = pd.Series(np.random.uniform(size=10))
a = first_location_of_minimum(x)

x.index = np.random.randint(100, size=10)
b = first_location_of_minimum(x)

a == b
#> False

Created on 2019-10-26 by the reprexpy package

AggLinearTrend(f_agg = 'var', ...)'s results were also incorrect if passed a Series.

Finally, I noticed that linear_trend_timewise is not implemented yet.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hey @jeffzi ,
Cool contribution.  We didn't have CircleCI set up to run forked PRs on this repo, could you try resolving the conflicts with master to trigger the tests?

I've just merged master into this fork. The tests take ~3min on my laptop. If you approve this PR, you may want to reduce the number of parameters tested, depending on the CI constraints.

Hello @rwedge
Is there any reason to not merge this PR?

Hey @jeffzi ,
I've been backlogged and haven't had a chance to review this PR.  I'll review it sometime this week.


refactor(pilot): extract method to get header operations

Signed-off-by: knight42<EMAIL_ADDRESS>The follow-up work for https://github.com/istio/istio/pull/25444#pullrequestreview-446893165
And to help us figure out who should review this PR, please
put an X in all the areas that this PR affects.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

/cc @costinm


merge frontend and backend tar into one binary tar

Tips

Thanks very much for contributing to Apache DolphinScheduler.
Please review https://dolphinscheduler.apache.org/en-us/community/index.html before opening a pull request.

What is the purpose of the pull request
merge frontend and backend tar into one binary tar
Brief change log
merge frontend and backend tar into one binary tar
Verify this pull request
mvn clean package -Prelease
There will generate apache-dolphinscheduler-incubating-${latest.release.version}-dolphinscheduler-bin.tar.gz and apache-dolphinscheduler-incubating-${latest.release.version}-src.zip in the folder of dolphinscheduler-dist/target

Codecov Report

Merging #1511 into dev will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##             dev   #1511   +/-   ##
=====================================
  Coverage   6.75%   6.75%
=====================================
  Files        272     272
  Lines      13612   13612
  Branches    2165    2165
=====================================
  Hits         920     920
  Misses     12628   12628
  Partials      64      64

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6de55c3...8c7736b. Read the comment docs.


Rate limiter whitelist

Is your feature request related to a problem? Please describe.
During load/performances tests, it would be nice to allow defining whitelisted IP (traffic injectors), so there is no need to disable this feature during the tests.
Also this could be interesting to be used when other internal trusted servers are calling the application for batch jobs.
Describe the solution you'd like
Add a new parameter, a list of IPs that will be whitelisted and not stored in the rate limiter DB, thus allowing no limits for specific hosts.
Describe alternatives you've considered
Disable the rate limiter during the tests

Hey @zguig52
Thanks for this idea, sounds really reasonable. Would you be interested in contributing to the module with this feature? I can provide any help needed :)

Hi @Baroshem ,
I would be very happy to contribute with this feature. I will try to get some time before the end of the month to work on it. I have first to read all the doc related to Nuxt plugin dev and setup the dev env with your plugin, I never worked on this.
Thanks for this :)

@zguig52 I think we can easily parse at server startup
Right now the runtime/nitro/plugins/00-routeRules plugin is already used to instantiate the appSecurityOptions singleton at startup.
This would also allow to use the nuxt-security:routeRules hook to dynamically modify the whitelist if needed

@vejja , thanks for your response. I will have a look to this file to check what can be performed at startup!


Update to Spring Boot 3

Spring Boot 2.7 is currently in use. It should be updatet to Spring Boot 3.