istio ambient with Ztunnel on premise - XDS client connection error

Hi everyone,
I'm trying to deploy istio with ambient mode activate on on-premise cluster. As DeamonSet Ztunnel is deploy on both master and worker.
But on master it can reach istio control plane.
warn	xds::client:xds{id=12}	XDS client connection error: gRPC connection error connecting to https://istiod.istio-system.svc:15012: status: Unknown, message: "client error (Connect)", source: dns error: failed to lookup address information: Name or service not known, retrying in 15s
Ztunnel on worker is fine. Only this error appear on control plane.
Istio is installed using istioctl :
istioctl install --set profile=ambient --set components.ingressGateways[0].enabled=true --set components.ingressGateways[0].name=istio -ingressgateway  --set components.egressGateways[0].enabled=true --set components.egressGateways[0].name=istio-egressgateway
Core DNS is deploy on both worker and master, but seem to not change something.
Affected version of istio

1.23.x
1.24.x

Complementary information
I'am not using waypoint
Kubernetes version 1.31
Thanks in advance for your help

Ztunnel has entirely standard Kubernetes networking setup, so if the DNS lookup isn't working, either the Istiod service doesn't exist or your cluster DNS is having issues

It's working on workers as Istiod service exist. Yes i think too that is DNS issue. I'm looking for why and how can i troubleshoot this.

If your master is not allowed to deploy applications, you donot need deploy ztunnel and istio-cni.
For "client error (Connect)", source: dns error: failed to lookup address information: Name or service not known, retrying in 15s, you may need to make sure if kube-proxy is there


Hapi v11.x Support

Please update to work with Hapi v11.x. Changelog is located at https://github.com/hapijs/hapi/issues/2850

This would be great!

Are you missing any new features? Whats lacking? Will have a look next week.

Hrm....I thought due to the way the peer dependencies are defined, it was not possible to use this with Hapi v11.x, but I'm thinking it was possibly a different issue. I can double-check tomorrow and follow up.

yeah this has been fixed with #50 (since 2.4.1)

Sorry, we had 2.4.0 installed and I didn't notice that 2.4.1 addressed the dependencies. Closing!


quests stopped working suddenly?

What is your Quests version / build number (do not say "latest")?
quests-4.3.0
paper 1.18.2
Describe the problem and how to replicate it.
bug 1:
Reach location
Bug2:
kill last entity on quest
All buggs revolve around finishing stages and moving to next stage of quest
Any console errors to share?
error on completing go to location stage:
https://pastebin.com/ZN32pmUF

error on entity death event on completing kill entity zombie:
https://pastebin.com/jXKnrcCa

If applicable, can you provide an example quest from quests.yml?
https://pastebin.com/mbRqUuSq

Already fixed in 11ffbf46be5b514b418161545633f4cb093f9caa
Use https://ci.codemc.io/job/PikaMug/job/Quests/320/


[BUG] Service request failed error when trying to start an automation runbook job.

Library name and version
Azure.ResourceManager.Automation 1.0.1
Describe the bug
We are observing a "Service request failed" exception when attempting to start an automation runbook job. Interestingly the error message also says "Status 200 (OK)" in addition to what appears to be information about the last successful job that was run (in this case manually started from the browser).
Stack Trace:
at Azure.ResourceManager.Automation.JobRestOperations.d__21.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Azure.ResourceManager.Automation.AutomationJobCollection.d__5.MoveNext()
Expected behavior
We should be able to start a runbook job without error.
Actual behavior
The SDK request fails with "Service request failed" error
Reproduction Steps
private async Task Test(AutomationAccountResource resource, string runbookName,
IDictionary<string, string> parameters)
{
var content = new AutomationJobCreateOrUpdateContent
{
RunbookName = runbookName,
RunOn = string.Empty
};
foreach (var param in parameters)
    content.Parameters[param.Key] = param.Value;

await _resource.GetAutomationJobs()
    .CreateOrUpdateAsync(WaitUntil.Started, runbookName, content);

}
Environment
Windows 10, .NET Framework 4.8

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

This should be a service issue, they didn't define response 200 for this operation in the API spec.
Route to service team to have a look.


Fix kibana's init container configure-kibana-token http request error 409

this is a fix proposal to this issue: https://github.com/elastic/helm-charts/issues/1714
kubernetes pods die regularly and new ones are created. kibana's init container does not work this way since it always tries to create a new token, which elasticsearch will deny if it already exists.
this PR makes kibana's init container attempt to delete the old token before attempting to create a new one. deleting a non-existing token returns a valid status code so this is not a breaking change. I chose this approach because I found no way of getting an existing token using elasticsearch API, the token is only returned on creation (please correct me if i'm wrong).
Also, i noticed that there is a helm cleanup job to cleanup the old token, but that one runs only when we delete the entire helm release, and will not trigger on pod termination/creation.

[x] Chart version not bumped (the versions are all bumped and released at the same time)
[ ] README.md updated with any new values or changes
[ ] Updated template tests in ${CHART}/tests/*.py
[ ] Updated integration tests in ${CHART}/examples/*/test/goss.yaml

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

Closing because it will have to be done in a different way in https://github.com/elastic/helm-charts/pull/1720.
Thanks for your work @mdnfiras


Websocket Reconnection Issues/Questions

Hey there. Thanks for the great API wrapper. I have a few questions that I didn't find the answer for.
TLDR:

Does the reconnection mechanism keep trying forever? Or does it give up after the 3 times?

Do I need to re-subscribe after getting a reconnected event?

subscribeContinuousContractKlines seems to be working fine
subscribeUsdFuturesUserDataStream is the one I'm having my most problems with

Also I keep getting the message below in my console, is this normal?
(node:1485) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [TLSSocket]. Use emitter.setMaxListeners() to increase limit

I have a strategy in mind that I'm coding and I'm indenting to leave this nodeJS app running 24/7. I know you have a reconnection mechanism in place for the WebsocketClient but is it guaranteed to try to reconnect forever if for some reason the computer lost its connection to the internet for hours?  Or does it give up after a few times? After reading other issues and peaking thru the code a little bit, I understood that (and correct me pls if I'm wrong) that it tries to reconnect 3 times but what happens after that? Does it give up and I have to reconnect it manually again? Because I had an internet outage for 2 hours and when the internet came back up, the WS did not reconnect. I just need to make sure this is the intended behavior.
Also regarding the Usd Futures User Data Stream. I understand it gets killed after one hour. Does it reconnect automatically after one hour? Or should I implement the reconnection logic myself? Because also, I was running my strategy today for a few hours and it took the wrong decisions because the user data stream wasn't streaming position updates even though I had no disconnects but when I checked the logs it kept saying.
 Completed keep alive cycle for listenKey(xxx) in market(usdm)

And do I need to re-subscribe to the user data stream upon receiving a reconnected event?

Does the reconnection mechanism keep trying forever? Or does it give up after the 3 times?

Yes, it keeps trying forever. The overall goal is for the ws client to worry about connectivity, so all you need to do is provide api credentials, ask for some topics and handle events as they arrive. The client will do the heavy lifting for you to ensure you stay connected.

Do I need to re-subscribe after getting a reconnected event?

No, that's why there are subscribe functions. The websocket client stores state on which topics you've subscribed to, so if a disconnect happens, it has all the info that it needs to replace a killed connection.
The reconnected event is useful for knowing you were disconnected for some time and may have missed some data. It's a good opportunity to check with the REST API, if you need to check for data that you missed since the last event (e.g. synchronising balance, orders, positions, etc).

subscribeContinuousContractKlines seems to be working fine
subscribeUsdFuturesUserDataStream is the one I'm having my most problems with

What kind of problem do you have with it, is it the event emitter leak below?

Also I keep getting the message below in my console, is this normal?(node:1485) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [TLSSocket]. Use emitter.setMaxListeners() to increase limit

This is not normal. It can happen if you have some recurring logic that keeps setting event listeners like:
someEmitter.on('someEvent', (event) => .....);
Every time you call someEmitter.on(..., it creates a new event listener. I would check for any functions you have that are calling themselves, I would also check for any logic you have inside a setInterval callback. Make sure these event listeners are only created once and never again.

I have a strategy in mind that I'm coding and I'm indenting to leave this nodeJS app running 24/7. I know you have a reconnection mechanism in place for the WebsocketClient but is it guaranteed to try to reconnect forever if for some reason the computer lost its connection to the internet for hours? Or does it give up after a few times? After reading other issues and peaking thru the code a little bit, I understood that (and correct me pls if I'm wrong) that it tries to reconnect 3 times but what happens after that? Does it give up and I have to reconnect it manually again? Because I had an internet outage for 2 hours and when the internet came back up, the WS did not reconnect. I just need to make sure this is the intended behavior.

I think you might be thinking of the keep-alive request. The user data stream is tied to a listen key. You have one listen key per account. Once it's created, by default it will expire after 1 hour, unless you call the keep alive function (even if you are still connected). The SDK will automatically keep a timer to send a keep-alive request on the listen key and keep track of it. If this request fails 3 times for whatever reason (e.g. disconnected wifi), it'll assume there's a problem with the listen key and/or network and a full reset is needed.
A full reset means killing all timers (heartbeats + keep alive) for that user data stream, killing the idle connection if it's still there, asking the api for a new listen key, and reconnecting with the new listen key. This is handled automatically under the hood in this workflow:
https://github.com/tiagosiebler/binance/blob/master/src/websocket-client.ts#L950-L996
You shouldn't need to do anything to handle these lifecycle events, the networking is managed for you by the SDK. I originally had a lot of pain with some of the other open source binance SDKs (also other exchanges), so I spent a ton of time building out how this works to ensure I don't have to worry about dead connections that stopped receiving data. One easy way to test stability is by screwing with the connection on our machine (wifi off/on) and messing around with vpn connectivity (get some interesting networking behaviour while a vpn is connecting or stalled).

Also regarding the Usd Futures User Data Stream. I understand it gets killed after one hour. Does it reconnect automatically after one hour? Or should I implement the reconnection logic myself? Because also, I was running my strategy today for a few hours and it took the wrong decisions because the user data stream wasn't streaming position updates even though I had no disconnects but when I checked the logs it kept saying.

This is actually handled for you, the user data stream typically sends an event if the listen key expires unexpectedly. I use that as a trigger to force a reconnect (kill old connection, ask for new listen key, reconnect with new listen key):
https://github.com/tiagosiebler/binance/blob/master/src/websocket-client.ts#L384-L392
I haven't seen this behaviour in a long time (in bots running for weeks 24/7). If you've found a way to reproduce the behaviour, I'd very much want to reproduce it too and investigate how to improve resilience even further.
Something I do want to ask is what kind of environment are you running on? Is it a local machine at home (e.g. connected to your wifi)? Is it a VPS? Dedicated server? cloud? Maybe that's a factor to reproducing this issue (most of my long-running testing is now on a dedicated server or vps).

 Completed keep alive cycle for listenKey(xxx) in market(usdm)

And do I need to re-subscribe to the user data stream upon receiving a reconnected event?

No, it's handled automatically. If a keep alive is successful, the same listen key remains active and nothing needs to change. If it fails or the response suggests a problem, the old connection should be killed and replaced under-the-hood. Since you're listening to events on the ws client, you don't need to do any actions to stay connected - the ws client will handle this "plumbing" for you automatically.

Thank you for the detailed explanation. Yes I'm running my nodeJS app currently on a local machine using pm2 on Node v20.5.1 with the plan of moving to a VPS once I did enough testing and confident enough that my code has no bugs.

This is not normal. It can happen if you have some recurring logic that keeps setting event listeners like:
someEmitter.on('someEvent', (event) => .....);

Yes I thought so but my code only adds these event listeners in a constructor and I only have one object with the constructor being called once. I built a wrapper on top of the WebSocketClient to make my custom events like onCandleClose, onBalanceUpdate, onPositionUpdate, etc...
Maybe I have to take a look once more or if I could provide you with a way to recreate this that would be helpful.
This is a very long shot 😅 and I'm not sure if this might be causing any issues but I'm using my Macbook M1 max in Clamshell mode with an external display and currently in the area I'm living in, we're experiencing power outages regularly. Whenever the power goes out, the macbook goes to sleep because in order for it to work with an external display and not sleep, it needs to be connected to a power source. I'm not sure how does the NodeJS process act when the computer sleeps and then wakes up again when the power is back. Maybe this is causing an issue with all the timers/heartbeat mechanism for reconnection. (idk?)
I looked at your code again and understood that if I call subscribeUsdFuturesUserDataStream it returns the existing connection (if it exists) and it does not create a new one. I tried setting a recurring call to subscribeUsdFuturesUserDataStream every 30 mins as a dirty way to ensure its alive. So far its working fine. (been running for 6 hours and so far it did not lose its connectivity to the user data stream.
I understand that I did not come to you with a "re-creatable" case here so I'd understand if you wanna close this issue. Thank you so much for your efforts and for this amazing API wrapper. I'll update this issue with my findings or what I did wrong once I figure it out (if the issue is still open) and thank you once more! 🙏

I think I've once reproduced a rare issue while my macbook was folded away / asleep & disconnected for an hour while commuting home. It's been a long time since I've seen it though, so I'm curious if there's still an issue lurking that only happens when a device is asleep & disconnected that long with the process still running. I'd say keep this open and see if we can pin down a clear pattern to reproduce. If there is still an issue it's a very rare one that doesn't happen in "normal" environments like a server, since I do also have processes on various servers running 24/7 uninterrupted with the user data stream...

I think that was the issue. I deleted all the custom code I've written for "reconnectivity" and now only depend on the wrapper's internal mechanism. (I add the listeners only once, and call the subscribe methods only once)
To test this, I created a EC2 instance on AWS and ran the bot on it. Been up for 10 hours so far with no issues. That's far more uptime than I've ever had without issues on my local machine.

Good to hear! Do let me know if you pin down a way to reproduce that issue semi-consistently though, or if you notice any other weird behaviours!

Closing this for now. User data websocket handling improved significantly with #380 (v2.9.0), to also fix a rare issue with reconnecting after a listen key expired (after being disconnected for extended periods of time - e.g. with the laptop closed/asleep)). If you notice any issues or weird behaviour with the latest version, feel free to reopen or make a new issue to investigate.


[Enhancement] Set min threads to 0 for some thread pools

Now the BE process may be running in shared-nothing mode or shared-data mode. Some threads are unnecessary for shared-data mode, and creating threads is unnecessary. By setting the minimum number of threads in some thread pools to 0, some unnecessary threads can be avoided in shared-data mode.
Fixes #issue
What type of PR is this:

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.1
[x] 3.0
[ ] 2.5
[ ] 2.4

@Mergifyio backport branch-3.1

@Mergifyio backport branch-3.0


Add telepot to list of install packages

2.85s$ pip install telepot
Collecting telepot
  Downloading telepot-12.3.tar.gz (70kB)
    100% |████████████████████████████████| 71kB 3.4MB/s
Requirement already satisfied: urllib3>=1.9.1 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from telepot)
Building wheels for collected packages: telepot
  Running setup.py bdist_wheel for telepot ... done
  Stored in directory: /home/travis/.cache/pip/wheels/29/1c/f9/fbedd9f3b62b222be9dd5b249e4eceafca375e0ec4f2032cf1
Successfully built telepot
Installing collected packages: telepot
Successfully installed telepot-12.3
install.3
1.00s$ pip install -r requirements.txt
Requirement already satisfied: Flask>=0.10.1 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 1))
Requirement already satisfied: Flask-Cors>=3.0.2 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 2))
Requirement already satisfied: pymongo>=3.0 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 3))
Requirement already satisfied: argparse in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 4))
Requirement already satisfied: requests in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 5))
Requirement already satisfied: python-dateutil in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 6))
Requirement already satisfied: pytz in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 7))
Requirement already satisfied: PyJWT in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 8))
Requirement already satisfied: bcrypt in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from -r requirements.txt (line 9))
Requirement already satisfied: click>=2.0 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from Flask>=0.10.1->-r requirements.txt (line 1))
Requirement already satisfied: Jinja2>=2.4 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from Flask>=0.10.1->-r requirements.txt (line 1))
Requirement already satisfied: Werkzeug>=0.7 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from Flask>=0.10.1->-r requirements.txt (line 1))
Requirement already satisfied: itsdangerous>=0.21 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from Flask>=0.10.1->-r requirements.txt (line 1))
Requirement already satisfied: Six in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from Flask-Cors>=3.0.2->-r requirements.txt (line 2))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from requests->-r requirements.txt (line 5))
Requirement already satisfied: certifi>=2017.4.17 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from requests->-r requirements.txt (line 5))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from requests->-r requirements.txt (line 5))
Requirement already satisfied: idna<2.7,>=2.5 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from requests->-r requirements.txt (line 5))
Requirement already satisfied: cffi>=1.1 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from bcrypt->-r requirements.txt (line 9))
Requirement already satisfied: MarkupSafe>=0.23 in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from Jinja2>=2.4->Flask>=0.10.1->-r requirements.txt (line 1))
Requirement already satisfied: pycparser in /home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages (from cffi>=1.1->bcrypt->-r requirements.txt (line 9))
6.56s$ nosetests tests/*
.................................E
======================================================================
ERROR: test_telegram_webhook (test_webhooks.WebhooksTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/guardian/alerta/tests/test_webhooks.py", line 551, in test_telegram_webhook
    response = self.app.post('/webhooks/telegram', data=self.telegram_ack % telegram_alert_id, headers=self.headers)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/werkzeug/test.py", line 801, in post
    return self.open(*args, **kw)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/testing.py", line 127, in open
    follow_redirects=follow_redirects)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/werkzeug/test.py", line 764, in open
    response = self.run_wsgi_app(environ, buffered=buffered)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/werkzeug/test.py", line 677, in run_wsgi_app
    rv = run_wsgi_app(self.application, environ, buffered=buffered)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/werkzeug/test.py", line 884, in run_wsgi_app
    app_rv = app(environ, start_response)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/travis/virtualenv/python2.7.13/lib/python2.7/site-packages/flask_cors/decorator.py", line 128, in wrapped_function
    resp = make_response(f(*args, **kwargs))
  File "/home/travis/build/guardian/alerta/alerta/app/auth.py", line 157, in wrapped
    return f(*args, **kwargs)
  File "/home/travis/build/guardian/alerta/alerta/app/webhooks/views.py", line 795, in telegram
    send_message_replay(alert, action, user, data)
  File "/home/travis/build/guardian/alerta/alerta/app/webhooks/views.py", line 773, in send_message_replay
    LOG.warning("Error sending reply message", exec_info=True)
  File "/opt/python/2.7.13/lib/python2.7/logging/__init__.py", line 1179, in warning
    self._log(WARNING, msg, args, **kwargs)
TypeError: _log() got an unexpected keyword argument 'exec_info'
-------------------- >> begin captured logging << --------------------
urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=0, status=None)
urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): api.telegram.org
urllib3.connectionpool: DEBUG: https://api.telegram.org:443 "POST /botNone/editMessageText HTTP/1.1" 404 55
--------------------- >> end captured logging << ---------------------


create ext dir and move driveworks

Description
We'd like to move driveworks directory to src/ext/nvidia/driveworks from src/driveworks .
Also, from now on, src/ext directory will be a directory that  have extended feature like driveworks.

Closing this as invalid due to upcoming repository reorganisation.


expand the tcp.ts integration tests

This addresses most of what we want to test, I think:
https://github.com/uProxy/uproxy/issues/486
This helped me spot some of the issues in freedom-for-chrome's TCP socket provider:
https://github.com/freedomjs/freedom-for-chrome/pull/62

@bemasc PTAL!

:+1:


filter_horizontal not supported

I added the UI soft dashboard theme to my project which has M2M relation and in admin I added to the ModelAdmin class the filter_horizontal which will help to select the items as the following photo. But this is not exist in the theme.

Hello @mominur-helios,
Please take a look at this use case that might relate to #9
ty!

Hello @khaled-hayek,
Can you please pull the latest changes. This issue is already fixed.

Hello @khaled-hayek,
Make sure you're using the latest version 1.0.10 and let us know if the issue is still there.
ty!


Client-side Docker image

Running weather-mv pipelines is difficult because the dependencies are hard to set up. A client-side Docker image for submitting the jobs can make life easier.

Instead of docker, I'd prefer if we implemented #179.

I see your point. One of the benefits to providing a Docker image is that it can include gcloud CLI and any necessary scripts in the future as well, so the barrier to entry for the users would be lower. This also simplifies the new README instructions.
The image can look like this:
Dockerfile
FROM ubuntu:22.04
FROM continuumio/miniconda3:4.12.0

# Install latest gcloud cli
RUN apt-get update
RUN apt-get install -y apt-transport-https ca-certificates gnupg curl
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
RUN apt-get update && apt-get -y install google-cloud-cli

# Create Conda environment
WORKDIR /weather
ADD tinyenv.yml /weather/environment.yml
RUN conda env update --name base --file environment.yml --prune

# Install weather-tools
ADD . /weather
RUN pip install --upgrade pip setuptools
RUN pip install .
RUN pip install weather_dl/
RUN pip install weather_mv/
RUN pip install weather_sp/

# make gcloud configurations persist
VOLUME ["./config"]

tinyenv.yml
name: weather-tools
channels:
- conda-forge
dependencies:
- python=3.8
- gdal
- metview-batch

The build takes 5-10 minutes and makes the quickstart easier.

I like that as an add on feature for the future. However, the priority is a conda environment because:

Our north star is for this project to be cloud agnostic instead of wed to GCP
Docker is a heavy dependency for our target users, who are scientists in the Python ecosystem (over data engineers). For these users, conda is the preferred way to manage environments.


The Spoof intelligence now has four reports and the content does not reflect this

On the portal https://protection.office.com/searchandinvestigation/dashboard
The Spoof intelligence now has four reports.

Likely Spoofed Domains over the past 7 days with suspicious and non suspicious.
Impersonations over the past 7 days with Domains impersonated and Users impersonated
The content does not reflect this.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2eb5aaaa-d804-cdbe-79ca-826cb67cc3e5
Version Independent ID: dd128a8c-afe2-c507-5f19-ed588f00bc7a
Content: Walkthrough - Spoof intelligence insight - Office 365
Content Source: microsoft-365/security/office-365-security/walkthrough-spoof-intelligence-insight.md
Service: o365-seccomp
GitHub Login: @chrisda
Microsoft Alias: chrisda

@wobblewobble, I updated the topic with a new screenshot and information based on current reality for the Spoof intelligence insight.
Technically, spoof intelligence and impersonation protection are different technologies. That impersonations insight is new, but is separate from spoof intelligence, so I'm not going to include it.
Thanks for the feedback.
Closing


Event log preprocessing and variant analysis

This pull request adds an IPython and RMarkdown notebooks for the process mining plots in the forthcoming data paper.

Thanks Niklas!


AWSJSON should return JSON string

Thanks @stevearoonie.  Please bump the patch version and we can get this merged.
Also, you should know that this project is being incorporated into AWS's amplify project.  If you're working on a new project I suggest using their module instead of this as it will be deprecated soon.

Also, you should know that this project is being incorporated into AWS's amplify project. If you're working on a new project I suggest using their module instead of this as it will be deprecated soon.

@cbaron that's exciting! What's going to be the future of the Serverless Framework integration?

@pianomansam -- good question.  To be honest I haven't dug into the amplify module, so I can't answer that at the moment.


Offline Modus

As a user I would like to have an offiline mode in order to be able to access mails that have been previously loaded even when when there's no internet connection.

We should consider where we want to persist the mails and ask the user before

@martintaraz Can you please explain what has been done to ease the implementation of an offline mode in the future?

related to #348
once that one is solved, emails can be persisted as maildir objects on disc.


Heads-up about upcoming changes to BuildPad

I want to release an update to BuildPad in the next few days, which brings with it some major changes. Since it's included in GW2-UOAOM, I want to give you an early heads-up to make the necessary changes to the addon manager. I can delay the update if you need extra time to adjust.

BuildPad will now be using ArcDPS's built-in extension autoupdate system, which silently downloads the new version at GW2's startup and replaces the old DLL with a new one.
DLL's name will be changing from "d3d9_arcdps_buildpad_YYYY-MM-DD.dll" to "arcdps_buildpad.dll", dropping both the "d3d9_" and the version parts. No filename shenanigans going forward.
Starting with the next update, the plugin will rename itself to "arcdps_buildpad.dll" automatically upon closing GW2.
The next update will still be downloaded from the website with the old name to ensure that the old autoupdate system handles the transition gracefully (as gracefully as possible, at least...), all future ones will be sent with "arcdps_buildpad.dll" in Content-disposition header.
It's "ok" if the addon manager downloads the next update as "arcdps_buildpad.dll", the update should still work, albeit with a small chance of problems due to the async nature of how ArcDPS now loads extensions.

Will

Since I received no response on this, I went forward with the update. Apologies to users of GW2-UOAOM who will encounter problems.

This impact gw2 radial? I just got a new laptop and i cant seem to get it to work.

Hey,
I have been noticing that my addon manager was constantly 'updating' buildpad recently, so this probably has to do with the changes you mentioned.
If buildpad now uses arcdps's update manager, I guess it's not worth it to include it in the addon manager ? any thoughts ?

It's worth to include it simply to let people know it exists and give them a convenient option of installing it. Unfortunately, both removing it from the manager and updating the manager to remove the special code that's no longer needed for buildpad would require addon manager maintainers to make changes and release a new version, which they don't seem to be in great haste of doing.
Unless you mean not removing it from the addon manager entirely, but simply disabling it on your end, in which case - sure, give it a shot. Just like any other arcdps plugin - you can install it manually by placing the dll in bin64. Buildpad was always capable of updating itself.


Crash

Version of Boardwalk:Beta 1.2
Version of Android:Android 5.1
Device model:Motorola G
Description of the problem:When I do a certain server on multiplayer it messes me up and crashes the app. What do I do?

IDK about other server but I think it crash too.

Also is there a 1.2.1 version if so can I have the link thank you.

Or can someone just give me the link to the latest update of the app please. I keep seeing different versions. Thanks again

http://zhuoweizhang.net/Boardwalk/Boardwalk-release.apk
This is the latest version of the app. This link will always download the latest version. The app lets you choose between different Minecraft versions - they aren't different versions of the app.


protobuf: Delete makefiles, regenerate protos consistently

Delete all the Makefiles except for the top-level one.
Recursive make is difficult to understand, keeping all
the Makefiles in sync is difficult (as witness the fact
that there were numerous inconsistencies between them),
and mostly they just ran trivial commands like "go test".
Replace all regeneration of .pb.go files with a single
regenerate.sh shell script.
Stop relying on the protobuf repository being installed
in ~/src/protobuf. The only things we need out of the
repository are standard .proto source files, which
protoc is perfectly capable of discovering on its own.
Regenerate .pb.go files which had been missed by
"make regenerate" before.

LGTM

Hi, why do we delete the Makefile? I think it's easier to install by run make install.
Now, how should I install this? go install?

The toplevel Makefile doesn't exist!


Fix broken Latex generation

What was wrong?
Turns out readthedocs build is broken because the latex generation broke
How was it fixed?
Re-add some missing info that latex needs.
To-Do

[x] Clean up commit history

[ ] Add entry to the release notes

Cute Animal Picture

I'm going to merge this without review to unbreak the RTD build.


FosUserBundle bug in PUT action

I have the same problem like in this closed issue: https://github.com/dunglas/DunglasApiBundle/issues/55
I installed FosOAuthServerBundle and made FosUser integration with ApiBundle, and now I have that situation:

Make request for OAuth token with grant_type password (admin user)
Make Put request for Users/1 (Admin) with Authorization Token (I’m logged as Admin user with this token)
Response is like the User Entity is empty so there are a validation errors like empty email and username.

I Found that the Class of User Entity from „findOrThrowNotFound” method is:
"Proxies_CG_\AppBundle\Entity\Profile\User"
But in „deserialize” method in PutAction we send class parameter:
„AppBundle\Entity\Profile\User” - so in ItemNormalizer class we have variable „overrideClass” with false and object cannot be initialize cause it expecting class „AppBundle\Entity\Profile\User” not proxy class.
If I make request for another User (not logged) or when I’m using token with grant_type client_credentials everything is ok.
How fix that?
Thanks

Can you post the full stack trace here please?

If the object_to_populate key exists and is associated with an instance of the proxy, it should work as expected.
Can you provide a failing test?

Ok, it looks like the problem is in Symfony directly.
https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Serializer/Normalizer/AbstractNormalizer.php#L264
Can you try to replace $class === get_class($context['object_to_populate']) with $context['object_to_populate'] instanceof $class === get_class()?
If it fixes your problem I'll open a PR against Symfony.

It will be ok if I replace with:
 $context['object_to_populate'] instanceof $class

https://github.com/symfony/symfony/issues/15627

Thanks


CLEVR

Why is this project code reporting errors in the CLEVR dataset?
The question is:
Traceback (most recent call last):
File "/mnt/public/home/s-xuk/mcan-gqa/run.py", line 160, in
execution.run(__C.RUN_MODE)
File "/mnt/public/home/s-xuk/mcan-gqa/utils/exec.py", line 33, in run
train_engine(self.__C, self.dataset, self.dataset_eval)
File "/mnt/public/home/s-xuk/mcan-gqa/utils/train_engine.py", line 143, in train_engine
) in enumerate(dataloader):
File "/mnt/public/home/s-xuk/anaconda3/envs/python37/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in next
data = self._next_data()
File "/mnt/public/home/s-xuk/anaconda3/envs/python37/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
return self._process_data(data)
File "/mnt/public/home/s-xuk/anaconda3/envs/python37/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
data.reraise()
File "/mnt/public/home/s-xuk/anaconda3/envs/python37/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise
raise self.exc_type(msg)
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/mnt/public/home/s-xuk/anaconda3/envs/python37/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
data = fetcher.fetch(index)
File "/mnt/public/home/s-xuk/anaconda3/envs/python37/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/mnt/public/home/s-xuk/anaconda3/envs/python37/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/mnt/public/home/s-xuk/mcan-gqa/openvqa/core/base_dataset.py", line 36, in getitem
frcn_feat_iter, grid_feat_iter, bbox_feat_iter = self.load_img_feats(idx, iid)
File "/mnt/public/home/s-xuk/mcan-gqa/openvqa/datasets/clevr/clevr_loader.py", line 163, in load_img_feats
grid_feat = np.load(self.iid_to_grid_feat_path[iid], allow_pickle=True)
KeyError: '44448'

【自动回复】您好，已收到邮件！This is an automatic reply, confirming that your e-mail was received. Thank you!


Persistancy against errors

Hi all.
thank you dev-team of this app. it is a good base and well written code.
i want you to know that i implemented on local

a nodes/mines gathering feature. that start a watch tower search, wait the report mail and then go the field for gathering
a troops training feature. that try to train highest(tier) available infantry troop. with a limit.

i will make a pull request after tests. if you want.
////////////////////////
i want also to know if the base code, your code is persistent to errors like the "network errors", "not online error" and any OtherException?
Thanks again.

лучше реализуй бота, который будет делать поиск монстров по заданным координатам. это очень полезно будет. и так же авто охоту на мобов каких я укажу. ну и так же по ресурсам как ты и написал.

a nodes/mines gathering feature.

I'm also working on the feature😄, rather than the watch tower search, I have an idea to traverse through all coordinate on the continent and then filter out everything we want(mine/goblin/...), what do you guys thinking?

a troops training feature.

It will be a nice to have feature, but in my opinion, I would make it optional, since not everyone need troops while farming.

i want also to know if the base code, your code is persistent to errors like the "network errors", "not online error" and any OtherException?

Actually I don't quite understand what you mean, basically all Exception defined in exceptions.py were catched in client.py, and it's just to make the code more readable.

I'm also working on the feature😄, rather than the watch tower search, I have an idea to traverse through all coordinate on the continent and then filter out everything we want(mine/goblin/...), what do you guys thinking?

Yes. i may be good because you will have an overview of the map and you will be able to filter and choose what you want.
however, i don't how you will traverse through all coordinate, but i think it will be like a brute force, right?
if so, it may be easily detected as bot. right?
how do you go through all coordinate?

It will be a nice to have feature, but in my opinion, I would make it optional, since not everyone need troops while farming.

ok. you're right. may be i will expose an argument/option to disable or configure these features. training, monster slaying, ...

Actually I don't quite understand what you mean, basically all Exception defined in exceptions.py were catched in client.py, and it's just to make the code more readable.

i mean some exceptions may stop a thread or the whole bot.

like one trying to claim a done research failed. and never claimed it again. may be the academy thread stopped???
DEBUG    | lokbot.client:post:73 - {"url": "kingdom/task/claim", "data": {"position": 5}, "elapsed": 0.514274, "res": {"result": false, "err": {"code": "not_online"}}}

and this one was because of network faillure for a few hours (i was out). and the bot stopped.
httpx.ConnectError: [Errno -2] Name or service not known

Do you know have a good idea to globally catch exceptions and recovery from this kind of failure???
Thanks again for the great job. :)

but i think it will be like a brute force, right?
if so, it may be easily detected as bot. right?

Yes, it is brute force, but we can have some priority strategy to do brute force.
As we know, the crystal mine only respawn on high-level lands, so we can sort lands by their level first, then calculate the distance between the land and our castle, and choose only nearest lands to search/traverse.

Do you know a good way to globally catch exceptions and recover from this kind of failure???

Those exceptions you mentioned, in my opinion, should not be recovered, but should be thrown, because it cannot be recovered since these exceptions are fatal(how can you recover from a network failure :P).

I think it may have a limited research to do. only those that improve resource production and gathering.

Currently all research code are defined here:
https://github.com/hldh214/lok_bot/blob/3a064baf059031afa6f85e279c0b0150ac61ae40/lokbot/enum.py#L154
And if you want to skip some category(for example: battle), you can modify code here, make a little if-condition to skip when the category_name is the one you don't like.
https://github.com/hldh214/lok_bot/blob/3a064baf059031afa6f85e279c0b0150ac61ae40/lokbot/farmer.py#L462-L464

As we know, the crystal mine only respawn on high-level lands, so we can sort lands by their level first, then calculate the distance between the land and our castle, and choose only nearest lands to search/traverse.

ok. you are right.
may be combining the time of respawn (hourly) and land filtering will reduce the probability of bot detection.

Those exceptions you mentioned, in my opinion, should not be recovered, but should be thrown, because it cannot be recovered since these exceptions are fatal(how can you recover from a network failure :P).

What i think is that a bot should always recover.
Can you think on how to integrate a retry mechanism like in this link?
How to handle a connection error gracefully in requests

And if you want to skip some category(for example: battle), you can modify code here, make a little if-condition to skip when the category_name is the one you don't like.

ok. i just didn't want to touch you code. i wanted to add code/features without touching yours. to reduce conflicts when merging with you future releases.
Best regards.

a bot should always recover.

It's not always true IMHO, maybe the network failure example I made above is not appropriate, instead, we already have this retry mechanism powered by https://github.com/jd/tenacity:
https://github.com/hldh214/lok_bot/blob/3a064baf059031afa6f85e279c0b0150ac61ae40/lokbot/client.py#L31-L36
Let me use another example: NoAuthException, which I thought no way to recover from it, and same for NeedCaptchaException or not_online you mentioned above.

we already have this retry mechanism powered by https://github.com/jd/tenacity

oh yeah. right. i will look at tenacity, where and which params value were used.

Let me use another example: NoAuthException, which I thought no way to recover from it, and same for NeedCaptchaException or not_online you mentioned above.

yes you are right for the NoAuthException. I did not look at the captcha yet.
but i think the not online error is random and relaunch or restart after a few minutes may work. right?


Is makeNodeJsModule outdated on recent Node/NPM versions?

@dsalaza4 I've been struggling to update a package using recent versions of NodeJs (18+) and NPM (9+).
My main problem is that peer dependencies are not being found in the local registry created by makes: https://github.com/fluidattacks/makes/blob/e8292670f46604d44063a15b28799f8d9b3d1e9e/src/args/make-node-js-modules/builder.sh#L24
Well reading at NPM changelog I saw that now peer dependencies are being installed locally as well, not only linked (like it used to be in NPM < 6) causing this kind of problem:
<IP_ADDRESS> - - [15/Mar/2023 17:12:15] code 404, message File not found
<IP_ADDRESS> - - [15/Mar/2023 17:12:15] "GET /react-side-effect HTTP/1.1" 404 -
<IP_ADDRESS> - - [15/Mar/2023 17:12:15] code 404, message File not found
<IP_ADDRESS> - - [15/Mar/2023 17:12:15] "GET /ansi-regex HTTP/1.1" 404 -
<IP_ADDRESS> - - [15/Mar/2023 17:12:15] code 404, message File not found
<IP_ADDRESS> - - [15/Mar/2023 17:12:15] "GET /ansi-regex HTTP/1.1" 404 -
npm ERR! code E404
npm ERR! 404 File not found - GET http://<IP_ADDRESS>:17426/react
npm ERR! 404
npm ERR! 404<EMAIL_ADDRESS>is not in this registry.
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

I tried to modify the codebase by removing the --registry flag and the build was successful but the modules are not being cached, causing it to do a rebuild here every time.
I've been looking for different solutions but I cannot find a way to deal with the peer dependencies without making the job to point to the remote registry. I've been reading about node2nix but it seems to not work with NodeJs 18+ yet.
Any suggestions here?
@dacevedo12 suggested taking a look at dream2nix.

This is also related to why sometimes you have to build a package-lock.json using the --legacy-peer-deps option, which removes the installation of peer dependencies.

If there are ways of integrating *2nix solutions so they do the heavy-lifting when it comes to providing specific lang packages, let's do that then!

Same problem trying to build an application.

Any solution?

We're using node2nix now.
Only node 18 is supported.
Everything in universe builds properly with this approach.
dream2nix looks pretty much abandoned.
node2nix does not look very well maintained, we'll see how we like using it.
Maybe we'll end up deprecating makeNodeJsModules and makeNodeJsEnvironment as building node_modules via nix derivations is just too complex.


Add new "Mobile" breakpoint

Description
Currently, the minimum number of columns we render in the "Grid" view is 6, this is too many for a mobile view. Anything on mobile (480px and below) we should be rendering a max of 3 columns.
Have you provided an initial effort estimate for this issue?
I have provided an initial effort estimate

Going by the Bootstrap model it's worth us introducing the <576px point


Lack of "move mySprite with buttons" block in "Throw a Bone"

Describe the bug
Lack of "move mySprite with buttons" block in "Throw a Bone"
Steps to reproduce the behavior
1.Navigate to https://arcade.makecode.com/beta#
2.Game Design Concepts---->Throw a Bone----->Start Tutorial
3.Click step 7
Expect behavior
There needs to add "move mySprite with buttons" block
Actual behavior
Lack of "move mySprite with buttons" block in "Throw a Bone"
Instructions:

Toolbox:

Additional context
1.OS: Windows(rs6)
2.arcade version: 0.18.7
3.Microsoft MakeCode version: 6.1.22

If this is a documentation issue or code?

I think just bad wording on my part - that portions a suggested follow up to the tutorial, each of these game design concepts end with a suggestion or two of opened ended things to try


Support multi-word salutations (like Air Commodore)

The current implementation does not support salutations of more than one word.
Air commodore is an English salutation that consists of more than one word.
What would it take to support more complex salutations?

Hi @rvanlaak. Interesting request. Tbh, i feel this is a quite uncommon case that i would almost consider out of scope. A lot of military ranks are composed of several words, but i would not necessarily consider them salutations in the sense of common language.
That said, i just had a little ponder about how tricky it would be to implement this and i could imagine that the parser could acquire some sort of rules for splitting the string from the language files. I can't currently fully imagine how that would work, but can imagine it far enough to think it could work, maybe even by accepting a list of strings that should be treated as a single part.
Can you think of more use cases for this than just this one?

I locally did get a prototype working that checks the salutations with more than one word if the other ones could not get found.
What I would suggest is to change the $parts array into its own value object so more logic can get encapsulated over there.
The parser itself in my opinion should not be responsible of this, the SalutationMapper should change the parts if needed.

Out of curiosity, what kind of logic would you like to encapsulate in the $parts object?


Updated integritycheck documentation to point to Trusted Signing Service

What Changed

Azure Code Signing was renamed to Trusted Signing service
Trusted Signing Service GA'ed and their documentation is now available. Replaced the link to point to their official documentation

@TylerMSFT

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


0.9.0 Release

I was holding off on setting up the 0.9.0 release, since there were several critical bug fixes in the pipeline.
Looks like that's all sorted now, so just checking if there's anything I should block on before kicking off this release?

No, kick it off :)

Only thought I have - do we care that we don't have docs for HashedTokenRouter  in our guides? Or is that more of an experiment that may eventually end up being the TokenRouter? In which case, better to leave it undocumented until testing is complete. I wasn't quite sure.

@Jake-Shadle with #965 merged, https://googleforgames.github.io/quilkin/main/book/services/xds.html#supported-apis is now out of date, yes?
Tracking documentation changes that will need to be made before a release can be cut.

Now https://github.com/googleforgames/quilkin/pull/980 is merged, just waiting on #970 doc updates.

Need these two implemented/merged before release. Everthing above is done/obsolete.

[ ] https://github.com/googleforgames/quilkin/pull/982
[ ] https://github.com/googleforgames/quilkin/issues/983

Okay, we have everything that was blocking release merged now! Any final objections before I kick this off @XAMPPRocky @Jake-Shadle ?

Nope

No

Release 0.9.0
Prerequisites

[ ] Have at least Editor level access to quilkin Google Cloud project.
[ ] Local gcloud configuration is pointing at the quilkin Google Cloud project.
[ ] Local git remote upstream points at git@github.com:googleforgames/quilkin.git.

Steps

[ ] Review that closed issues have appropriate tags for the changelog.
[ ] Review that merged PRs have appropriate tags for the changelog.
[ ] Run git remote update && git checkout main && git reset --hard upstream/main to ensure your code is in line
with upstream.
[ ] Update Cargo version for release

[ ] Edit the version field in ./Cargo.toml and remove the -dev suffix.
[ ] Edit the quilkin-macros dependency in ./Cargo.toml and remove the -dev suffix.
[ ] Edit the version field in ./macros/Cargo.toml and remove the -dev suffix.

[ ] cd to ./build/release and run make to submit the cloud build
[ ] Download all the artifacts from the cloud build.
[ ] Review license.html to ensure that there aren't any new MPL, or CDDL dependencies from the last
release. If there are:

[ ] Add the dependencies to archive_dependencies.sh
so that the source is archived in the container image.
[ ] Reset checklist back to "run make to submit the cloud build", and start from there again.

[ ] Run cd macros && cargo publish --dry-run --allow-dirty and ensure there are no issues.
[ ] Run cargo clippy in the root directory, and ensure there are no issues.
[ ] Add a release item to README.md "Documentation" > "Releases" list with related links in reverse chronological
order.
[ ] Review any data-proofer-ignore attributes from links in the documentation in ./docs, and remove any no
longer needed.
[ ] Create a draft Github release

[ ] Populate the tag with v0.9.0
[ ] Click Generate release notes to generate the change log for this release.
[ ] Copy the release notes from the draft release and paste it at the top of CHANGELOG.md.
[ ] Using the
Github release template
update the generated release notes with a description, and relevant changelog sections.
[ ] Attach all the cloud build artifacts to the draft GitHub release.

[ ] Submit these changes as a PR, and merge with approval.
[ ] Run git remote update && git checkout main && git reset --hard upstream/main to ensure your code is in line
with upstream.
[ ] Run git checkout -b release-0.9.0 && git push upstream release-0.9.0 to create a release branch.
[ ] Publish to crates.io/crates/quilkin-macros: run cd macros && cargo publish
[ ] Publish to crates.io/crates/quilkin: run cargo publish
[ ] Submit the release.
[ ] Post announcemnts

[ ] mailing list.
[ ] Discord #announcement
[ ] Twitter account.

[ ] Update Cargo version for development

[ ] Edit Cargo.toml and increment the minor version and apply the -dev suffix to the
version.
[ ] Edit the quilkin-macros dependency in ./Cargo.toml and increment the minor version
and apply the -dev suffix to the version.
[ ] Edit the version field in ./macros/Cargo.tomland increment the minor version
and apply the -dev suffix to the version.

[ ] Submit this change as a PR, and merge with approval.

Congratulation! 🎉 You have successfully released Quilkin!


quick search: including "county" should work

Context
on our quick search, https://data-sources.pdap.io/, the suggested search is "complaints" in "allegheny county". however, when you make this search, 0 results are returned: https://data-sources.pdap.io/search/complaints/allegheny county
however, "complaints" in "allegheny" does work.
Requirements

searching for a county (like "allegheny county") should return the same results as regardless of whether the word "county" is included.
there are a variety of ways to handle this with varying degrees of sophistication. we could remove "county" from searches, or only check against the "county" field if "county" is present. or something else.

Tests

write a test with a few different counties to check this is working

Open questions

I can't think of other types of region people might type here...but if there are any we might want to consider those too.

https://github.com/Police-Data-Accessibility-Project/data-sources-app/pull/87/files


Max and min distances between planets/mazes and stations

Description

Adds max and min distances between planets/mazes/systems and stations. This makes it so that stations, including the one that the player spawns at, stay within a certain distance away from those structures.

Testing

The default distances that I chose are several hundred units away. However, you can change these, as they are set by constants in the GalaxyFiller class. A good way to test would be to change the min and max distances in the code to something smaller (30 for the min and 50 for the max work well), so that you can see how the object is closer to the initial point. Or you can take a look on the map with varying values for the max and min.

Notes

It might be interesting to allow the player to select values for the maximum and minimum distances. I'm not sure how this would fit in to the currently fairly simplistic options available to the player during game creation, however.

Can one of the admins please verify this patch?

@GooeyHub ok to test

Hooray Jenkins reported success with all tests good!

Hey there @RatMoleRat are you planning on cleaning this up for merge? Let me know :)

Unfortunately this PR is now incompatible. The repo has been rebased to remove the JREs from history (significantly reducing the size).
Please reclone for any further contributions.
Sorry for the inconvenience.


Relax the IP/domain filtering mechanism when someone makes a PR

Actually I'm not sure what was going on. idtoken: write is already present.


👌 IMPROVE: Support numref placeholders

This PR allows {number} and {name} to work if referenced in the same role while %s only works if referenced by itself. This seems to be the behavior of the general numref role. For instance, {number} {name} %s will display number and name while "%s" won't work. The same is true for {number} %s and {name} %s. However, if "%s" is introduced by itself, it will display the number of the exercise directive.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@63524aa). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #5   +/-   ##
=========================================
  Coverage          ?   96.80%
=========================================
  Files             ?        3
  Lines             ?      344
  Branches          ?        0
=========================================
  Hits              ?      333
  Misses            ?       11
  Partials          ?        0

Flag
Coverage Δ

#pytests
96.80% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 63524aa...efc37bf. Read the comment docs.


ClassNotFoundException - DSpaceRestApplication

Hi,
According to this commit -https://github.com/4Science/DSpace/commit/a0f88cd94ea146a9a5ee1c4bf6939e9495d655c3
class org.dspace.rest.DSpaceRestApplication has been removed but it is still referenced in the web.xml of dspace-rest project
https://github.com/4Science/DSpace/blob/dspace-5_x_x-cris/dspace-rest/src/main/webapp/WEB-INF/web.xml
which is why the project dspace-rest while being loaded into tomcat container is error-ing out with ClassNotFoundException - DSpaceRestApplication
Is that class not required anymore? Can we get rid of it from the web.xml?
Thanks,
Debashis

Ok thank you. I will fix it shortly.


Suggestions for Migration/Theming chapters in the docs

Hi! Thanks so much for providing this library. I just migrated our codebase from v5 → v6 and thought it would be useful to share some gotchas I ran into around theming:
It took me a long time to realize why modifying the fontWeight of my Button's baseStyle wasn't working: I had set height={40} in numerous instances of my app, and this property was overwriting my custom font weight from being applied. Going through my app and replacing height={40} with size="large" allowed my custom font weight to appear. There's only one line in the migration chapter that mentions this:

Additionally, we've unified the sizing API so SegmentedControl also accepts a size prop just like Button, TextInput, Group, etc.

I think that more attention should be brought to the size prop.
It's not clear to me when a rule should be placed in baseStyle vs when it should be placed in appearances.default. For example, this lets me set a background color on Select:
Select: {
  baseStyle: {
    background: 'linear-gradient(to bottom, #FFFFFF, #F4F5F7)',
    fontWeight: 500,
  },
  appearances: {
    default: {
      color: '#425a70',
    },
  },
},

But this does not:
Select: {
  baseStyle: {
    fontWeight: 500,
  },
  appearances: {
    default: {
      background: 'linear-gradient(to bottom, #FFFFFF, #F4F5F7)',
      color: '#425a70',
    },
  },
},

The Theming API Overview section is ambiguous in detail: "Note that some components don't have this." Which ones?
It was helpful for me to log classicTheme to console to know where to extend rules. It would be helpful if more details about the shape were present in the docs rather than repeating /* ... */.
It's much easier to extend a theme with deepmerge rather than spreading objects over and over:
const myTheme = {
  components: {
    Button: {
      appearances: {
        superdanger: {
          background: 'indianred',
        },
      },
    },
  },
}

const extendedTheme = merge(classicTheme, myTheme)
// console.log(extendedTheme)

export default function MyThemeProvider(props: Props): JSX.Element {
  return <ThemeProvider value={extendedTheme}>{props.children}</ThemeProvider>
}

That's all for now, thanks again for a relatively painless upgrade!

@jasonbarry This is all super helpful -- deepmerge especially is helpful. Might be worth us exporting extendTheme() that effectively does that under the hood.
If you have more examples / ideas for theming docs, please don't hesitate to submit a PR - we're open for ideas! Theming is a new concept for us in EG. Going to close this for now - definitely top of mind.


What do you do when these steps don't work?

What do you do when these steps don't work?
So following these steps, get to the ConfigMgr wizard step and it's grayed out? Running latest SCCM 1802 with latest KB but am stuck. I have a support call open with Micrsosoft and they too are also stumped?!
I'm looking to just do co-management with domain joined devices so haven't configured the Gateway etc.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 07e9b780-6cad-d3f3-d0f2-e13a577890cc
Version Independent ID: 18c7f0aa-884a-349b-e416-74d02adc4474
Content: Co-management for Windows 10 devices - Configuration Manager
Content Source: sccm/core/clients/manage/co-management-overview.md
Product: configuration-manager
GitHub Login: @aczechowski
Microsoft Alias: aaroncz

Thanks for taking the time to share this with the Configuration Manager documentation team. I’m sorry that you’re having problems, and I want to make sure it gets to the right people that can help. There are a couple of options available to provide the best level of support for this:

Post on Configuration Manager forums. These are moderated by our community MVPs who are very experienced,   knowledgeable, and helpful.

Use Microsoft Support to search for the issue, or open a case with professional support.

Contact your Microsoft Premier team, such as a Premier Field Engineer or Technical Account Manager.

I'm closing this issue as there's nothing actionable for docs at this time. If you have feedback on the contents of the article, please reopen this issue.

I'm sorry @aczechowski but there is clearly a missing piece to the documentation that should be added... It has to do with permissions!
It turns out the SCCM console in build 1802 (with or without the latest KB) has been changed so the the co-management feature is only available (not greyed out) to an 'Full Administrator' with ALL security scopes. The Default security scope is not enough, but no where in the documentation you have here have you suggested you need to be a 'Full Administrator' on all security scopes. This is contrary to all other Cloud services where the Default scope is enough and 1709 where this wasn't an issue.
I'd recommend that you add at least a note to this documentation to say you must be an administrator to all security scopes. I'm also not the only person with this issue:
https://configurationmanager.uservoice.com/forums/300492-ideas/suggestions/33766552-unable-configure-co-management-grayed-out-on-1802
I'll make a comment on this to help people get co-management enabled.
Quite disappointing that this was just closed with little to no care. Clearly the instructions to do co-management are not clear enough for people to get this enabled.

Thank you @TrentSteenholdt for providing further details so that I can understand this is not just a support request, but something specific that we need to include in the documentation. I've clarified the title and reopened the issue. I confirmed with our engineering team that these permissions are required.
Regarding your final comment, I'll also note: the Microsoft Open Source Code of Conduct, which outlines the expectations for community interactions in and around docs.microsoft.com, is designed to help "provide a welcoming and inspiring community for all." As I stated earlier, I'm sorry that you are having difficulties with this feature, and I want to improve the documentation to help you and others be successful, but this is not a support forum. I'm sorry if you had different expectations.

I fixed this while adding new content for 1806. The updates for this article should publish live later today around midday. I'm closing this issue.


Resolve backport/backport 4560 to 1.3

Description
Resolves backport issue
Issues Resolved
List any issues this PR will resolve, e.g. Closes [...].
Checklist

[ ] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

1.3 handles the header a little differently than the 2.x line, so I'm going to hold off on merging this until we can figure out how to make it work.

This can be closed since we are getting new updates that will replace this nav pane.


📝 Docs: Add docs for useLayer composable

useLayer is evolving composable and started as an experiment. As this is used as a base for various components and provides greater flexibility even at the component level it now requires its own docs.

useLayer hook is get computed classes and styles by some params?
But I think the name of useLayer itself is not very intuitive?

Thanks for your suggestion.
Yes, naming is really hard 💯
I finally decided to use  "layer" as a name because it returns the styles for the background & text on it. Basically creating a layer. This is similar to Paper in MUI & Sheets in vuetify.
What is your suggestion?


4.0: Integrate thumbnail generation into the software

As a user, I want to be able to generate my thumbnail from the current data in TSH by clicking a button in the TSH UI
Additionally, I want to be able to configure the following parameters in the UI: (Which are already configurable in the thumbnail script)

Import a custom foreground image
Import a custom background image
Import a custom icon
Set if the generated thumbnail should use team names
Set if the generated thumbnail should display sponsor names
Set if the generated thumbnail should display the phase name
Set the font used for the thumbnail (If possible based on system fonts, if not based on a list of free-to-use fonts)

I'm no python dev but I have some basics with it ! Just wanted to say that I'm currently trying to integrate it as you said, with settings

I would follow the path:
1- Download QT Designer, create a base .UI file similar to others in the program
2- Create it's own Widget file/class, put it in a new tab
3- Implement settings and a button that calls the function

For now, I just create "manually" a button in ScoreBoardWidget that call a "generate()" method. So I had to copy/move the original script to /src so I can import the script.

May I make a PR just for the button or it's better to wait for settings tab ?

@joaorb64 I'll let you decide on that one, personally I'd at least move the button into a new tab, but I can also see it working better on the ScoreBoardWidget with later a settings button that pops open a settings window from a user experience perspective

Yo! I think it's fine to have the button over there, also because it uses the data from the scoreboard. Seems good to me!
Later you could add a button with a cog icon like the others that opens all the settings! And maybe use SettingsManager to save the settings (probably name them all using SomeBaseName_Setting1?)
You may submit the changes :)

Just be careful when you do the PR though! I've made some edits to the thumbnail script to add some features today which will make it easier to add settings later ;-)

I made a PR, there is other thing to edit like testing if there is enough data (name, characters etc) and maybe renaming the output file
And then the settings :)
thanks for your answer !

I've left a few comments on your PR! I'll try and test the build itself tomorrow if I can :-)

Soon, I may change it to a window and a button but now it's in a Tab 😅
Don't mind custom icon 😬

That looks pretty good! Does it save those settings in user_data/settings.json (or another file in user_fata) so it reloads when the program is closed?

Indeed ! I pushed it on https://github.com/joaorb64/TournamentStreamHelper/pull/64, it's saved in user_data/settings.json on thumbnail {...}, and when you change a value, it also change the preview

On a side note, if that can help I might have found a solution we can get inspiration from in order to select a font from the fonts installed on the computer, but I'm not sure
Ah nice ! I'll save it for later !

Also, while I think about it, can you please move the scripts/thumbnail_base directory to the assets directory? This way we don't have to store assets in the scripts directory

thumbnail_base moved !

Wow, this looks really, really good!
Excited with this PR! Awesome work both of you :D

I don't know how to make this apparent, but the user should probably have the opportunity to pick the asset pack used for the thumbnail generator among those he has installed in the game? Not every user will have the full asset pack for their game after all, and some users might just prefer to do, say Smash Bros. murals over the renders
In terms of options in the list we can probably reuse the asset packs' names as defined in the config.json file
As for what should be default, I think going for full and then base_files/icon if full doesn't exist would be a good way to do it

Using QFont and ImageFont from PIL doesn't seem to be easy.
ImageFont need a path to the font, but QFont doesn't have it.
After some search, I found this post
So maybe using QImage is a solution ?

I'm afraid using QImage might require a lot of rewrites on the script itself ^^ That being said, doesn't QStandardPaths.standardLocations(QStandardPaths.FontsLocation) give the absolute path to installed fonts?

Yeah I'm afraid too ! 😬
For QStandardPaths.standardLocations(QStandardPaths.FontsLocation) I'll give it another chance, I tried it but don't remember how it went 😅

You might need the PySide library honestly, I'll look into it on my end as well

OK so I've installed PySide using pip and then used the code in the second answer of this post. I was able to access the system folder for fonts that way, however on Windows 10 it was missing the user installed fonts in %localappdata%\Microsoft\Windows\Fonts. Still, we could just append that value to the directory list if the system is Windows and the directory exists as a workaround (Using font_paths.append(f"{os.getenv('LOCALAPPDATA')}\\Microsoft\\Windows\\Fonts")

Looks good then !
Other problem is how to select fonts via the UI ?
Via a ComboBox I think but if we can only show path or "the_font.ttf"
There are already font widget like QFontCombobox or QFontDialog but only return font family (and not the file) and we don't want that :/

Maybe is there any library to get the system fonts from the font family name and use the Qt ready stuff?

The adapted version would look something like this:
from PySide2.QtCore import QStandardPaths
from PySide2.QtGui import QFontDatabase
from PySide2.QtWidgets import QApplication
import sys, os

def getFontPaths():
    font_paths = QStandardPaths.standardLocations(QStandardPaths.FontsLocation)
    if sys.platform == "win32":
        font_paths.append(f"{os.getenv('LOCALAPPDATA')}\\Microsoft\\Windows\\Fonts")

    accounted = []
    unloadable = []
    family_to_path = {}

    db = QFontDatabase()
    for fpath in font_paths:  # go through all font paths
        for filename in os.listdir(fpath):  # go through all files at each path
            path = os.path.join(fpath, filename)

            idx = db.addApplicationFont(path)  # add font path

            if idx < 0: unloadable.append(path)  # font wasn't loaded if idx is -1
            else:
                names = db.applicationFontFamilies(idx)  # load back font family name

                for n in names:
                    if n in family_to_path:
                        accounted.append((n, path))
                    else:
                        family_to_path[n] = path
                # this isn't a 1:1 mapping, for example
                # 'C:/Windows/Fonts/HTOWERT.TTF' (regular) and
                # 'C:/Windows/Fonts/HTOWERTI.TTF' (italic) are different
                # but applicationFontFamilies will return 'High Tower Text' for both
    return unloadable, family_to_path, accounted

app = QApplication()
unloadable, family_to_path, accounted = getFontPaths()

And then family_to_path.keys() gives us the list of fonts

If I can find a way to get the path from a QFont element, I will update you all ^^

OK, with further testing, the script above only picked the first variation it found of a font, so it sometimes resulted in the font returned not being bold. In order to alleviate that, I edited the script to associate a font name to the list of paths corresponding to fonts with that name (Which thus kinda would serve as a style select)
from PySide2.QtCore import QStandardPaths
from PySide2.QtGui import QFontDatabase, QFont, QFontInfo
from PySide2.QtWidgets import QApplication
import sys, os, json

def getFontPaths():
    font_paths = QStandardPaths.standardLocations(QStandardPaths.FontsLocation)
    if sys.platform == "win32":
        font_paths.append(f"{os.getenv('LOCALAPPDATA')}\\Microsoft\\Windows\\Fonts")

    unloadable = []
    family_to_path = {}

    db = QFontDatabase()
    for fpath in font_paths:  # go through all font paths
        for filename in os.listdir(fpath):  # go through all files at each path
            path = os.path.join(fpath, filename)

            idx = db.addApplicationFont(path)  # add font path

            if idx < 0: unloadable.append(path)  # font wasn't loaded if idx is -1
            else:
                names = db.applicationFontFamilies(idx)  # load back font family name

                for n in names:
                    if n in family_to_path:
                        family_to_path[n].append(path)
                    else:
                        family_to_path[n] = [path]
                # this isn't a 1:1 mapping, for example
                # 'C:/Windows/Fonts/HTOWERT.TTF' (regular) and
                # 'C:/Windows/Fonts/HTOWERTI.TTF' (italic) are different
                # but applicationFontFamilies will return 'High Tower Text' for both
    return unloadable, family_to_path

An example of call would be:
unloadable, family_to_path = getFontPaths()
print(family_to_path["SimSun"])

With the result of family_to_path["SimSun"] being the list of fonts called SimSun

This is the best way I found to get the list system fonts while keeping PIL in the backend ^^ (Default should still probably be the copy of OpenSans present in assets though)

Wow ! Thanks for your research and test !
So there isn't a way to have another name for fonts family that has multiple path ? like :
{ 'Agency FB': ['C:/Windows/Fonts\AGENCYB.TTF', 'C:/Windows/Fonts\AGENCYR.TTF'], ... }
will only give 'Agency FB'

Yeah, I tried looking for a way to differentiate bold and regular but without having to go deep into QFont I don't think there is, so I think the best way is to just get the list of fonts with the same name and then separate them as something like "Type 1", "Type 2" etc.
Given that the preview on the right also autoupdates upon select, it shouldn't be that bad from a user perspective ^^

I see, I'll try something

Also technically speaking there is no direct user agency on text sizes given that the text in the image is always set to fit in a rectangle, so that helps in reducing the number of options

It was my next question ^^
Ok looks good to me, maybe adding the file name after Type X so user can guess ?

It might get a little cluttered with the filename honestly but I guess it could work if you can dynamically display the filename (not file path) underneath the selection
Speaking of, since we technically have two fonts for the thumbnail (One for the player text and one for the match info), maybe it would be better to have the type menu as a smaller menu on the right of the font menu, but on the same line? (Could also just display as 1 and 2 if so) Food for thought

Still have some thing to do but it works kinda ! I'll see that tomorrow ^^' (python is cool)

I don't know how to make this apparent, but the user should probably have the opportunity to pick the asset pack used for the thumbnail generator among those he has installed in the game? Not every user will have the full asset pack for their game after all, and some users might just prefer to do, say Smash Bros. murals over the renders
In terms of options in the list we can probably reuse the asset packs' names as defined in the config.json file
As for what should be default, I think going for full and then base_files/icon if full doesn't exist would be a good way to do it

Is the next thing to add, I almost forgot about it

I think having the asset pack choice below the generate button makes sense since the asset packs available will depend on the selected game

Are there other asset pack that has subfolder like base_files ? Or is it the only one ?
Just need to edit the script to use the asset pack saved in settings !

I guess theoretically they could? I haven't tested though but if you have a prefix with a /, if the program is robust enough you'd have the config in the root of the folder and a subfolder with the assets
I'm not gonna test though ^^

On a side note, since, again, installed asset packs depend on the game, make sure to save the used asset pack on a per-game basis ;-)

On a side note, since, again, installed asset packs depend on the game, make sure to save the used asset pack on a per-game basis ;-)

For the moment, I have a QComboBox where I add TSHGameAssetManager.instance.selectedGame.get("assets") item, that contains only installed asset pack for the current game selected. If you change game, it also change the asset combo box.
So I think it's good for this point !

Neat! I think then the last parameter to add then is the width and color of the separators ^^

BTW I'm out of town this week-end so I'll test the new commits when I get back

aaand here we are !
Asset can be choose just under the Generate thumbnail button and separators settings are here !

The only thing left I think about is having an option to mirror the asset for player 2, other than that, I think this issue is done with https://github.com/joaorb64/TournamentStreamHelper/pull/64 (need test of course)

I've been able to test the UI, and I just have some remarks:

I think it'd be more user friendly if the asset selection menu displayed the name of the asset pack as available in config.json rather than the directory name
There is a small issue with handling of the Losers box (I'll comment a fix on the PR)
Other than that it looks to me like it's ready to be released! Nice work!

if the items in TSHGameAssetManager.selectedGame has a name, the menu will display it !
and I edit with your fix
Thanks you !

I'll try to test it today, I believe from the three of us I'm the only one using Linux?
I love everything about this feature and this thread overall, really really happy with the results and by the fact that you guys are contributing! Thanks for your hard work!

I think yes ^^
If you are worried about fonts, I think it's good with what Wolfy did to get all system font :
font_paths = QStandardPaths.standardLocations(QStandardPaths.FontsLocation)
if sys.platform == "win32":
     font_paths.append(f"{os.getenv('LOCALAPPDATA')}\\Microsoft\\Windows\\Fonts")

at least I hope so !
You're welcome ! It's also a pleasure to contribute to your work ! (and learning more about Python haha)

I indeed believe you're the only one of us three who's on Linux @joaorb64 ^^
On a side note @Kekwel can you add pyside to dependencies/requirements.txt just in case please?

Closing the issue as the thumbnail feature is now integrated in TSH and released