TypeScript language server does not highlight the @satisfies keyword in JSDoc.

TypeScript Version: 5.3.3 (selected from workspace)

With TypeScript v5.4.2 in VS Code, with this JSDoc:
/** @satisfies {Record.<SupportedShell, String>} */

The JSDoc tag @satisfies is syntax highlighted:

So this issue can be closed?


Capital City Report 2

All the capital cities in a continent organised by largest population to smallest.
Has to display: Name, Country, District, Population

done


fix: downgrade instance synth order issues to warnings

Motivated by a user's question, this reduces the severity of the "cannot find synthesization order" message from an error to a warning. This isn't a hard error, so having it be a warning communicates that elaboration of the instance command indeed completes relatively normally.
It also adds a little bit of explanation about how the instance will likely be inapplicable, and it mentions the synthInstance.checkSynthOrder option.

This isn't a hard error

Just checking, do you think this was intentional and should stay that way?

Triage team agrees: making this a hard error seems like the safer option

Just checking, do you think this was intentional and should stay that way?

I am not sure if this question was aimed at me or @kmill.  I don't have a preference - ultimately, I was just confused when I saw "error: ..." but the code still ran.

@Kha Should we leave the synthInstance.checkSynthOrder option? My thoughts are

if we have the option, then since the check can be turned off, it's clearly safe to proceed, so we should log a warning like in this PR; but
if we want it to be a hard error due to safety, then we shouldn't be able to proceed without the check.

I mostly agree with you, but this is not the only error that can be turned off. set_option hygiene is another one that comes to mind.


Phish link

ctt.pt-particulares.mandalozz.com
Newly Registered Domain.
👇 The phish email it came with.
https://www.reddit.com/r/Scams/comments/gfqm9a/portuguese_ctt_pts_snail_mail_service_phishscam/

@romaincointepas This might be of interest to you too.

Thank you @CyanoTex.  This has been added.


[Q] About hosting arena-hard-v0.1/question.json in the Hugging Face Hub

Description
Hi here! Awesome job with Arena Hard 👏🏻 I just opened this issue since we, at @argilla-io, are currently exploring the usage of distilabel for running benchmarks such as Arena Hard, and we were wondering if it's a problem or if you have any issue with https://huggingface.co/datasets/alvarobartt/lmsys-arena-hard-v0.1 being hosted in the Hugging Face Hub on our end.
We uploaded it there because we couldn't find the dataset per se in the Hub, but could only find https://huggingface.co/spaces/lmsys/arena-hard-browser with the question.jsonl file there.
If there's already a dataset or you'd like us to transfer that to your org we'll happily do so, hope this is not an issue, but just let us know!
Congrats for the awesome job on evaluating LLMs again!

Hi friend! We love to hear that people find this benchmark useful. We would like to host arena hard auto questions on our own huggingface. We are releasing a new set of the prompts soon and will continuously releasing new and new version for the community so it would be easier for us to maintain it on our huggingface. Thanks!

Hey here! That's great to hear, then I'll keep that private for me to test and wait until you release yours to point our examples there, thanks for the clarification!

Hi here @CodingWithTim! I've just re-opened this issue to ask about the progress of hosting the dataset within the https://huggingface.co/lmsys organization on the Hugging Face Hub.
Is there any short-mid term plan on doing so? If that's the case, we would be happy to help if needed!

@alvarobartt Thanks for letting us know! I just pushed the dataset to LMSYS huggingface link! Glad to see this being helpful to folks! Feel free to give us feedback anytime!


Grossmont Union High School District

I'm applying for an educational license for the Grossmont Union High School District, located in the San Diego Area. As a junior in this district, I'm hoping to use JetBrains products to help in my workflow for our new Computer Science classes!

@TheCoder11 Pull request merged. Thank you!


Add API interface for user log entries

@kmart2234 in automating some of our stuff, we were grepping the user-detail page in pagerduty which uses this API. Looks like an undocumented endpoint /shrug


end missing for function set_mapping

https://github.com/ray-x/navigator.lua/blob/ae251289b34bc970adc74aba8859f422e65c52e4/lua/navigator/lspclient/mapping.lua#L275

thanks. it was fixed


Retry more commands in apt addon

At TravisCI of ruby/ruby repository, we often see random failures on apt plugin, but we do not want to see CI failure on that part.
I'd like to suggest introducing retries for the following failures:
sudo -E apt-add-repository -y ...

May 26: https://travis-ci.org/ruby/ruby/jobs/537363191
May 31: https://travis-ci.org/ruby/ruby/jobs/539694628

sudo -E apt-get -yq --no-install-suggests --no-install-recommends $(travis_apt_get_options) install ...

May 26: https://travis-ci.org/ruby/ruby/jobs/537354169
May 26: https://travis-ci.org/ruby/ruby/jobs/537354958
May 31: https://travis-ci.org/ruby/ruby/jobs/539259851

Hello there.
We've tried retry logic earlier with #600. This turned out to be rather problematic; the failures with the apt command tends to leave the system in an inconsistent state, from which the identical command cannot continue. There are subsequent attempts such as #605 and #1343, but the issue remains.

FYI we recently (last weekend) enabled an Apt Caching Proxy for all of .org and .com, this should improve Apt performance and reliability.

Hello, thank you for your responses!

We've tried retry logic earlier with #600. This turned out to be rather problematic; the failures with the apt command tends to leave the system in an inconsistent state, from which the identical command cannot continue. There are subsequent attempts such as #605 and #1343, but the issue remains.

I see. Sorry for duplicating the history. So retrying apt-get ... install ... may not be a good idea.
How about apt-add-repository ... one? Have you ever also retried that and experienced the similar issue, or do you think the underlying cause is the same and it's likely to trigger a similar problem?

FYI we recently (last weekend) enabled an Apt Caching Proxy for all of .org and .com, this should improve Apt performance and reliability.

That's a good news :)  I'll monitor the reliability for a while.

We hit a failure on apt-get install again https://travis-ci.org/ruby/ruby/jobs/547497336. Is the "Apt Caching Proxy" stale now? Is there anything we can do to make the CI stable?

Hi @k0kubun , this is really odd and really interesting. It doesn't look like network connectivity but instead the APT server reporting that no item exists.
Our APT caching will drop items after 3 hours, but then download them again and stream them to the client, so I don't think it is the issue.
I do wonder if we should add the retries logic just to the APT addon.
@BanzaiMan thoughts?


passThrough() not applied when fetching local files

When using fetch to get a local file, mirage throws a no route defined to handle this request even is passThrough() is used in config.js.
The potential solutions suggested in this Stackoverflow question didn't work.
I have reproduced the error here. Unfortunately, Ember Twiddle didn't seem to work for fetching local files, so I had to create a new app to reproduce the error. The app simply fetches a markdown file and renders it in a component.
All of the potential solutions from the Stackoverflow question are commented out in config.js.

Sorry about the confusion!
passthrough() is such a weird API that we intend to improve in the near future. In the meantime...
The reason you're running into this is because passthrough() is being called after urlPrefix and namespace are being set, and so the passthrough call is taking those into account.
The solution is to either move the passthrough call to the top of the config file (before you set namespace and urlPrefix, or to reset the namespace/urlPrefix before calling passthrough again
this.urlPrefix = ''
this.namespace = ''
this.passthrough() // this should work

Closing bc I tested on your repo and it seems to work, let me know if you run into anything else!

The above worked great! Is there any way to pass through all external requests?
For example, I know that I can use this.passthrough('https://example.com') to make Mirage ignore requests to http://example.com or this.passthrough('https://example.com/**') to make Mirage ignore requests to http://example.com/foo or http://example.com/bar.
With
this.urlPrefix = ''
this.namespace = ''
this.passthrough()

Mirage still intercepts the requests to external urls.

Right now if you use the normal form of passthrough you have to whitelist each external domain. But there's a function form you can try, something like:
this.passthrough(request => {
  let isExternal = request.url.startsWith('http')

  return isExternal
});

Great! This code has worked for both use cases:
  this.urlPrefix = ''
  this.namespace = ''
  this.passthrough(request => {
    let isExternal = request.url.startsWith('http')

    return isExternal
  });
  this.passthrough()

So requests to both /markdown/markdown-file.md and https://example.com are being passed through. Thank you so much.

Awesome glad to hear!
We have an API coming to Mirage, server.canHandle(requests) which will return a Bool if Mirage has a route handler defined for a particular url. Once that lands I think we'll make
this.passthrough(request => !this.canHandle(request))

the default for passthrough which should address a lot of issues similar to this one!


How to get the length of the video from Player or Controller?

You can access the duration of the currently playing Media using Player.state.duration or Player.streams.duration.
You can refer to our example application for code samples & other usage examples.
Close the issue if you feel your question has been answered.

@alexmercerind  Thanks for your response.


Could not set up API to Code Review site for Danger - Github Enterprise + Jenkins

Report
What did you do?
bundle exec danger
What did you expect to happen?
Running your Dangerfile and finding the CI
What happened instead?

Could not set up API to Code Review site for Danger

For your GitHub repo, you need to expose: DANGER_GITHUB_API_TOKEN
You may also need: DANGER_GITHUB_HOST, DANGER_GITHUB_API_BASE_URL
Found these keys in your ENV: JENKINS_HOME, MAIL, LANGUAGE, SSH_CLIENT, USER, RUN_CHANGES_DISPLAY_URL, SHLVL, NODE_LABELS, HUDSON_URL, OLDPWD, HOME, BUILD_URL, HUDSON_COOKIE, JENKINS_SERVER_COOKIE, WORKSPACE, DANGER_GITHUB_API_TOKEN, CHANGE_ID, LOGNAME, NODE_NAME, DANGER_GITHUB_API_BASE_URL, _, CHANGE_TARGET, EXECUTOR_NUMBER, XDG_SESSION_ID, BUILD_DISPLAY_NAME, HUDSON_HOME, JOB_BASE_NAME, PATH, BUILD_ID, XDG_RUNTIME_DIR, DANGER_GITHUB_HOST, BUILD_TAG, JENKINS_URL, ARTIFACTORY_KEY, LANG, JOB_URL, CHANGE_TITLE, BUILD_NUMBER, JENKINS_NODE_COOKIE, SHELL, CHANGE_AUTHOR_DISPLAY_NAME, RUN_DISPLAY_URL, CHANGE_AUTHOR, HUDSON_SERVER_COOKIE, JOB_DISPLAY_URL, JOB_NAME, PWD, SSH_CONNECTION, BRANCH_NAME, CHANGE_URL, RUBYOPT, RUBYLIB, GEM_PATH, GEM_HOME, BUNDLE_BIN_PATH, BUNDLE_GEMFILE, MANPATH.
Your Environment

Which CI are you running on? Jenkins

Are you running the latest version of Danger? Using danger 5.3.3

What is your Dangerfile?

rubocop.lint
commit_lint.check

Did you discover what was off?
On Jul 11, 2017 14:34, "Christian Mitchell"<EMAIL_ADDRESS>wrote:

Closed #858 https://github.com/danger/danger/issues/858.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/danger/danger/issues/858#event-1159360037, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAC_jqSdWx0AUnRMM5QrTB3yTxVIdWeBks5sM8BDgaJpZM4ORQYB
.

We did something like this in a Jenkinsfile:
withEnv([
        "DANGER_GITHUB_HOST='https://github.company.com'",
        "DANGER_GITHUB_API_BASE_URL='https://github.company.com/api/v3/'"
      ]) {
        sh "bundle exec danger"
      }

If you single-tick the ENV values, things don't work. It needs to look like this instead:
withEnv([
        "DANGER_GITHUB_HOST=https://github.company.com",
        "DANGER_GITHUB_API_BASE_URL=https://github.company.com/api/v3/"
      ]) {
        sh "bundle exec danger"
      }


Distance rates - Inconsistency in rate format in confirmation page vs workspace editor page

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.77-0
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4581663
Issue reported by: Applause - Internal Team
Action Performed:

Go to staging.new.expensify.com
Go to workspace settings
Go to Distance rates
Create a few distance rates with different decimal places (1, 1,1, 1,11)
Note that the distance rates are always shown in three decimal places
Go to workspace chat > + > Submit expense
Go to Distance and add waypoints
Proceed to confirmation page
Click Distance rate

Expected Result:
Distance rates in confirmation page should be shown in the same format as the distance rates in workspace settings.
The rates should be in 3 decimal places in confirmation page
Actual Result:
The distance rates in the confirmation page always omit the ending zero
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/Expensify/App/assets/78819774/cc7406b1-7d98-43e0-9f96-971764f1e71c
View all open jobs on GitHub

Proposal
Please re-state the problem that we are trying to solve in this issue.
The alternate text in the step rate page is not the same format as in the workspace rates
What is the root cause of that problem?

the way we render the milage rate is different in both the PolicyDistanceRatesPage and the IOURequestStepDistanceRate
in the PolicyDistanceRatesPage we use CurrencyUtils.convertAmountToDisplayString(value.rate, value.currency ?? CONST.CURRENCY.USD)}
and in the IOURequestStepDistanceRate we use DistanceRequestUtils.getRateForDisplay

What changes do you think we should make in order to solve the problem?

either to unifiy both of them to use convertAmountToDisplayString or the DistanceRequestUtils.getRateForDisplay
or we can just change the CUSTOM_UNIT_RATE_BASE_OFFSET
to 100.0 instead of 100 the same way as in the convertAmountToDisplayString here

Proposal
Please re-state the problem that we are trying to solve in this issue.
The distance rate shown in workspace distance rate page and in money request confirmation distance rate has different format.
What is the root cause of that problem?
They have different logic to format the rate, currency, and unit.
For workspace distance rate page, we use CurrencyUtils.convertAmountToDisplayString to format the currency and amount.
https://github.com/Expensify/App/blob/dc17b1b1f2a1c9fa2c375ea3db0f4ef804ed0ac0/src/pages/workspace/distanceRates/PolicyDistanceRatesPage.tsx#L105-L107
For money request distance rate page, we use DistanceRequestUtils.getRateForDisplay
https://github.com/Expensify/App/blob/dc17b1b1f2a1c9fa2c375ea3db0f4ef804ed0ac0/src/pages/iou/request/step/IOURequestStepDistanceRate.tsx#L65
https://github.com/Expensify/App/blob/dc17b1b1f2a1c9fa2c375ea3db0f4ef804ed0ac0/src/libs/DistanceRequestUtils.ts#L116-L130
The amount is taken from PolicyUtils.getUnitRateValue and the currency is from CurrencyUtils.getCurrencySymbol.
What changes do you think we should make in order to solve the problem?

In DistanceRequestUtils.getRateForDisplay, use CurrencyUtils.convertAmountToDisplayString to get the formatted amount and currency.
-return `${currencySymbol}${formattedRate} / ${singularDistanceUnit}`;
+return `${CurrencyUtils.convertAmountToDisplayString(rate, currency)} / ${singularDistanceUnit}`;

We can optionally replace this
https://github.com/Expensify/App/blob/dc17b1b1f2a1c9fa2c375ea3db0f4ef804ed0ac0/src/pages/workspace/distanceRates/PolicyDistanceRatesPage.tsx#L105-L107
with DistanceRequestUtils.getRateForDisplay.
With this change, the amount will now have a comma separator (in en).
Before:

After:

Woah, no-one is assigned. Adding label!

Thanks @jliexpensify , good 👀
@shawnborton 👀 plz, in one spot of the app we show 3 decimal places for distance and the other we show 2. It's inconsistent but I'm unsure if we need/want to show 3 everywhere (mainly since the employee/submitter isn't going to care if the reimbursement rate is .12 or .125 per mile)

This has been labelled "Needs Reproduction". Follow the steps here: https://stackoverflowteams.com/c/expensify/questions/16989

Hmm that's a great question @trjExpensify @JmillsExpensify @neil-marcellini do you have any context there? I think we should always show the three digits as we do in the workspace editor, right?

Following OldDot we show the three decimal places when the last digit isn't a 0. If it's a 0 we trim it. So I think we should follow that.
https://github.com/Expensify/App/assets/16232057/58401cb5-8c3e-4250-8322-80dbce7907dd

(mainly since the employee/submitter isn't going to care if the reimbursement rate is .12 or .125 per mile)

The IRS standard rate is currently 0.655 per mile, so we for sure don't want to trim that third decimal place when it's a number greater than 0.

Thanks, looks like this is working as expected, so closing

we show the three decimal places when the last digit isn't a 0. If it's a 0 we trim it. So I think we should follow that.

comment/reopen if ya disagree


dfe-analytics configuration table & web request user_id pseudonymisation script

Currently we have a number of services with pseudonymised user IDs in database tables streamed to BigQuery, but where the same IDs are not pseudonymised in the user_id field in web request events streamed to BigQuery. This means that none of the security benefits of pseudonymisation are realised because the user_ids are still stored in BigQuery in their raw form anyway!
This PR provides:

A stored procedure to convert all raw user_ids in the events table to pseudonymised user_ids
To support this, a table of dfe-analytics configurations over time created using the new initialise_analytics event_type introduced in dfe-analytics 1.9.0
Fixes data_functions.eventDataCreateOrReplace() function to handle strings that include quotation marks correctly
Updates test data to use candidates table instead of subjects to allow this to be tested more easily

Once merged we will need to send each team the following instructions if they wish to pseudonymise their user_ids:

Ensure that the id field in a database table that matches some of the values in the user_id field in your events table in BigQuery is in both your analytics.yml and analytics_pii.yml files.
Set DfE::Analytics.config.pseudonymise_web_request_user_id = true in your application. See https://github.com/DFE-Digital/dfe-analytics#user-id-pseudonymisation.
Run your Dataform pipeline.
Invoke the stored procedure called pseudonymise_request_user_ids located in the same dataset as your events table, setting its user_table parameter to the name of the table you identified in step 1.
Identify any parts of your Dataform codebase which join raw user_ids to pseudonymised user_ids using something like JOIN ... ON pseudonymised_user_id = TO_HEX(SHA256(raw_user_id)) and update them so they no longer use TO_HEX() and SHA256() GoogleSQL functions to pseudonymise the raw ID before joining. This is necessary because the raw ID will no longer be stored in BigQuery.
Full refresh your Dataform pipeline.

DfE::Analytics.config.pseudonymise_web_request_user_id = false

In the PR description above, this is actually wrong. It refers to the def-analytics GEM which is also wrong I believe. I think the intention here is to start psuedonymising right ?
In which case it should be:
DfE::Analytics.config.pseudonymise_web_request_user_id = true

Good spot @asatwal - I've amended this. Will be important for this to be correct as I'll copy and paste these instructions into the release notes and instructions we send round teams.


Privacy Manfiest

From Apple docs:

Starting in spring 2024, you must include the privacy manifest for any SDK listed below when you submit new apps in App Store Connect that include those SDKs, or when you submit an app update that adds one of the listed SDKs as part of the update. Signatures are also required in these cases where the listed SDKs are used as binary dependencies.

Could you please add a privacy manifest for this library?

I dont see Statsig, nor any library the statsig SDK depends on in the list that you linked - it sounds like it is not required for using the statsig sdk.

Closing per the above - Statsig SDK is not relevant here

@tore-statsig this was mostly for future-proofing the library. I imagine at some point in the future they might require this manifest for every library.

Reopening as we do need to add this due to usage of UserDefaults
https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api#4278401

Published here https://github.com/statsig-io/ios-sdk/releases/tag/1.39.2


logs in console

hi @oblomov-dev
the console printing might be irrelevant as of new debugger tools
           `            if (sap.z2ui5.checkLogActive) {` && |\n|  &&
           `                console.log('Request Object:');` && |\n|  &&
           `                console.log(sap.z2ui5.oBody);` && |\n|  &&
           `            }` && |\n|  &&

this code in get_js method of Z2UI5_CL_FW_HTTP_HANDLER

Thank you for the hint, deleted it with https://github.com/abap2UI5/abap2UI5/pull/791


Create XX-dao-utility-score.md

replaces HIP-78

rendered link: https://github.com/ferebee/HIP/blob/patch-7/XX-dao-utility-score.md

Using other branch


Initialization of Neural networks in Pytorch Deep CFR

I ran the code of the pytorch implementation of Deep CFR on Leduc poker. However, the current implementation cannot converge and the results sometimes become even worse along with the training, as below:
1
4.217283273153504
2
3.209117667122859
3
2.3267549939496353
4
2.473830942307169
5
2.7929482331832025
6
3.252047158838253
7
3.3688507064656976
8
3.3506459572731355
9
3.2942269981974626
10
3.349534382995187
11
3.342262661706414
12
3.3200865692030903
13
3.3861909419035703
14
3.4463490050742402
15
3.4930911396839925

The tf version works well. Similar to the one in NFSP, I found that the tf version uses a different initialization of the networks. So I adapted the Sonnet initialization in NFSP to Deep CFR. The results are as follows:
1
4.754486997415914
2
2.700013812921206
3
1.6279546327154253
4
1.3503307298273948
5
1.0704846559989847
6
1.0538017888846307
7
0.949124458864229
8
0.9312268595393176
9
0.864773300329345
10
0.7689361350033274
11
0.7637610376557935
12
0.7403772000585912
13
0.6589145084244787
14
0.5804421013613964
15
0.520045173061336

The later one is similar to the tf implementation (sometimes better):
I0617 07:19:59.767846<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 0 conv: 3.048379395963053
I0617 07:23:09.881250<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 1 conv: 9.48192721261234
I0617 07:26:19.034047<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 2 conv: 5.171986619866603
I0617 07:29:34.269373<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 3 conv: 3.911214665171007
I0617 07:32:50.712246<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 4 conv: 3.2244062674338334
I0617 07:36:12.457256<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 5 conv: 2.5505362425269125
I0617 07:39:38.116594<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 6 conv: 2.153656803765398
I0617 07:43:10.559897<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 7 conv: 1.7259271699052574
I0617 07:46:39.927793<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 8 conv: 1.524976493750408
I0617 07:50:12.040321<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 9 conv: 1.3242997209137242
I0617 07:53:48.313006<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 10 conv: 1.1116873745104647
I0617 07:57:21.149296<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 11 conv: 0.9340368653378026
I0617 08:00:49.280863<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 12 conv: 0.7963322550904859
I0617 08:04:19.089864<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 13 conv: 0.7409843627321417
I0617 08:07:51.988271<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 14 conv: 0.6904493037290826
I0617 08:11:08.892899<PHONE_NUMBER>22240 deep_cfr_tf.py:117] Iteration: 15 conv: 0.668540484233273

So I think the current implementation of Deep CFR with Pytorch should be updated. Below is my implementation.
# Copyright 2019 DeepMind Technologies Ltd. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Implements Deep CFR Algorithm.
See https://arxiv.org/abs/1811.00164.
The algorithm defines an `advantage` and `strategy` networks that compute
advantages used to do regret matching across information sets and to approximate
the strategy profiles of the game.  To train these networks a fixed ring buffer
(other data structures may be used) memory is used to accumulate samples to
train the networks.
"""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import collections
import random
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

import math
from scipy import stats

from open_spiel.python import policy
import pyspiel

AdvantageMemory = collections.namedtuple(
    "AdvantageMemory", "info_state iteration advantage action")

StrategyMemory = collections.namedtuple(
    "StrategyMemory", "info_state iteration strategy_action_probs")

# def reset_params(m):
#     """Reinitializes the parameters of the given layer.
#     Args:
#       m: (nn.Module) layer object
#     """
#     if isinstance(m, nn.Linear):
#         m.reset_parameters()

class SonnetLinear(nn.Module):
    """A Sonnet linear module.
    Always includes biases and only supports ReLU activations.
    """

    def __init__(self, in_size, out_size, activate_relu=True):
        """Creates a Sonnet linear layer.
        Args:
          in_size: (int) number of inputs
          out_size: (int) number of outputs
          activate_relu: (bool) whether to include a ReLU activation layer
        """
        super(SonnetLinear, self).__init__()
        self._activate_relu = activate_relu
        self._in_size = in_size
        self._out_size = out_size
        # stddev = 1.0 / math.sqrt(self._in_size)
        # mean = 0
        # lower = (-2 * stddev - mean) / stddev
        # upper = (2 * stddev - mean) / stddev
        # # Weight initialization inspired by Sonnet's Linear layer,
        # # which cites https://arxiv.org/abs/1502.03167v3
        # # pytorch default: initialized from
        # # uniform(-sqrt(1/in_features), sqrt(1/in_features))
        self._weight = None
        self._bias = None
        self.reset()

    def forward(self, tensor):
        y = F.linear(tensor, self._weight, self._bias)
        return F.relu(y) if self._activate_relu else y

    def reset(self):
        stddev = 1.0 / math.sqrt(self._in_size)
        mean = 0
        lower = (-2 * stddev - mean) / stddev
        upper = (2 * stddev - mean) / stddev
        # Weight initialization inspired by Sonnet's Linear layer,
        # which cites https://arxiv.org/abs/1502.03167v3
        # pytorch default: initialized from
        # uniform(-sqrt(1/in_features), sqrt(1/in_features))
        self._weight = nn.Parameter(
            torch.Tensor(
                stats.truncnorm.rvs(
                    lower, upper, loc=mean, scale=stddev, size=[self._out_size,
                                                                self._in_size])))
        self._bias = nn.Parameter(torch.zeros([self._out_size]))

class MLP(nn.Module):
    """A simple network built from nn.linear layers."""

    def __init__(self,
                 input_size,
                 hidden_sizes,
                 output_size,
                 activate_final=False):
        """Create the MLP.
        Args:
          input_size: (int) number of inputs
          hidden_sizes: (list) sizes (number of units) of each hidden layer
          output_size: (int) number of outputs
          activate_final: (bool) should final layer should include a ReLU
        """

        super(MLP, self).__init__()
        self._layers = []
        # Hidden layers
        for size in hidden_sizes:
            self._layers.append(SonnetLinear(in_size=input_size, out_size=size))
            input_size = size
        # Output layer
        self._layers.append(
            SonnetLinear(
                in_size=input_size,
                out_size=output_size,
                activate_relu=activate_final))

        self.model = nn.ModuleList(self._layers)

    def forward(self, x):
        for layer in self.model:
            x = layer(x)
        return x

    def reset(self):
        for layer in self._layers:
            layer.reset()

# class MLP(nn.Module):
#     """Basic MLP implementation using PyTorch."""

#     def __init__(self, layers):
#         """Initialize a MLP in PyTorch.
#         Args:
#           layers: (list[int]) Layer sizes.
#         """
#         super(MLP, self).__init__()
#         layers = [
#             nn.Linear(layers[i], layers[i + 1]) for i in range(len(layers[:-1]))
#         ]
#         self._model = nn.Sequential(*layers)

#     def forward(self, model_input):
#         return self._model(model_input)

#     def reset(self):
#         """Reinitializes the network."""
#         self._model.apply(reset_params)

class ReservoirBuffer(object):
    """Allows uniform sampling over a stream of data.
    This class supports the storage of arbitrary elements, such as observation
    tensors, integer actions, etc.
    See https://en.wikipedia.org/wiki/Reservoir_sampling for more details.
    """

    def __init__(self, reservoir_buffer_capacity):
        self._reservoir_buffer_capacity = reservoir_buffer_capacity
        self._data = []
        self._add_calls = 0

    def add(self, element):
        """Potentially adds `element` to the reservoir buffer.
        Args:
          element: data to be added to the reservoir buffer.
        """
        if len(self._data) < self._reservoir_buffer_capacity:
            self._data.append(element)
        else:
            idx = np.random.randint(0, self._add_calls + 1)
            if idx < self._reservoir_buffer_capacity:
                self._data[idx] = element
        self._add_calls += 1

    def sample(self, num_samples):
        """Returns `num_samples` uniformly sampled from the buffer.
        Args:
          num_samples: `int`, number of samples to draw.
        Returns:
          An iterable over `num_samples` random elements of the buffer.
        Raises:
          ValueError: If there are less than `num_samples` elements in the buffer
        """
        if len(self._data) < num_samples:
            raise ValueError("{} elements could not be sampled from size {}".format(
                num_samples, len(self._data)))
        return random.sample(self._data, num_samples)

    def clear(self):
        self._data = []
        self._add_calls = 0

    def __len__(self):
        return len(self._data)

    def __iter__(self):
        return iter(self._data)

class DeepCFRSolver(policy.Policy):
    """Implements a solver for the Deep CFR Algorithm with PyTorch.
    See https://arxiv.org/abs/1811.00164.
    Define all networks and sampling buffers/memories.  Derive losses & learning
    steps. Initialize the game state and algorithmic variables.
    Note: batch sizes default to `None` implying that training over the full
          dataset in memory is done by default.  To sample from the memories you
          may set these values to something less than the full capacity of the
          memory.
    """

    def __init__(self,
                 game,
                 policy_network_layers=(256, 256),
                 advantage_network_layers=(128, 128),
                 num_iterations: int = 100,
                 num_traversals: int = 20,
                 learning_rate: float = 1e-4,
                 batch_size_advantage=None,
                 batch_size_strategy=None,
                 memory_capacity: int = int(1e6),
                 policy_network_train_steps: int = 1,
                 advantage_network_train_steps: int = 1,
                 reinitialize_advantage_networks: bool = True):
        """Initialize the Deep CFR algorithm.
        Args:
          game: Open Spiel game.
          policy_network_layers: (list[int]) Layer sizes of strategy net MLP.
          advantage_network_layers: (list[int]) Layer sizes of advantage net MLP.
          num_iterations: (int) Number of training iterations.
          num_traversals: (int) Number of traversals per iteration.
          learning_rate: (float) Learning rate.
          batch_size_advantage: (int or None) Batch size to sample from advantage
            memories.
          batch_size_strategy: (int or None) Batch size to sample from strategy
            memories.
          memory_capacity: Number af samples that can be stored in memory.
          policy_network_train_steps: Number of policy network training steps (per
            iteration).
          advantage_network_train_steps: Number of advantage network training steps
            (per iteration).
          reinitialize_advantage_networks: Whether to re-initialize the advantage
            network before training on each iteration.
        """
        all_players = list(range(game.num_players()))
        super(DeepCFRSolver, self).__init__(game, all_players)
        self._game = game
        if game.get_type().dynamics == pyspiel.GameType.Dynamics.SIMULTANEOUS:
            # `_traverse_game_tree` does not take into account this option.
            raise ValueError("Simulatenous games are not supported.")
        self._batch_size_advantage = batch_size_advantage
        self._batch_size_strategy = batch_size_strategy
        self._policy_network_train_steps = policy_network_train_steps
        self._advantage_network_train_steps = advantage_network_train_steps
        self._num_players = game.num_players()
        self._root_node = self._game.new_initial_state()
        self._embedding_size = len(self._root_node.information_state_tensor(0))
        self._num_iterations = num_iterations
        self._num_traversals = num_traversals
        self._reinitialize_advantage_networks = reinitialize_advantage_networks
        self._num_actions = game.num_distinct_actions()
        self._iteration = 1

        # Define strategy network, loss & memory.
        self._strategy_memories = ReservoirBuffer(memory_capacity)
        self._policy_network = MLP(self._embedding_size, list(advantage_network_layers),
                self._num_actions)
        # Illegal actions are handled in the traversal code where expected payoff
        # and sampled regret is computed from the advantage networks.
        self._policy_sm = nn.Softmax(dim=-1)
        self._loss_policy = nn.MSELoss()
        self._optimizer_policy = torch.optim.Adam(
            self._policy_network.parameters(), lr=learning_rate)

        # Define advantage network, loss & memory. (One per player)
        self._advantage_memories = [
            ReservoirBuffer(memory_capacity) for _ in range(self._num_players)
        ]
        self._advantage_networks = [
            MLP(self._embedding_size, list(advantage_network_layers),
                self._num_actions) for _ in range(self._num_players)
        ]
        self._loss_advantages = nn.MSELoss(reduction="mean")
        self._optimizer_advantages = []
        for p in range(self._num_players):
            self._optimizer_advantages.append(
                torch.optim.Adam(
                    self._advantage_networks[p].parameters(), lr=learning_rate))
        self._learning_rate = learning_rate

    @property
    def advantage_buffers(self):
        return self._advantage_memories

    @property
    def strategy_buffer(self):
        return self._strategy_memories

    def clear_advantage_buffers(self):
        for p in range(self._num_players):
            self._advantage_memories[p].clear()

    def reinitialize_advantage_network(self, player):
        self._advantage_networks[player].reset()
        self._optimizer_advantages[player] = torch.optim.Adam(
                    self._advantage_networks[player].parameters(), lr=self._learning_rate)

    def reinitialize_advantage_networks(self):
        for p in range(self._num_players):
            self.reinitialize_advantage_network(p)

    def solve(self):
        """Solution logic for Deep CFR.
        Traverses the game tree, while storing the transitions for training
        advantage and policy networks.
        Returns:
          1. (nn.Module) Instance of the trained policy network for inference.
          2. (list of floats) Advantage network losses for
            each player during each iteration.
          3. (float) Policy loss.
        """
        advantage_losses = collections.defaultdict(list)
        for n_iter in range(self._num_iterations):
            print(n_iter)
            for p in range(self._num_players):
                for _ in range(self._num_traversals):
                    self._traverse_game_tree(self._root_node, p)
                if self._reinitialize_advantage_networks:
                    # Re-initialize advantage network for player and train from scratch.
                    self.reinitialize_advantage_network(p)
                # Re-initialize advantage networks and train from scratch.
                advantage_losses[p].append(self._learn_advantage_network(p))
            self._iteration += 1
            if n_iter % 1 == 0 and n_iter != 0:
                policy_loss = self._learn_strategy_network()
                average_policy = policy.tabular_policy_from_callable(
                    self._game, self.action_probabilities)
                pyspiel_policy = policy.python_policy_to_pyspiel_policy(average_policy)
                conv = pyspiel.nash_conv(self._game, pyspiel_policy)
                print(conv)
        # Train policy network.
        policy_loss = self._learn_strategy_network()
        return self._policy_network, advantage_losses, policy_loss

    def _traverse_game_tree(self, state, player):
        """Performs a traversal of the game tree.
        Over a traversal the advantage and strategy memories are populated with
        computed advantage values and matched regrets respectively.
        Args:
          state: Current OpenSpiel game state.
          player: (int) Player index for this traversal.
        Returns:
          (float) Recursively returns expected payoffs for each action.
        """
        expected_payoff = collections.defaultdict(float)
        if state.is_terminal():
            # Terminal state get returns.
            return state.returns()[player]
        elif state.is_chance_node():
            # If this is a chance node, sample an action
            action = np.random.choice([i[0] for i in state.chance_outcomes()])
            return self._traverse_game_tree(state.child(action), player)
        elif state.current_player() == player:
            sampled_regret = collections.defaultdict(float)
            # Update the policy over the info set & actions via regret matching.
            _, strategy = self._sample_action_from_advantage(state, player)
            for action in state.legal_actions():
                expected_payoff[action] = self._traverse_game_tree(
                    state.child(action), player)
            cfv = 0
            for a_ in state.legal_actions():
                cfv += strategy[a_] * expected_payoff[a_]
            for action in state.legal_actions():
                sampled_regret[action] = expected_payoff[action]
                sampled_regret[action] -= cfv
#             print("sampled regret")
#             print(sampled_regret)
            sampled_regret_arr = [0] * self._num_actions
            for action in sampled_regret:
                sampled_regret_arr[action] = sampled_regret[action]
            self._advantage_memories[player].add(
                AdvantageMemory(state.information_state_tensor(), self._iteration,
                                sampled_regret_arr, action))
            return cfv
        else:
            other_player = state.current_player()
            _, strategy = self._sample_action_from_advantage(state, other_player)
            # Recompute distribution for numerical errors.
            probs = np.array(strategy)
            probs /= probs.sum()
            sampled_action = np.random.choice(range(self._num_actions), p=probs)
            self._strategy_memories.add(
                StrategyMemory(
                    state.information_state_tensor(other_player), self._iteration,
                    strategy))
            return self._traverse_game_tree(state.child(sampled_action), player)

    def _sample_action_from_advantage(self, state, player):
        """Returns an info state policy by applying regret-matching.
        Args:
          state: Current OpenSpiel game state.
          player: (int) Player index over which to compute regrets.
        Returns:
          1. (list) Advantage values for info state actions indexed by action.
          2. (list) Matched regrets, prob for actions indexed by action.
        """
        info_state = state.information_state_tensor(player)
        legal_actions = state.legal_actions(player)
        with torch.no_grad():
            state_tensor = torch.FloatTensor(np.expand_dims(info_state, axis=0))
            raw_advantages = self._advantage_networks[player](state_tensor)[0].numpy()
        advantages = [max(0., advantage) for advantage in raw_advantages]
#         print("advantages")
#         print(advantages)
        cumulative_regret = np.sum([advantages[action] for action in legal_actions])
        matched_regrets = np.array([0.] * self._num_actions)
        if cumulative_regret > 0.:
            for action in legal_actions:
                matched_regrets[action] = advantages[action] / cumulative_regret
        else:
            matched_regrets[max(legal_actions, key=lambda a: raw_advantages[a])] = 1
        return advantages, matched_regrets

    def action_probabilities(self, state):
        """Computes action probabilities for the current player in state.
        Args:
          state: (pyspiel.State) The state to compute probabilities for.
        Returns:
          (dict) action probabilities for a single batch.
        """
        cur_player = state.current_player()
        legal_actions = state.legal_actions(cur_player)
        info_state_vector = np.array(state.information_state_tensor())
        if len(info_state_vector.shape) == 1:
            info_state_vector = np.expand_dims(info_state_vector, axis=0)
        with torch.no_grad():
            logits = self._policy_network(torch.FloatTensor(info_state_vector))
            probs = self._policy_sm(logits).numpy()
        return {action: probs[0][action] for action in legal_actions}

    def _learn_advantage_network(self, player):
        """Compute the loss on sampled transitions and perform a Q-network update.
        If there are not enough elements in the buffer, no loss is computed and
        `None` is returned instead.
        Args:
          player: (int) player index.
        Returns:
          (float) The average loss over the advantage network.
        """
        for _ in range(self._advantage_network_train_steps):

            if self._batch_size_advantage:
                if self._batch_size_advantage > len(self._advantage_memories[player]):
                    ## Skip if there aren't enough samples
                    return None
                samples = self._advantage_memories[player].sample(
                    self._batch_size_advantage)
            else:
                samples = self._advantage_memories[player]
            info_states = []
            advantages = []
            iterations = []
            for s in samples:
                info_states.append(s.info_state)
                advantages.append(s.advantage)
                iterations.append([s.iteration])
            # Ensure some samples have been gathered.
            if not info_states:
                return None
            self._optimizer_advantages[player].zero_grad()
            advantages = torch.FloatTensor(np.array(advantages))
            iters = torch.FloatTensor(np.sqrt(np.array(iterations)))
            outputs = self._advantage_networks[player](
                torch.FloatTensor(np.array(info_states)))
            loss_advantages = self._loss_advantages(iters * outputs,
                                                    iters * advantages)
#             print("loss_advantages")
#             print(loss_advantages)
            loss_advantages.backward()
            self._optimizer_advantages[player].step()

        return loss_advantages.detach().numpy()

    def _learn_strategy_network(self):
        """Compute the loss over the strategy network.
        Returns:
          (float) The average loss obtained on this batch of transitions or `None`.
        """
        for _ in range(self._policy_network_train_steps):
            if self._batch_size_strategy:
                if self._batch_size_strategy > len(self._strategy_memories):
                    ## Skip if there aren't enough samples
                    return None
                samples = self._strategy_memories.sample(self._batch_size_strategy)
            else:
                samples = self._strategy_memories
            info_states = []
            action_probs = []
            iterations = []
            for s in samples:
                info_states.append(s.info_state)
                action_probs.append(s.strategy_action_probs)
                iterations.append([s.iteration])

            self._optimizer_policy.zero_grad()
            iters = torch.FloatTensor(np.sqrt(np.array(iterations)))
            ac_probs = torch.FloatTensor(np.array(np.squeeze(action_probs)))
            logits = self._policy_network(torch.FloatTensor(np.array(info_states)))
            outputs = self._policy_sm(logits)
            loss_strategy = self._loss_policy(iters * outputs, iters * ac_probs)
            loss_strategy.backward()
            self._optimizer_policy.step()

        return loss_strategy.detach().numpy()

(The code also includes an ugly implementation of reinitialization. More elegant way may be used.)
The main function is as follows (from the codebase)
# Copyright 2019 DeepMind Technologies Ltd. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Python Deep CFR example."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import pyspiel
from absl import app
from absl import flags
from absl import logging
from algorithms_pytorch import deep_cfr_pytorch as deep_cfr

from open_spiel.python import policy
from open_spiel.python.algorithms import expected_game_score

FLAGS = flags.FLAGS

flags.DEFINE_integer("num_iterations", 1000, "Number of iterations")
flags.DEFINE_integer("num_traversals", 1500, "Number of traversals/games")
flags.DEFINE_string("game_name", "leduc_poker", "Name of the game")

def main(unused_argv):
    logging.info("Loading %s", FLAGS.game_name)
    game = pyspiel.load_game(FLAGS.game_name)
    deep_cfr_solver = deep_cfr.DeepCFRSolver(
        game,
        policy_network_layers=(64, 64, 64),
        advantage_network_layers=(64, 64, 64),
        num_iterations=FLAGS.num_iterations,
        num_traversals=FLAGS.num_traversals,
        learning_rate=1e-3,
        batch_size_advantage=2048,
        batch_size_strategy=2048,
        memory_capacity=int(1e7),
        policy_network_train_steps=5000,
        advantage_network_train_steps=750,
        reinitialize_advantage_networks=True
    )

    _, advantage_losses, policy_loss = deep_cfr_solver.solve()

    for player, losses in advantage_losses.items():
        logging.info("Advantage for player %d: %s", player,
                     losses[:2] + ["..."] + losses[-2:])
        logging.info("Advantage Buffer Size for player %s: '%s'", player,
                     len(deep_cfr_solver.advantage_buffers[player]))
        logging.info("Strategy Buffer Size: '%s'",
                     len(deep_cfr_solver.strategy_buffer))
        logging.info("Final policy loss: '%s'", policy_loss)

    average_policy = policy.tabular_policy_from_callable(
    game, deep_cfr_solver.action_probabilities)
    pyspiel_policy = policy.python_policy_to_pyspiel_policy(average_policy)
    conv = pyspiel.nash_conv(game, pyspiel_policy)
    logging.info("Deep CFR in '%s' - NashConv: %s", FLAGS.game_name, conv)

    average_policy_values = expected_game_score.policy_value(
    game.new_initial_state(), [average_policy] * 2)
    logging.info("Computed player 0 value: %.2f (expected: %.2f).",
                 average_policy_values[0], -1 / 18)
    logging.info("Computed player 1 value: %.2f (expected: %.2f).",
                 average_policy_values[1], 1 / 18)

if __name__ == "__main__":
    app.run(main)

Hi @rainwangphy, great thanks! However: can you plaease submit a pull request with your changes? This way it's easier to see the lines that you have changed and we can pull it in for import (then I will also tag the original author).

Hi @rainwangphy, great thanks! However: can you please submit a pull request with your changes? This way it's easier to see the lines that you have changed and we can pull it in for import (then I will also tag the original author).

Yes. I will do this. Thanks

@MJ10, can you take a look at this, and let me know what you think? There is a PR for it now too: https://github.com/deepmind/open_spiel/pull/634.

@rainwangphy , I noticed your comparisons are only up to 15 iterations. Could you both for longer, e.g. at least up to, say, 300? You can use this paper to see what the TF Deep CFR graph should look like: https://arxiv.org/abs/2103.00187

@rainwangphy , I noticed your comparisons are only up to 15 iterations. Could you both for longer, e.g. at least up to, say, 300? You can use this paper to see what the TF Deep CFR graph should look like: https://arxiv.org/abs/2103.00187 (see Figure 3)

@lanctot Sure. I will run the codes for more iterations and compare the results. That may take a bit longer time and I will update when I finished.

Below is the figure with the results of different implementation of deep CFR with 300 iterations (I only run once due to the limited resources.)

The 'deep cfr pt original' is the original implementation of pytorch deep cfr and the 'deep cfr pt' is the new implementation. The 'deep cfr tf' is the tensorflow implementation.

Wonderful, thank you!
I will import the PR and it will likely get merged on Monday.

Thanks for the experiments @rainwangphy!
It is surprising to me that just the init would make this big a difference. Out of curiosity, I had the older environment which I used to test, so I tested it again, It looks like in 5/10 runs Pytorch and TF results matched. It looks like I should've run it few more times when I initially raised the PR. Apologies for that.
Regardless, this is a good bug to spot. I looked at your PR, and also looked at the TF2, Pytorch, and Sonnet code. The situation is:
TF/Sonnet version used Truncated Normal initialization.
TF2 version uses Kaiming/He uniform initialization
Pytorch version uses Kaiming/He uniform initialization.
So it looks like if initialization is the issue then the TF2 version would also be affected. But looking at the code again I spotted another major thing that to me seems like the most likely cause for this issue, and that is the activation. The original PyTorch implementation does not have any activation function (major oversight on my part). which explains why it doesn't reliable converge. In the layer, you reimplemented you added the activation function and that might've been responsible for fixing the issue. To confirm this, could you please try a run in your version where you remove the custom initialization and use the torch default (kaiming_uniform). I am trying to set up open_spiel again to test this but I am getting some weird compatibility issues on my laptop. Will try to fix those and check myself too,
If it is indeed the case that the activation missing is causing the issue, then it would be a much simpler fix (just adding a nn.Relu() in the layers initialization in the MLP class)
Thanks again, this is a great find! (And apologies for my oversights in the initial implementation)

Thanks for your comments @MJ10! It'd be a good idea to run the TF2 impl as well just to be sure, but IIRC it was a lot slower and we vould mot figure out why. @StochasticEntropy, did you ever run it in Leduc up to 300 iterations? Did it look like the above graphs?

Thanks for the experiments @rainwangphy!
It is surprising to me that just the init would make this big a difference. Out of curiosity, I had the older environment which I used to test, so I tested it again, It looks like in 5/10 runs Pytorch and TF results matched. It looks like I should've run it few more times when I initially raised the PR. Apologies for that.
Regardless, this is a good bug to spot. I looked at your PR, and also looked at the TF2, Pytorch, and Sonnet code. The situation is:
TF/Sonnet version used Truncated Normal initialization.
TF2 version uses Kaiming/He uniform initialization
Pytorch version uses Kaiming/He uniform initialization.
So it looks like if initialization is the issue then the TF2 version would also be affected. But looking at the code again I spotted another major thing that to me seems like the most likely cause for this issue, and that is the activation. The original PyTorch implementation does not have any activation function (major oversight on my part). which explains why it doesn't reliable converge. In the layer, you reimplemented you added the activation function and that might've been responsible for fixing the issue. To confirm this, could you please try a run in your version where you remove the custom initialization and use the torch default (kaiming_uniform) but keep the relu. I am trying to set up open_spiel again to test this but I am getting some weird compatibility issues on my laptop. Will try to fix those and check myself too,
If it is indeed the case that the activation missing is causing the issue, then it would be a much simpler fix (just adding a nn.Relu() after each layer in the layers initialization in the MLP class)
Thanks again, this is a great find! (And apologies for my oversights in the initial implementation)

Thanks. I am glad to find this, which is also important for my work. Thanks for the great implementations @lanctot @MJ10 .

This is fixed by https://github.com/deepmind/open_spiel/pull/634, which I have now imported. It should be merged into master tomorrow. Thanks again @rainwangphy !

Hi @lanctot, that's not a problem with the TF2 implementation, so I agree that it was likely the missing activation if that was the case. I did two test runs to confirm:

IIRC the TF2 was slower in inference/data gathering (but I couldn't figure out why, maybe that's also improved now with TF2.5) and faster in training than TF1, resulting in similar times. But I didn't confirm that now.


Allow custom/multi hosted mqttServer

:warning: Do not merge yet!!! :warning:

I removed the unecessary Singleton on MqttRouter,
because it is instanciated only on UseAttributeRouting, or WithAttributeRouting functions in ServiceCollectionExtensions.cs
It allow me to have multiple MqttRouter with it's own MqttServer.
then the MqttControllerContext is providing the right MqttServer.
var controllerContext = new MqttControllerContext()
{
    MqttContext = context,
    MqttServer = Server
};

That's it, this PR is ready to merge for me! ✅
Let me know if I have to change something.
Regards,


Homework 2 submitted

What changes are you trying to make? (e.g. Adding or removing code, refactoring existing code, adding reports)
SQL homework_2 submitted
What did you learn from the changes you have made?
I learned SQL statement SELECT, JOIN, WHERE, etc
Was there another approach you were thinking about making? If so, what approach(es) were you thinking of?
Have applied 2 method in applicable.
Were there any challenges? If so, what issue(s) did you face? How did you overcome it?
there were few challenges to overcome
How were these changes tested?
Tested in DB Browser
A reference to a related issue in your repository (if applicable)
na
Checklist

[ ] I can confirm that my changes are working as intended

HW2 marked.


Incorrect stringToSign created for API calls that include query parameters without assignation

If you have a path that includes a query parameter without an "=" sign the stringToSign that is built during the V4 signature construction is wrong. The code was outputting the query parameter without the "=" sign. AWS expects the "=" to be there.
It seems that this only affects a number of S3 calls and a couple of calls in Cloudfront. I have checked both of systems and the code fixes the issue. This fixes https://github.com/swift-aws/aws-sdk-swift/issues/117

@adam-fowler nice catch.
I worry a little about changing the url in the signer because urlEncodedQueryParams are sent in the body when the http method is not a GET, and they won't match what's in the signature.
It's also sad to have to perform another operation on the url in the signer...
I think this can be fixed in createAWSRequest without requiring another operation on the url string in the signer. If the URLQueryItem has a nil value, then setting the item name to attr= with value nil looks like it will produce the expected result.
Are you open to trying this?
`--- a/Sources/AWSSDKSwiftCore/AWSClient.swift
+++ b/Sources/AWSSDKSwiftCore/AWSClient.swift
@@ -401,11 +401,15 @@ extension AWSClient {
     // construct query array
     var queryItems = urlQueryItems(fromDictionary: queryParams) ?? []

   // add new queries to query item array. These need to be added to the queryItems list instead of the queryParams dictionary as added nil items to a dictionary doesn't add a value.
   if let pathQueryItems = parsedPath.queryItems {
       for item in pathQueryItems {

           queryItems.append(URLQueryItem(name:item.name, value:item.value))

           if let value = item.value {

               queryItems.append(URLQueryItem(name: item.name, value: value))

           } else {

               queryItems.append(URLQueryItem(name: "\(item.name)=", value: nil))

           }
       }
   }

@@ -447,7 +451,7 @@ extension AWSClient {
if let value = dict[key] {
queryItems.append(URLQueryItem(name: key, value: String(describing: value)))
} else {

           queryItems.append(URLQueryItem(name: key, value: nil))

           queryItems.append(URLQueryItem(name: "\(key)=", value: nil))`

@adam-fowler nevermind.

value needs to be an empty string, try this I meant
--- a/Sources/AWSSDKSwiftCore/AWSClient.swift
+++ b/Sources/AWSSDKSwiftCore/AWSClient.swift
@@ -401,11 +401,15 @@ extension AWSClient {

         // construct query array
         var queryItems = urlQueryItems(fromDictionary: queryParams) ?? []
-
+
         // add new queries to query item array. These need to be added to the queryItems list instead of the queryParams dictionary as added nil items to a dictionary doesn't add a value.
         if let pathQueryItems = parsedPath.queryItems {
             for item in pathQueryItems {
-                queryItems.append(URLQueryItem(name:item.name, value:item.value))
+                if let value = item.value {
+                    queryItems.append(URLQueryItem(name: item.name, value: value))
+                } else {
+                    queryItems.append(URLQueryItem(name: "\(item.name)=", value: ""))
+                }
             }
         }

@@ -447,7 +451,7 @@ extension AWSClient {
             if let value = dict[key] {
                 queryItems.append(URLQueryItem(name: key, value: String(describing: value)))
             } else {
-                queryItems.append(URLQueryItem(name: key, value: nil))
+                queryItems.append(URLQueryItem(name: "\(key)=", value: ""))
             }
         }

Your code works as well, but for me it feels like you are massaging the inputs to a system to get it to work instead of fixing the system.
As for your concern about the urlEncodedQueryParams in the body for query based systems, the signature is only produced off the query params in the path which never get placed in the body. And as pointed out earlier this change only affects S3 and Cloudfront which use XML.
If you still prefer I'm happy to put that in though.

I see your point.
But I'm still feeling pragmatic here. True the system doesn't support nil values for query params, and my proposed change does not "fix" that. But it seems like support for this minority case adds an excessive tax on the majority. I'll let you decide though, since you discovered this.
urlEncodedQueryParams are placed in the body when the serviceProtocol.type is query and the http method is not GET. Maybe it's not an issue though as you say.

I think it sounds like there's a QueryReformatter which takes a good chunk of the urlencoding, query-rewriting, and assorted massaging away from both the Client and the Signer, tho that might be slightly longer term.
@adam-fowler do you know of a decent way to write a testcase to catch this as well? Just thoughtful of how we can catch any regressions going forward 🤔

@jonnymacs I'll go with your method. If I can think of a better way to support it within the V4 signature I might suggest a change later
@Yasumoto I can stick a test in the aws-sdk-swift S3 tests. If I go with Jonny's fix it's a lot harder to add a test at the core level, as I have to run the whole pipeline.


[Add]  Appwrite for database and authentication

Site URL
https://appwrite.io/docs/authentication
Description of the above provided site
Appwrite: The open-source backend server that simplifies application development with features like authentication, database management, and file storage.
Category
Backend
Sub Category
authentication, database
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I am willing to work on this issue (optional)
[X] I am a GSSoC'23 contributor

Just to confirm, are you in GSSOC @Mihan786Chistie?

Just to confirm, are you in GSSOC @Mihan786Chistie?

@CBID2 I'm in GSSOC'23


[PR] The proj files have been updated to enable SourceLink

CSProj files have been updated to enable SourceLink in your nuget
[This pull request was created with an automated workflow]
I noticed that your repository and Nuget package are important for our .NET community, but you still haven't enabled SourceLink.
We have to take 2 steps:

Please approve this pull request and make .NET a better place for .NET developers and their debugging.
Then just upload the .snupkg file to https://www.nuget.org/ (now you can find the snupkg file along with the .nuget file)

You can find more information about SourceLine at the following links
https://github.com/dotnet/sourcelink
https://www.hanselman.com/blog/ExploringNETCoresSourceLinkSteppingIntoTheSourceCodeOfNuGetPackagesYouDontOwn.aspx
If you are interesting about this automated workflow and how it works
https://github.com/JTOne123/GitHubMassUpdater
If you notice any flaws, please comment and I will try to make fixes manually

this project does not produce a package. why would sourcelink be required?

https://www.nuget.org/packages/TextCopy/
it is not required, it is just nice to have
I hope it will increase good debug experience of your nuget
if you disagree, just close

i do think sourcelinking is valuable. but the specific project you have updated in this PR is used for in-solution-weaving, so it doesnt produce a nuget and is never run outside this solution

ok, I see, than sorry)


Code adapted to PHP 8; upgrade to PHPUnit 9.4

Removed a deprecated call to libxml_disable_entity_loader().
Upgraded to PHPUnit 9.4; used $this->expectException() instead of annotations, fixed inherited method declarations.

Tested with PHP versions: 7.1.33, 7.2.34, 7.3.24, 7.4.12, 8.0.0RC5.
Fixes #124

@peterdevpl Awesome! 😄 I'm merging it :)


cmake: Fix compiling and running on PC

For some time now, NevoX hasn't been able to compile successfully on PC using CMake. This PR fixes that by using a #ifdef preprocessor directive to filter out the WinAPI code that is used only with nxdk, and also by telling the CMakeLists.txt file to copy the required files to the binary directory, which previously wasn't done, allowing NevoX to be compiled and used without any additional steps required.

Fixed in 8f5ee44. Thanks!


Muhammadali

Test provider google and debug.
Add documentation for google provider, drf and django.

@Muhammadali1620 any news on this?


Update master from dev

Additions regarding adding SSH keys to servers

May want to add some documentation about how to run the thing!

Pending #4


Location Map is not fully displayed in Android Location Page

After performing location operation to the device, when the location tab is clicked, the location is not properly displayed.

The location map is not fully displayed when I click the location tab for the first time , after zooming out and then zooming in it is fully displayed.


Parameter info shows signature of outer method instead of the current one

Version: VS 17.9.0 P2
I encounter this issue all the time.

Not only typing and triggering Parameter Info by (, but even an explicit call to request Parameter Info displays the wrong signature:
https://github.com/dotnet/razor/assets/9047283/f6ae95db-ff42-4c32-90e0-b0ad3ac5f41c
ContentBroken.razor
@inject IStringLocalizer<SharedResource> Localizer
@inject HttpClient Http

<FontAwesomeIcon Name="@Icon" />

@Localizer["This content has been either deleted, not visible to you, or broken."]

@code {
	[Parameter]
	public string? Icon { get; set; } = "heart-crack";

	protected override async Task OnInitializedAsync()
	{
		await Http.SetProfilePictureAsync(new SetProfilePictureRequest("123"));
	}
}

SetProfilePictureRequest.cs
namespace App.Shared.Contracts;

public record class SetProfilePictureRequest(
	string? PictureId
);

Duplicate of https://github.com/dotnet/razor/issues/9426 but I like your pictures better, so I'm going to close that one.


Zone deadlock?

Consider the following code:
import 'dart:async';
import 'dart:collection';

Queue<Function> _microtasks = new Queue();

void flushMicrotasks() {
  while (_microtasks.isNotEmpty) {
    _microtasks.removeFirst()();
  }
}

Future<void> main() async {
  final Zone zone = Zone.current.fork(specification: new ZoneSpecification(
    scheduleMicrotask: (_, __, ___, Function microtask) {
      _microtasks.add(microtask);
    },
  ));

  Future<Null> future;
  zone.run(() {
    future = () async {
      print('1');
      await Zone.root.run(() async {});
      print('2');
      await null;
      print('3');
    }();
    flushMicrotasks();
  });

  await new Future<Null>.value(future).timeout(const Duration(minutes: 1));
}

By all accounts, it should print the following:
1
2
3

... yet it prints this:
1
2

... then hangs, eventually timing out after 1 minute.
Now, comment out the following line:
      //await Zone.root.run(() async {});

Voilà!  It prints:
1
2
3

This happens both with --preview-dart-2 and without... and both with --sync-async and without.

/cc @lrhn

It's this with or without the_nullFuture CL? (See other thread). (On my
phone so can't look it up.)
If the null future is in the root zone this code probably won't finish. I
think it should finish with my CL.
On Thu, Apr 26, 2018, 05:48 Vyacheslav Egorov<EMAIL_ADDRESS>wrote:

/cc @lrhn https://github.com/lrhn
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/32982#issuecomment-384496127,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIO2jcLyFPRpchidV4Oj4QF2JlNHyr7Iks5tsTWbgaJpZM4TkbvO
.

You only flush microtasks once. That's not enough.
That happens immediately, synchronously, after calling the async function.
Async functions run their initial synchronous part when you call them, so the order of execution is:
print('1');
await Zone.root.run(() async {});  // schedules in the root zone
flushMicrotasks();  // queue is still empty, function call completes.
// root zone call and `await` statement completes.
print('2');
await null;  // schedules in the custom zone

and that is it. Nobody calls flushMicrotasks() again, so the scheduled await never completes.

@lrhn if I understand correctly, then the solution that would be impervious to the number of await statements in the root zone block would be to add the following segment (shown in the context of main):
Future<void> main() async {
  final Zone zone = Zone.current.fork(specification: new ZoneSpecification(
    scheduleMicrotask: (_, __, ___, Function microtask) {
      _microtasks.add(microtask);
    },
  ));

  Future<Null> future;
  zone.run(() {
    future = () async {
      print('1');
      await Zone.root.run(() async {});
      print('2');
      await null;
      print('3');
    }();
    flushMicrotasks();
  });

  // vvvvvvvvvvvvvvvvvvvvvvv
  await null;
  while (_microtasks.isNotEmpty) {
    flushMicrotasks();
    await null;
  }
  // ^^^^^^^^^^^^^^^^^^^^^^^

  await new Future<Null>.value(future).timeout(const Duration(minutes: 1));
}


Executing Process with Parameters it need to return Boolean

i'm trying to run processes for creating dynamic views .after running the process i need to return something as Boolean. Based on the Boolean expression  i can use dynamic view in the next line to fetch data from cube.
right now i'm using
execute(process_name, parameters=None, **kwargs)
can you please help me out i need to get the status of the process based on that i can plan for next event

Hi @Rnvksm ,
if you are on TM1 11 there is a more convenient way to execute a TI process through TM1py.
The execute_with_return method is convenient because it provides more information in case the process fails.
Here is how you can use it:
import configparser

from TM1py.Services import TM1Service

config = configparser.ConfigParser()
config.read(r'..\config.ini')

with TM1Service(**config['tm1srv01']) as tm1:
    success, status, error_log_file = tm1.processes.execute_with_return(
        process_name="Bedrock.Server.wait",
        pWaitSec=2)

    if success:
        print("Process executed successfully")
    else:
        print("Process execution failed. Status: " + status + ". Path to error log file: " + error_log_file)


Adopt a codestyle?

Hi @markpeek
Just wondering if it'd be worth as a validator to adopt something like black  and use pre-commit to standardize code-style and pre-commit validations ?
Just a thought ;)

@markpeek @PatMyron thoughts?

flake8 is in place, which I prefer over black:
https://github.com/cloudtools/troposphere/blob/136c72ad9afd3ed6a07eb7d0571ae532264e7ebe/.travis.yml#L23

Just putting in my two cents, usage of black is not a question of preference but time savings and efficiency.
For maintainers and contributes, not needing to even think of formatting code because it is done automatically can be invaluable in large scale open source projects. It also cuts down on the effort needed to review PRs as the formatting it enforces favors smaller diffs (not a huge difference here but it is noticeable).
Pairing black with isort and running them via pre-commit saves me and my team quite a bit of time during development and when reviewing PRs.

Indeed I do too @ITProKyle hence the suggestion. Could be good timing too to adopt one with the 3.0 release.
@PatMyron correct me if I am wrong but flake8 is more focused on code quality with regards to PEP etc. than codestyle like black does?

@markpeek I am guessing we can close this one too now we have 3.0 :rocket:
So glad about isort + black


docs: update guide

:tada: This PR is included in version 1.0.0-next.19 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


After reboot pi can't start application

after i reboot my raspberry pi 3 i can't to start application.
i use database is mysql
it result after i run
"sudo java -Xms32m -Xmx256m -Dlogback.configurationFile=../conf/logback.xml -Dmc.conf.file=../conf/mycontroller.properties -cp ../lib/* org.mycontroller.standalone.StartApp
"

thank you for your attention ;).

@toygame Why don't you try mycontroller/bin/start.sh?

i have a same problem when i try, and i think problem occur from mysql because when i backup with mysqldump it have error "mysqldump error output: mysqldump: Got error: 1033: Incorrect information in file: './mycontroller/example.frm' when using LOCK TABLES"

@toygame I guess this very late response. Sorry for that. can you stop MyController and do MySQL backup?


[Contributing] repo probably shouldn't have both .prettierrc.json and tsfmt.json

Because it has .prettierrc.json and I had the prettier extension installed and active, when I save any files within this repo, it formats them with prettier.  The precommit hooks then complain that the file is formatted improperly.

Nope this is not a duplicate of that issue

@jedwards1211 What specific rules conflict?
If I remove it then I have to manually enable the formatting rules for my code...

Or maybe I misunderstand, is prettier used for different file types than tsfmt or something?  I assume there would be no point to using both on ts files.  Also I had assumed that the prettier extension made itself the default formatter in VSCode because a config was found, but seems like it was just that I was already using the prettier extension

In any case maybe for some people the guidance to just run the Format Document command on .ts in this repo just works, but for me everything is messed up because it formatted with prettier instead.

Yeah, our idea was to leave it there, so team member with prettier (me) can use it, but we didn't enforce everyone onto prettier.
If you are contributing, you should use TS formatter. There should be a choice when 2 formatters are available.

And is there any configuration file in the repo that can make it use tsfmt on save by default/ask to install the tsfmt extension?  If nothing else we should make a package script to run the formatter

For me, I run my files on prettier. We have precommit hook to check formatting:

If it succeeds, I do nothing
If it fails, I format with TS formatter again

Is there any configuration file in the repo that can make VSCode use tsfmt on save by default/ask to install the tsfmt extension?

No. Some of us would be annoyed by that.
When you commit the file the hygiene will complain if it sees non-formatted code. If it doesn't complain, you are good.

I see.  This confused me a lot when making my first PR 🙃


[Backport 3.11.1][JENKINS-69149] Set 'Known hosts file' strategy as default

Target branch should be changed to equal to 3.11.1.
Backport to 3.11.1 for https://github.com/jenkinsci/git-client-plugin/pull/882
JENKINS-69149

@Pldi23 I've created the https://github.com/jenkinsci/git-client-plugin/tree/3.11.x branch so that you can change the target of this pull request to that branch.

Thanks Mark, destination branch changed to 3.11.x

fwiw it would've been nice for this to be categorized as a breaking change as well, since it totally had potential to break existing usage relying on the previous behavior

fwiw it would've been nice for this to be categorized as a breaking change as well, since it totally had potential to break existing usage relying on the previous behavior

I've consistently seen that any change to the git client plugin will be perceived as a breaking change by some users.  That's one of the reasons that a future version of the git client plugin will switch to versions numbers generated by continuous delivery.  Users will need to read the changelog to decide if a release is a breaking change.

I hear that, truly I do, and I appreciate all the work you all do maintaining this.
It's just that in this case it seems like a default auth check behavior was changed, which in most contexts I would consider a breaking change unless it happened to make things less strict by default.
however, since this effectively changed from "trust on first connection" to "only trust known connections", I'd def lean towards calling it a breaking change for the decent amount of folks probably relying on that first connection to just work.

1 for trust on first connection.
Otherwise how can I setup initial entry in known_hosts file

@clockrun You're welcome to set it to "first connection".  It is configured from the "Configure global security" page.  We had many complaints from OpenSSH 7.4 users (CentOS 7, AWS Linux 2, etc.) that prompted the change from the "first connection" as default to "known hosts file".  We don't plan to change it back.
With static agents, there is not much difference between "first connection" and "known hosts file", since the known hosts file likely exists in both cases.
The git plugin documentation includes an example that configures for a manually specified known hosts file.  You can do the same or you can configure it from the "Configure global security" page to specify the contents of the known hosts file.

Completely agree with @joshsleeper that this is indeed a breaking change, as it broke CI for my entire engineering team (we use ephemeral Jenkins agents) before we identified the issue and applied the workaround. This situation should not be possible for a patch update.

Users will need to read the changelog to decide if a release is a breaking change.

This isn't scalable, especially for contexts where several "safe" updates are automated by bots. (Also, a breaking change is usually not ambiguous–for example, any change in default behavior can be seen as breaking or potentially breaking.)
What could be more reasonable is to enforce using conventional commits in this repo to mark each code change appropriately at development time, and implement CD that will take these into account in its versioning. This will make it easier for Jenkins users, especially those using lots of plugins, to know when they should be able to safely upgrade or when they should expect breakage and read the changelog.
Please, please consider something like this to maintain semantic versioning over the {increment}.{hash} versioning system described in the page you linked to! Thanks! 🙏


Add material for ENGAGE capacity building workshop

This PR adds the links for the ENGAGE capacity building material.

Thanks @LauWien - I changed the link for the Strommarkttreffen to point to pure directly instead of showing the DOI.
About the languages, I think that it should be clear that by default all material here is in English - and we only add the language if it is ambiguous (which it is because the Strommarkttreffen is a German-language event).

About the languages, I think that it should be clear that by default all material here is in English - and we only add the language if it is ambiguous (which it is because the Strommarkttreffen is a German-language event).

Makes sense! We could then think about also deleting "(english)" in the bullet point of "Tutorial notebook", but also good as is.


Symfony 5 + preloading triggers Warning: Cannot declare class

Using preloading on Symfony 5 triggers the following error:
ErrorException:
Warning: Cannot declare class Sentry\SentryBundle\EventListener\RequestListenerRequestEvent, because the name is already in use

  at /var/www/html/vendor/sentry/sentry-symfony/src/EventListener/RequestListener.php:17
  at include_once('/var/www/html/vendor/sentry/sentry-symfony/src/EventListener/RequestListener.php')
     (/var/www/html/var/cache/dev/ContainerTnf83nT/App_KernelDevDebugContainer.php:613)
  at ContainerTnf83nT\App_KernelDevDebugContainer->ContainerTnf83nT\{closure}()
     (/var/www/html/vendor/symfony/dependency-injection/Container.php:156)
  at Symfony\Component\DependencyInjection\Container->set('kernel', object(Kernel))
     (/var/www/html/vendor/symfony/http-kernel/Kernel.php:551)
  at Symfony\Component\HttpKernel\Kernel->initializeContainer()
     (/var/www/html/vendor/symfony/http-kernel/Kernel.php:131)
  at Symfony\Component\HttpKernel\Kernel->boot()
     (/var/www/html/vendor/symfony/http-kernel/Kernel.php:191)
  at Symfony\Component\HttpKernel\Kernel->handle(object(Request))
     (/var/www/html/public/index.php:28)

It's probably related to the class_alias functions

Hey there! 3 months since the patch. May it be possible to get a new bug fix release for this? Thanks!

My bad, I forgot about this because that wasn't included in the changelog. I'll try to do it ASAP.


SQLite accelerator doesn't support date_part

Queries that use e.g. extract(year), like some of those from TPCH (Q9) fail on SQLite accelerated datasets:
Query Error Unable to query arrow: ConnectionError Rusqlite("no such function: date_part in SELECT profit.nation, profit.o_year, sum(profit.amount) AS sum_profit FROM (SELECT nation.n_name AS nation, date_part('YEAR', orders.o_orderdate) AS o_year, ((lineitem.l_extendedprice * (CAST(1 AS DECIMAL(20,0)) - lineitem.l_discount)) - (partsupp.ps_supplycost * lineitem.l_quantity)) AS amount FROM part JOIN supplier ON true JOIN lineitem ON true JOIN partsupp ON true JOIN orders ON true JOIN nation ON true WHERE (((((((supplier.s_suppkey = lineitem.l_suppkey) AND (partsupp.ps_suppkey = lineitem.l_suppkey)) AND (partsupp.ps_partkey = lineitem.l_partkey)) AND (part.p_partkey = lineitem.l_partkey)) AND (orders.o_orderkey = lineitem.l_orderkey)) AND (supplier.s_nationkey = nation.n_nationkey)) AND part.p_name LIKE '%green%')) AS profit GROUP BY profit.nation, profit.o_year ORDER BY profit.nation ASC NULLS LAST, profit.o_year DESC NULLS FIRST at offset 108")

SQLite accelerator most likely needs a dialect set, similar to the fix for MySQL in #1951

Upstream datafusion change: https://github.com/apache/datafusion/pull/12161


Make the quickstarts independent of which Camel version they are using

So end users are not stuck on what version that may be in the fabric8 bom.
IMHO really fabric8 bom should not have any AMQ/Camel/CXF dependency etc. But really let end user pick and chose what they want to use.

:+1:

This requires the archetype-builder to allow to put in version placeholders in the generate archetypes

Almost though I got it covered then the generated mvn archetype may have a name clash, so the user entered version is not inserted into the generated pom.xml

Darn in some mvn archetype hell where it wont pickup my latest changes, but it looks like the placeholders are all in place. So just trying with a trashed local m2 repo.

Yay the maven archetype catalog is a bit of a mess to control. But with a trashed m2 I only have my local stuff now, and I was able to generate a project and specify Camel 2.16.0 instead of the default 2.16.1
 <!-- configure the versions you want to use here -->
    <camel.version>2.16.0</camel.version>
    <weld.version>2.3.1.Final</weld.version>

To generate local use
mvn archetype:generate -DarchetypeCatalog=local

Next release of fabric8 has the needed change in the archetype builder that these quickstarts uses, but then we can choose our Camel / ActiveMQ / spring-boot versions and so on.

@davsclaus actually productization guideline was to put dependency versions in parent fabric8 BOM so that users don't have to specify versions explicitly. The redhat branch only needed versions for plugins, which don't inherit from imported poms, afaik.

yes but this is upstream, we want to be free and let people chose their bits


std types for datalink

Note sure if this should be in ringserver or libdali, but would be nice if there was a convention on types in datalink stream ids.
Currently /MSEED is common for miniseed2. Perhaps also define:
/MSEED3 for miniseed3
/JSON for JSON
/BZJSON for bzip2 compressed JSON
Not sure there would ever be a use case for this, but perhaps also QuakeML and StationXML types?

My intention is to use:

/MSEED for miniSEED of undefined version
/MSEED2 for miniSEED v2
/MSEED3 for miniSEED v3

If anywhere it should go into the DataLink documentation included with libdali.  These will always be guidelines in my mind and not requirements or limits to what can be used.


Changed theme to sphinx_rtd_theme

The theme for the documentation has been changed to the default ReadTheDocs one, wich uses a much nicer layout and design of the classes and their properties:

Codecov Report

Merging #52 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master     #52   +/-   ##
======================================
  Coverage    91.6%   91.6%
======================================
  Files          31      31
  Lines        1405    1405
======================================
  Hits         1287    1287
  Misses        118     118

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c7299b...649d700. Read the comment docs.


Update Line 2

The title just said Continuous Integration whereas the blurb below says Continuous Integration and Continuous Deployment.

Why:

Closes [issue link]
What's being changed:

Check off the following:

[x] I have reviewed my changes in staging. (look for the deploy-to-heroku link in your pull request, then click View deployment)
[x] For content changes, I have reviewed the localization checklist
[x] For content changes, I have reviewed the Content style guide for GitHub Docs.

Writer impact (This section is for GitHub staff members only):

[x] This pull request impacts the contribution experience

[x] I have added the 'writer impact' label
[x] I have added a description and/or a video demo of the changes below (eg. a "before and after video")

@mattrowe85 Thanks so much for opening a PR! I'll get this triaged for review ⚡


Tell HHAST to only look at open files

Given ALE only cares about open files, this has no observable change,
except for significantly reduced resource usage.
Didn't include this in the original PR as I thought there was some subtle incompatibly. Turns out that actually, HHAST was ignoring rootUri/rootPath, and always used $CWD - vscode and atom spawn it from cwd, but that's not required. Have hopefully fixed that issue in the next release of hhast.

(the default behavior is to prioritize open files, but to start linting the whole tree the moment the initialize command is received)

Cheers! :beers:


Pro UI version is different on the home page and details

Not reproducable at the moment so I'm closing this. There is a UI test available for this, so if it happens again, we will further investigate it.


Fix missing reference errors when loading assetbundle information

Create branch: issue-1-fix-missing-reference-errors-when-loading-assetbundle-information from main.


Fixed Bug Resting\Bun in oven changePrices bug

changePrices tries to convert Resting \ Bun in the oven status into US dollars which displays a NaN status. This change prevents that from happening.

The return; in my fix should have been a continue;
                      let text = item.getElementsByTagName('span')[0].innerText;

                      if(text.toUpperCase().indexOf("HOURS")>0){
                          continue;
                      }

Before this fix

After this fix

This should be fixed now.


Create free-courses-zh.md

Chinese Tutorial 🤔
Check yourself 😉
If there is any error in this please close it
(Close This free-courses-zh.md)

Review Note
To be Merged jointly, one following by the other, with #7014

Chinese Tutorial 🤔
Check yourself 😉
If there is any error in this please close it
(Close This free-courses-zh.md)

... and of course close related PRs

Review Note
To be Merged jointly, one following by the other, with #7014

Chinese Tutorial 🤔
Check yourself 😉
If there is any error in this please close it
(Close This free-courses-zh.md)

... and of course close related PRs

Review Note
To be Merged jointly, one following by the other, with #7014

Chinese Tutorial 🤔
Check yourself 😉
If there is any error in this please close it
(Close This free-courses-zh.md)

... and of course close related PRs

Please read https://www.loc.gov/standards/iso639-2/faq.html#24  At least one of these courses is in cantonese, and we need to decide how to handle Mandarin vs. Cantonese (not an issue for written resources.
Please close #7014 and include the changes there with this PR. As we've seen with other new lists, having two PRs is very error prone.
I will not merge this PR without feedback from Mandarin/Cantonese speakers.

Let me close these two @eshellman

I'm closing it because I can't add Chinese here


Git changes panel shows mid-checkout changes (is not atomic)

Issue Type: Bug
To reproduce, it is necessary to have a large repo and 2 branches with a large number of changes. The VSCode repo is likely large enough, Chromium certainly is large enough provided branches with a large enough changeset are used.

Open the git changes panel (under the source control tab)
In the terminal, git checkout ... to another branch
Observe git changes panel updates before checkout complete, showing the incomplete working directory changes.

The nature of this issue means it cannot always be reproduced. Available system resources, persistent storage speed and the file watcher implementation are just some of the factors that would influence the ability to reproduce this. Once reproduced on a given machine (e.g. by finding a repo with which the issue can be manifested), it should be possible to reproduce with a high degree of success.
VS Code version: Code 1.58.2 (c3f126316369cd610563c75b1b1725e0679adfb3, 2021-07-14T22:09:06.581Z)
OS version: Darwin x64 20.5.0
Restricted Mode: No

System Info

Item
Value

CPUs
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (16 x 2400)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
64.00GB (26.99GB free)

Process Argv
--crash-reporter-id b35ae86e-0d23-4efb-8807-593dfaef059f

Screen Reader
no

VM
0%

Extensions (16)

Extension
Author (truncated)
Version

Bookmarks
ale
13.1.0

cucumberautocomplete
ale
2.15.1

vscode-css-modules
and
1.2.3

vscode-bazel
Baz
0.4.1

vscode-css-modules
cli
0.2.3

vscode-eslint
dba
2.1.23

gitlens
eam
11.6.0

EditorConfig
Edi
0.16.4

vscode-edge-devtools
ms-
1.2.0

debugger-for-edge
msj
1.0.15

indent-rainbow
ode
7.5.0

vscode-commons
red
0.0.6

vscode-yaml
red
0.22.0

vscode-stylelint
sty
0.86.0

markdown-all-in-one
yzh
3.4.0

vscode-proto3
zxh
0.5.4

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
pythonvspyt602:30300191
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492cf:30256860
pythonvspyt639:30300192
pythontb:30283811
pythonvspyt551cf:30311713
vspre833:30321513
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
pythondataviewer:30285071
vscus158:30321503
pythonvsuse255:30340121
vscod805cf:30301675
pythonvspyt200:30340761
vscextlangct:30333562
binariesv615:30325510
vsccppwt:30329788
pythonvssor306:30344512
bridge0708:30335490

Throwing this out there for the sake of ideation (since over dependence on file watching in a massive monorepo has scaling issues).
Git supports global hooks. With this it may be possible to tweak the git integration such that refreshes are guided by the start and completion of git commands. Much like how the git integration optionally provides a git credential manager implementation for WSL.
If viable this could drastically reduce system resource utilisation while git commands are executed, and reduce the opportunity for mid-process changesets to be shown in changes panel.

Thanks for filing the issue and apologies for not reacting to this until now. The "Source Control" view is refreshes based on file-system events as well as the completion of git operations. One potential solution to this which I can explore is to discard the file-system events while a write operation is running since we will refresh the state when the write operation completes.


Ayo the pizza here

AHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Changes

Checks
By submitting your pull request for review, you agree to the following:

[ ] This contribution was created in whole or in part by me, and I have the right to submit it under the terms of this repository's open source licenses.
[ ] I understand and agree that this contribution and a record of it are public, maintained indefinitely, and may be redistributed under the terms of this repository's open source licenses.
[ ] To the best of my knowledge, all proposed changes are accurate.

I'm good at building with btools in Roblox I don't know how to code but I messed someone my code and idk what's gonna happen, AND I'm only good for building and talking,making friends my Roblox username is called cheepytheturkey

What do u mean do not merge this?


Implement support for an MLLP connection idle timeout.

Fixes #11

@rectalogic Looks reasonable to me. Thanks for the improvement!  Does it solve the issue you've run into with the VPN connection loss?
Do you mind adding a quick note to docs/changelog.rst about the new getTimeout()? I'm think 0.4.1 works for this.


Strange behaviour in JS functions

Hi all! First of all, this is the first version of the algorithm, so I know things could get done little better. I'm implementing Pearson's correlation coefficient.

get all movies I rated
get all movies from other users
find intersections
calculate the similarity coefficient

Look at the line before return movies. If I execute queries directly, I get the list of movies for userId = 1.
var gdb = orient.getGraph();

// get all movies from user
var movies = gdb.command( "sql", "select movieId, name from (select expand(out('Rate')) from User where userId = ?);", [ userId ] );

var allUsers = gdb.command( "sql", "select userId from User;" );

for(var i = 0; i < allUsers.length; ++i) {
  var user = allUsers[i];

  // get all movies user rated
  var movies = gdb.command( "sql", "select movieId, name from (select expand(out('Rate')) from User where userId = ?);", [ user.userId ] );

  // testing
  return movies;
}

When I return a user, I get:
[
    {
        "@type": "d",
        "@rid": "#-2:0",
        "@version": 0,
        "userId": 1
    }
]

My conclusion is that I cannot get the value user.userId from the top of the for loop. When I execute the SQL directly:
select movieId, name from (select expand(out('Rate')) from User where userId = 1);

I get the results:
1       Toy Story (1995)
266    Legends of the Fall (1994)

Hi @warriorkitty
I think the user is an ODocument (a Java object), not a JS one, could you please try with
var movies = gdb.command( "sql", "select movieId, name from (select expand(out('Rate')) from User where userId = ?);", [ user.field("userId") ] );

and let me know if it works?
Thanks
Luigi

I've tried returning the value:
return { userId: user.field("userId") };

Getting an error:
Error on parsing script at position #0: Error on execution of the script\nScript: getRecommendations\n------^\nTypeError: v(null)[#-2:0] has no such function \"field\" in <eval> at line number 13\nTypeError: v(null)[#-2:0] has no such function \"field\"

Ops, sorry, you are using the Graph API (not Document), so the query is returning a vertex.
This is the right syntax:
return { userId: user.getProperty("userId") };

Thanks
Luigi

That works. You are awesome, as usual. Thank you.


Autolink can't start immediately after [

From the gfm spec:

All such recognized autolinks can only come at the beginning of a line, after whitespace, or any of the delimiting characters *, _, ~, and (.

Here's a go program that runs markdown and goldmark on the following input: https://go.dev/play/p/kTjBshQ82iQ
[https://golang.org]

Rendered in github as
[https://golang.org]
The output of the program is
markdown:

[https://golang.org]

goldmark:

[https://golang.org]

The spec is full of lies:

input
GitHub

xhttps://go.dev
xhttps://go.dev

0https://go.dev
0https://go.dev

%https://go.dev
%https://go.dev

αhttps://go.dev
αhttps://go.dev

[https://go.dev
[https://go.dev

It is pretty funny that you can have an autolink after a 0 or α or % but not [.

How many of these did you find? I am inclined to leave [https://go.dev] auto-linking.

I think I saw just one or two on the sample I looked at. I'm okay with leaving this as is

Please reopen if anything changes.