AuthorSameAsContributor

Your PR targeted the live branch, which is not allowed. The PR has been redirected to the default branch (main).

Microsoft authors of articles need to make updates through the private repository, so the updates can be staged and validated by current validation rules. After you review staging and fix all validation issues, you can add the sign-off comment to let the PR reviewers know the updates are ready to be merged.
It's easy to access the private repo version of your articles in GitHub. Just add -pr to the URL, as follows:
 Public repo URL: https://github.com/MicrosoftDocs/{docs-repo}
 Private repo URL: https://github.com/MicrosoftDocs/{docs-repo-pr}
We have closed this pull request. Please resubmit the updates through the private repo. For more information, see the contributor guide.

Microsoft authors of articles need to make updates through the private repository, so the updates can be staged and validated by current validation rules. After you review staging and fix all validation issues, you can add the sign-off comment to let the PR reviewers know the updates are ready to be merged.
It's easy to access the private repo version of your articles in GitHub. Just add -pr to the URL, as follows:
 Public repo URL: https://github.com/MicrosoftDocs/{docs-repo}
 Private repo URL: https://github.com/MicrosoftDocs/{docs-repo-pr}
We have closed this pull request. Please resubmit the updates through the private repo. For more information, see the contributor guide.

Docs Build status updates of commit ae8996c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

CSIDevDocsPublic/auto-merge.md
:white_check_mark:Succeeded

csidevdocs-public/docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

PRMerger Results

Issue
Description

Added or Deleted File(s)
This PR contains added or deleted file(s).

File Change Percent
This PR contains file(s) with more than 20% file change.

Json File
This PR contains json file(s).

Repository root file
This pull request contains a file submitted to the root of the repository.

Docs Build status updates of commit ae8996c:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error: PullRequestMergeConflict] Cannot sync git repo to specified state. Please make sure the pull request is mergeable.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Your PR targeted the live branch, which is not allowed. Automatically redirect to the default branch (main) failed. Cannot change the base branch of a closed pull request.


Account drop-down options are inconsistent with selected third-party account

On the market page, the drop-down options are limited to your own accounts (which makes sense) but it is possible to land on this page with somebody else's account selected via the dashboard (in this case I selected fav). This makes it inconsistent with the options available in the drop-down.

I don't really see this as a huge issue. Obviously you can't make any trades if you don't own that account...need to think through the best way to handle it....maybe an icon next to the account that indicates it's not one of your accounts essentially disabling trade buttons, etc. Will slate this for UI in mid December.

Sorry wmbutler but this was such an easy fix I just did it: if you've previously selected an account that isn't yours and then go to the trade page, that account will now be included in the dropdown.


Upgrade to SONiC 202205

@ksandeep-palc Thanks. Please clean up the commits.
Also, let's use the following for git use info.

git config user.name "Goldstone"
git config user.email<EMAIL_ADDRESS>

@ishidawataru - we need to generate an image based on the changes in this PR and build the usonic-debug:202205 image and push the ghcr.io, so that I can do a full build in goldstone-buildimage. Atleast the system is coming up and is stable, and I am able to do a no-shut from gscli which is getting reflected in bcmsh 'ps' output, so we are good.

@ishidawataru please also merge - https://github.com/microsonic/sonic-buildimage/pull/1 I am not able to add either tag you or add you as reviewer in that PR.

@ksandeep-palc Have you already built an image based on these changes?

@ksandeep-palc Have you already built an image based on these changes?

Yeah @ishidawataru I have build an INSTALLER image based on these changes.

@ksandeep-palc That's nice. Are you using the sm-sonic-202205 branch to build the image? Does goldstone-buildimage repo need any modification other than sonic-buildimage submodule update to accomodate the changes in this PR?

@ksandeep-palc Please fix the conflicts.

@ksandeep-palc Please fix the conflicts.

@ishidawataru Sorry I dint get what conflicts we are referring to, I don't see any conflicts in this PR

@ksandeep-palc Please fix the conflicts.

@ishidawataru Sorry I dint get what conflicts we are referring to, I don't see any conflicts in this PR

OK you are referring to goldstone-buildimage PR conflicts.

@ksandeep-palc This repo uses rebase and merge policy to merge the PR. This branch cannot be rebased due to conflicts. Please do git rebase origin/202205 on this branch and fix the conflicts.

@ishidawataru Please see if this can be merged now, else will have to do it similar to goldstone-buildimage PR.

closed by #14


SSLError raised while 'r.config._ssl_url = None'

Issue Description
According to a comment on issue #202, you can disable SSL by adding .config._ssl_url = None to a script.
However, even with SSL disabled, I am encountering multiple SSL Errors when trying to use PRAW on a company network that uses self-signed certificates.
Code
import praw

reddit = praw.Reddit(client_id='####',
                     client_secret='####',
                     user_agent='windows:testapp (by /u/stevoisiak)')
reddit.config._ssl_url = None  # Disable SSL

for submission in reddit.subreddit('learnpython').hot(limit=10):
    print(submission.title)

Traceback
Traceback (most recent call last):
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3-1.22-py3.6.egg\urllib3\connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3-1.22-py3.6.egg\urllib3\connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3-1.22-py3.6.egg\urllib3\connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3-1.22-py3.6.egg\urllib3\connection.py", line 326, in connect
    ssl_context=context)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3-1.22-py3.6.egg\urllib3\util\ssl_.py", line 329, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\ssl.py", line 814, in __init__
    self.do_handshake()
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests-2.18.4-py3.6.egg\requests\adapters.py", line 440, in send
    timeout=timeout
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3-1.22-py3.6.egg\urllib3\connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3-1.22-py3.6.egg\urllib3\util\retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.reddit.com', port=443): Max retries exceeded with url: /api/v1/access_token (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\requestor.py", line 47, in request
    return self._http.request(*args, timeout=TIMEOUT, **kwargs)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests-2.18.4-py3.6.egg\requests\sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests-2.18.4-py3.6.egg\requests\sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests-2.18.4-py3.6.egg\requests\adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.reddit.com', port=443): Max retries exceeded with url: /api/v1/access_token (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/svascellar/.PyCharmCE2018.1/config/scratches/praw test.py", line 8, in <module>
    for submission in reddit.subreddit('learnpython').hot(limit=10):
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\praw\models\listing\generator.py", line 52, in __next__
    self._next_batch()
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\praw\models\listing\generator.py", line 62, in _next_batch
    self._listing = self._reddit.get(self.url, params=self.params)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\praw\reddit.py", line 371, in get
    data = self.request('GET', path, params=params)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\praw\reddit.py", line 486, in request
    params=params)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\sessions.py", line 182, in request
    params=params, url=url)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\sessions.py", line 125, in _request_with_retries
    retries, saved_exception, url)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\sessions.py", line 91, in _do_retry
    params=params, url=url, retries=retries - 1)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\sessions.py", line 125, in _request_with_retries
    retries, saved_exception, url)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\sessions.py", line 91, in _do_retry
    params=params, url=url, retries=retries - 1)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\sessions.py", line 113, in _request_with_retries
    data, files, json, method, params, retries, url)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\sessions.py", line 98, in _make_request
    params=params)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\rate_limit.py", line 32, in call
    kwargs['headers'] = set_header_callback()
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\sessions.py", line 142, in _set_header_callback
    self._authorizer.refresh()
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\auth.py", line 299, in refresh
    self._request_token(grant_type='client_credentials')
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\auth.py", line 138, in _request_token
    response = self._authenticator._post(url, **data)
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\auth.py", line 29, in _post
    data=sorted(data.items()))
  File "C:\Users\svascellar\AppData\Local\Programs\Python\Python36-32\lib\site-packages\prawcore\requestor.py", line 49, in request
    raise RequestException(exc, args, kwargs)
prawcore.exceptions.RequestException: error with request HTTPSConnectionPool(host='www.reddit.com', port=443): Max retries exceeded with url: /api/v1/access_token (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))

System Information

PRAW Version: 5.4.0
Python Version: 3.6.4
Operating System: Windows 7 Professional - Service Pack 1 (64 bit)

The approach used then isn't going to work today.
PRAW uses requests, which has support for disabling certificate validation via verify=False: http://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification
This use to be a first-class feature in PRAW3, but it looks like it didn't make it to PRAW4+, and you appear to be the first to ask for it. Perhaps you can figure out what changes need to be made to change the internal request session to disable verification? It might be as easy as something like reddit.http.verify=False, but I'm guessing it's a bit more complicated than that.

@bboe Thanks for the helpful response. I'll try seeing if I can configure requests to disable SSL.
It's worth mentioning there's a Stack Overflow question on this issue.

Where does PRAW initialize Requests?
There's a docstring referencing Requests.session in reddit.py, but I can't find where the library is called.
https://github.com/praw-dev/praw/blob/49f33154b901d5686571d120d69dfe74b1a4d309/praw/reddit.py#L110-L120

It nested in prawcore:
https://github.com/praw-dev/prawcore/blob/master/prawcore/requestor.py#L33

Note that Requestor.request passes all its arguments directly to its session's request method. So you might not need to go that deep.

@Stevoisiak I was able to use a custom requests Session using the following code:
import requests

import praw

s = requests.Session()
s.verify = '/path/to/certfile'
reddit = praw.Reddit(requestor_kwargs={'session': s})  # and other params

I believe this is what you need to do when using self-signed certificates (source: Requests documentation), but I don't have experience with that so I may be wrong.
I hope this helps!

@jarhill0 Specifying a .pem certificate worked perfectly.
import praw
import requests

session = requests.Session()
session.verify = r"C:\path\to\cert\certificate.pem"
reddit = praw.Reddit(client_id='###',
                     client_secret='###',
                     user_agent='windows:testapp (by /u/stevoisiak)',
                     requestor_kwargs={'session': session})

for submission in reddit.subreddit('learnpython').hot(limit=10):
    print(submission.title)

Alternatively, you can fully disable SSL with session.verify = False. Although this will cause each request to raise an InsecureRequestWarning.
(Be warned that disabling SSL is strongly discouraged by the urllib3 documentation)
import praw
import requests

session = requests.Session()
session.verify = r"U:\Documents\Other\Certificates\NORTHWELL-ENTCAROOT-CA.pem"
reddit = praw.Reddit(client_id='###',
                     client_secret='###',
                     user_agent='windows:testapp (by /u/stevoisiak)',
                     requestor_kwargs={'session': session})

for submission in reddit.subreddit('learnpython').hot(limit=10):
    print(submission.title)

@jarhill0 You can write up the docs. Feel free to copy from my SO answer if you'd like.
Thank you for your help on this issue!


naming, new properties

Trying to better align _active vs _enabled with upstream terminology.
Added a bunch of new properties. Still not everything but closer.

@PeteBa can you review this against your knowledge of OpenSprinkler and let me know if the terminology better aligns?

On the integration side I think it would just be enabled but then adding a bunch of extra datapoints to the state attributes (the one changes meaning, but it's the same name I think).
My brain is starting to go in circles with this..but I really like how the integration is coming together honestly. I think with a little more work on the services we'll have a kick-ass 1.0 :)

Should we merge this?

Let me see if I can get 1 or 2 more datapoints in there (trying to figure out the externalIP bit shifting stuff). Then I'd say yes. Then we can begin to update the integration with all the new stuff.

Doing a release


Add the Original link to the article page

The link to the Original content is very useful for the odd feed that doesn't fetch correctly and the Fetch Original Content function doesn't work either (NPR...). But currently if you are looking at an article you have to mark it unread then back out to the list of articles and click on the Original link. Simply adding the identical link as already exists in the view of all the articles to the view of a single article would speed things up a lot.

Clicking on the article's title will lead you to the original link if I'm not mistaken. Can you please try it?

it does! didn't even notice it was a link sorry for using up your time, much appreciate the answer.


🚀 Final Group Capstone - Book A Flight

Features included in this pull request are as follows:

[x] The user logs gains access to the website by typing username and email

[x] In the navigation panel, the user can see links to:

Flights

"Reserve" form.

"My reservations".

"Add Flights

"Delete Flight

[x] On the main page, the user can see a list of flights

[x] When the user selects a specific item, they can see the details page with its full description

[x] In the details page, the user can click the "Reserve" button

[x] When the user clicks the "Add Flight" link in the navigation panel they can see a form for adding a new item

[x] The app is responsive

[x] When the user clicks the "Delete" link in the navigation panel they can see a list of all flights with title and "Delete" button.

[x] When the user clicks the "Delete" button, the selected flight is marked as removed and does not show on the main list anymore.

[x] To reserve a flight, the user has to select a date and city

[x] The user can also access the "Reserve" page from the navigation panel.

[x] When the user clicks the "My reservations" link in the navigation panel they can see a list of their reservations (with information about the flight name, date and city).

[x] Full documentation for the API was added to the readme file..

Technical set up

[x] The repository is set up on GitHub and Gitflow was used
[x] Postgres was used as your database
[x] Rails framework was used to create the backend API.
[x] React & Redux were used to create frontend UI.
[x] Credit was given to the author of the design in the readme file.

Thank you so much @Meltrust for your relentless effort and skill with which you figured out hidden faults in our code. We sincerely appreciate your efforts. Thank you.


Update or replace manifestVersion

Summary of the new feature / enhancement
In prior conversations and in the decomposed schema, I have the manifestVersion key in the resouce manifest documented as defining the version of the resource manifest schema that the manifest adheres to, requiring it to be semver and limited to the versions of the schema we support, which is currently 1.0.
A few things:

1.0 isn't valid semver by the regex recommended on semver.org - it would have to be 1.0.0 (and should maybe more accurately be 0.1.0)
I thought I remembered there being a check on this version in the code, but a quick check didn't turn one up.
If this is actually referencing the canonical schema for the manifest, should this maybe just be the $schema key and point to the resource manifest schema's $id?

Proposed technical implementation details (optional)
Either:

Update manifestVersion to use a valid semantic version string, like 0.1.0
Replace manifestVersion with the $schema keyword and require the specified schema to be one of the list of supported schemas.

@michaeltlombardi I think using $schema is the better option, can you suggest the URL?

I would recommend the bundled schema for the manifest, which is currently:
https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2023/08/bundled/resource/manifest.json

This will also automatically associate the JSON file with the schema for IntelliSense and hover help (though not the extended authoring schema, which is something we can investigate - same URL, but manifest.vscode.json).
I would recommend making the value an enum so we can be forward-thinking about managing schema versions. Also, in the next release, we could mark the manifestVersion key as deprecated and indicate they should migrate to using $schema, then remove manifestVersion in the next release or two.


use up to date props on state change

Fixes #69

Coverage increased (+0.01%) to 97.917% when pulling cdc94c40f9c8850295495e99fdd650c369fcc507 on hotfix-props-state-change into 4d87e99d5cff57439600e027cfbf10965f1471ac on master.


[Project] Dokku ClickHouse

It would be great to have a Dokku database plugin for ClickHouse.
Official databases plugins for Dokku:

https://github.com/dokku/dokku-postgres
https://github.com/dokku/dokku-mysql
https://github.com/dokku/dokku-mongo
https://github.com/dokku/dokku-mariadb
https://github.com/dokku/dokku-elasticsearch
https://github.com/dokku/dokku-redis
https://github.com/dokku/dokku-memcached
https://github.com/dokku/dokku-couchdb
https://github.com/dokku/dokku-rethinkdb

The plugin is available here: https://github.com/dokku/dokku-clickhouse
Coding Socks sponsored this plugin under my name on https://opencollective.com/dokku.


 When vector is eventually destroyed is tried to free memory by
calling MemoryPool which has been destroyed.
This change adds a check for remaining allocated bytes being zero to memory pool
destructor. This gives a clearer error earlier (when pool is destroyed, not
vector).
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: (0 vs. 192) Memory pool should be destroyed only after all allocated memory has been freed. Remaining bytes allocated: 192
Retriable: False
Expression: 0 == remainingBytes
Function: ~MemoryPoolImpl
File: /Users/mbasmanova/cpp/velox-1/./velox/common/memory/Memory.h
Line: 434

Looks like we already discover some bugs in unit tests

Looks like we already discover some bugs in unit tests

Looks like so. I'm looking into these.

Here are failing tests:
132: [ RUN      ] ApproxDistinctTest.groupByIntegers
131: [ RUN      ] RowNumberTest.basic
135: [ RUN      ] LastAggregateTest.tinyIntGroupBy
138: [ RUN      ] TpchConnectorTest.simpleAggregation
139: [ RUN      ] AssertQueryBuilderTest.orderedResults

I'm seeing that memory tracker sometimes doesn't decrement usage on free. CC: @oerling
  // Increments outstanding memory by 'size', which is positive for
  // allocation and negative for free. If there is no reservation or
  // the new allocated amount exceeds the reservation, propagates the
  // change upward.
  // Sometimes the memory pool wants to mock an update for quota
  // accounting purposes and different memory usage trackers can
  // choose to accommodate this differently.
  virtual void update(int64_t size, bool /* mock */ = false) {
    if (size > 0) {
      int64_t increment = 0;
      {
        std::lock_guard<std::mutex> l(mutex_);
        if (usedReservation_ + size > reservation_) {
          increment = reserveLocked(size);
        }
      }
      checkAndPropagateReservationIncrement(increment, false);
      usedReservation_.fetch_add(size);
      return;
    }
    // Decreasing usage. See if need to propagate upward.
    int64_t decrement = 0;
    {
      std::lock_guard<std::mutex> l(mutex_);
      auto newUsed = usedReservation_ += size;
      auto newCap = std::max(minReservation_.load(), newUsed);
      auto newQuantized = quantizedSize(newCap);
      if (newQuantized != reservation_) {
        decrement = reservation_ - newQuantized;
        reservation_ = newQuantized;
      }
    }
    if (decrement) {
      decrementUsage(type_, decrement);
    }
  }

@xiaoxmeng is going to take a closer look.


[udp] add 'kNetifThreadNative` which disallows platform UDP use

This commit introduces kNetifThreadNative as a network interface option for UDP sockets. Unlike other options, this disallows the use of platform UDP for the socket, indicating that the socket should use the OpenThread internal Thread network interface only.
This model replaces the previous approach where ShouldUsePlatformUdp() would check the socket port against a set of port numbers used by different modules (such as MLE, TMF, Joiner Router, etc) to determine whether platform UDP APIs should be used. With the new model, each module decides whether to associate its socket with the kNetifThreadNative.
This is a more flexible and extensible model, ensuring that sockets that should not use the platform do not waste resources (they will not be created or opened/closed on the platform). This also help avoid edge cases where platform UDP operations may unintentionally fail when the platform socket is not actually needed.

This PR current constains the commits from the folliwng PRs. Please check and review the last commit. Thanks.

https://github.com/openthread/openthread/pull/10962
https://github.com/openthread/openthread/pull/10964

@marius-preda, adding you just FYI. This is related to what we discussed in the last Thread MM. It should give us more control over UDP sockets and how/when otPlatUdp APIs are used.

@marius-preda, adding you just FYI. This is related to what we discussed in the last Thread MM. It should give us more control over UDP sockets and how/when otPlatUdp APIs are used.

Thanks @abtink for submitting this. Sorry I was in PTO last week and din not have a chance the review this PR. Overall looks great but I have one question: Why did you use kNetifThreadHost for the SRP server? I would expect it to be the same as the other modules that use kNetifThreadInternal.

Why did you use kNetifThreadHost for the SRP server? I would expect it to be the same as the other modules that use kNetifThreadInternal.

Thanks @marius-preda . My intention was to keep the behavior similar to what it was before this PR and just add the new kNetifThreadInternal mechanism. I used it to replace the same sockets/ports previously covered by ShouldUsePlatformUdp().
That's why I kept the Sep::Server socket as before (associated with kNetifThreadHost). We can definitely consider changing it to "internal" in a separate PR.

Edit: Just noticed I may have unintentionally changed the SRP client socket in the PR to use kNetifThreadInternal (which was not covered earlier by ShouldUsePlatformUdp()). :)


Allow pulling in this package via NPM. Don't push breaking backend changes

In the  documentation  it is strictly mentioned to only reference this library via https://www.paypalobjects.com/api/checkout.js.
Doing so does not protect our application against breaking javascript changes between versions of this library.
However, pulling in our own version via npm resulted in another application of ours breaking on Friday morning. Upgrading it from 4.0.36 to 4.0.82 resolved the issue. I'm speculating, but I'm assuming our application eventually broke Friday morning due to backend Paypal API changes.
The current system of including this library is a catch 22:

Pull in this library from PayPal's CDN and be victim to this client pushing breaking front-end changes. OR
Pull in this library via NPM and risk (unknowingly) being on an unsupported version that breaks when PayPal updates its backend API without support for older versions.

I believe the most standard solution is to have this library follow semantic versioning, and provide backend API support for previous versions (while providing notice of version deprecation so devs have time to upgrade their apps). It would also be nice to have release notes / upgrade guide for each release. As we upgraded our app from 4.0.36 to 4.0.82 we ran into breaking changes that do not belong in patch releases and had to track down what changes were made in code.
If this is not an option, what do you recommend?
Thanks.

Hi @feeekkk
Our goal is to be 100% backwards compatible, given that so many merchants are already integrated with our CDN. That should include any server-side changes.
Can you elaborate on the issue you saw with 4.0.36? I'd like to track that down as there may be others using that version.
I would still recommend using the CDN url rather than npm, given:

There should be absolutely no breaking changes in the public api provided by checkout.js

This also gives you a site-speed increase -- checkout.js is also loaded inside the checkout and button iframes / popups, so using the CDN version means your customers get a cached version when they hit the paypal app

RE point 1:
There unfortunately was a breaking change that we came across while upgrading patch versions. I think it was from the resolution of this issue https://github.com/paypal/paypal-checkout/issues/136 (which would have been a major bump in semver)
IE we had to change our implementation from paypal.ppxo.checkout.setup to paypal.checkout.setup. Some release notes or upgrade guide for this would have been nice!

The issue that we saw with 4.0.36 that we discovered on Friday:
Refused to display 'https://www.paypal.com/webapps/hermes/fallback?product=ec&fallback=1&reason=ui_unhandled_typeerror&token={redacted}' in a frame because it set 'X-Frame-Options' to 'sameorigin'.

This was resolved when upgrading to the latest version.

token={redacted} -- can you share the token? Then I can find the error. It's totally harmless to share.
I'm sorry about the paypal.ppxo change -- you're right, that should have been a major bump. I'm really sorry about the inconvenience there.

Sorry, wanted to err on the safe side.
https://www.paypal.com/webapps/hermes/fallback?product=ec&fallback=1&reason=ui_unhandled_typeerror&token=EC-3HT5503439288842X
Thanks for looking into it!

Aha yep. I see the error -- it's one that we identified already, and the fix is going out with the next release of the checkout web-app.
This kind of issue is tricky to solve with versioning alone, since it relies on something like:

New version of checkout.js adds a new prop which can be passed down in paypal.Button.render(), like someCallback
Child window consumes window.xprops.someCallback()
Works great for new versions of checkout.js where someCallback has some default set, but gives a null-pointer for old versions

Since the checkout webapp itself can't be easily versioned, the best thing we can do about issues like that is:

Null checks for anything consumed in the webapp from checkout.js
Better testing

Normally we've been pretty good about the null checks and testing, but this one slipped through the cracks. Again, apologies about that.

So I guess this core issue is the "webapp itself can't be easily versioned" 😛
For now I see the safest thing for us to do is pull in directly from the CDN, however I am not a fan of this approach as it doesn't protect our application from any breaking public API changes in the CDN version that could potentially occur in the future.
Thanks for looking into the issue with 4.0.36 so quickly, however this was a critical issue for us so we updated ASAP.

Sounds good for the time being. For what it's worth, any breaking changes in the public api would definitely be considered a bug, and we have 600+ tests for all of the public apis we expose through checkout.js, so I'm hoping this should be a pretty safe option for you.


Sample 02.echobot can't run on app service

Version
botbuilder python sdk4.5
Describe the bug
For Sample   02.echobot
Successfully run in local and bot framework emulator
but can not run on azure after deploy to app service
(I connect to bot channel and use test in web chat)
log show:
2019-11-05T01:39:55.458414062Z
2019-11-05T01:39:55.458496562Z   _____
2019-11-05T01:39:55.458502062Z   /  _  \ __________ _________   ____
2019-11-05T01:39:55.458505962Z  /  /_\  \___   /  |  \_  __ \_/ __ \
2019-11-05T01:39:55.458510062Z /    |    \/    /|  |  /|  | \/\  ___/
2019-11-05T01:39:55.458513962Z \____|__  /_____ \____/ |__|    \___  >
2019-11-05T01:39:55.458518062Z         \/      \/                  \/
2019-11-05T01:39:55.458540762Z
2019-11-05T01:39:55.458544462Z A P P   S E R V I C E   O N   L I N U X
2019-11-05T01:39:55.458548063Z
2019-11-05T01:39:55.458552163Z Documentation: http://aka.ms/webapp-linux
2019-11-05T01:39:55.458555863Z
2019-11-05T01:39:56.815906832Z Starting OpenBSD Secure Shell server: sshd.
2019-11-05T01:39:56.879851978Z App Command Line not configured, will attempt auto-detect
2019-11-05T01:39:56.880581883Z Launching oryx with: -appPath /home/site/wwwroot -output /opt/startup/startup.sh -virtualEnvName antenv -defaultApp /opt/defaultsite -bindPort 8000
2019-11-05T01:39:57.471166003Z Oryx Version : 0.2.20191004.5, Commit: 95ca7f51b147da7b085922507f46106c664ae2a3
2019-11-05T01:39:57.471984108Z
2019-11-05T01:39:57.510537377Z Found build manifest file at '/home/site/wwwroot/oryx-manifest.toml'. Deserializing it...
2019-11-05T01:39:57.518767735Z Build Operation ID: |Mgf/GA1X9ZE=.6cd603b5_
2019-11-05T01:39:57.535159549Z No framework detected; using default app from /opt/defaultsite
2019-11-05T01:39:57.535621452Z Generating `gunicorn` command for 'application:app'
2019-11-05T01:40:00.086740549Z Writing output script to '/opt/startup/startup.sh'
2019-11-05T01:40:00.198100826Z Found virtual environment .tar.gz archive.
2019-11-05T01:40:00.198694830Z Removing existing virtual environment directory /antenv...
2019-11-05T01:40:00.288933559Z Extracting to directory /antenv...
2019-11-05T01:40:04.376490474Z Using packages from virtual environment antenv located at /antenv.
2019-11-05T01:40:04.376803676Z Updated PYTHONPATH to ':/antenv/lib/python3.7/site-packages'
2019-11-05T01:40:06.081427668Z [2019-11-05 01:40:06 +0000] [41] [INFO] Starting gunicorn 19.9.0
2019-11-05T01:40:06.100083798Z [2019-11-05 01:40:06 +0000] [41] [INFO] Listening at: http://<IP_ADDRESS>:8000 (41)
2019-11-05T01:40:06.101004804Z [2019-11-05 01:40:06 +0000] [41] [INFO] Using worker: sync
2019-11-05T01:40:06.111977581Z [2019-11-05 01:40:06 +0000] [44] [INFO] Booting worker with pid: 44
2019-11-05T01:40:07.644092669Z <IP_ADDRESS> - - [05/Nov/2019:01:40:07 +0000] "GET /robots933456.txt HTTP/1.1" 404 232 "-" "-"
2019-11-05 01:39:49.044 INFO  - Starting container for site
2019-11-05 01:39:49.074 INFO  - docker run -d -p 42573:8000 --name linebotdemo02_1_20b6493b -e WEBSITE_SITE_NAME=linebotdemo02 -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=linebotdemo02.azurewebsites.net -e WEBSITE_INSTANCE_ID=2f82fcda71047f2cee039aca2508dd3ba4f1da517325421091357c6b28aa93c6 -e HTTP_LOGGING_ENABLED=1 appsvc/python:3.7_1908260522
2019-11-05 01:39:55.658 INFO  - Initiating warmup request to container linebotdemo02_1_20b6493b for site linebotdemo02
2019-11-05 01:40:07.637 INFO  - Container linebotdemo02_1_20b6493b for site linebotdemo02 initialized successfully and is ready to serve requests.
2019-11-05T01:54:01.102712852Z <IP_ADDRESS> - - [05/Nov/2019:01:54:01 +0000] "POST /api/messages HTTP/1.1" 404 232 "-" "BF-DirectLine (Microsoft-BotFramework/3.2 +https://botframework.com/ua)"

Test in web chat issue message:
There was an error sending this message to your bot: HTTP status code NotFound

@sherlock666 The next Python SDK release will contain better instructions and ARM templates for deploying to Azure.  In fact, I just tested and merged an ARM template for deploying to an existing resource group.  We will provide another template to deploy to a new resource group shortly.
See Pull Request #415


Automatically generate jtag/jtag.sv

With the automation provided by JusTAG, we're very close to being able to do push-button updates to the register map (i.e., edit markdown files and regenerate the JTAG implementation.)  However, there is still one manual step, which wiring between interfaces in jtag/jtag.sv.  I think this could be automated because we know the destination interface for each register by the name of the markdown file it came from.  Adding this automation would eliminate a potential source of error while making it easier to add/change debug features of the chip.
I noticed that there is a least one bit of combo logic in jtag.sv, which is the reset gating of the analog core reset signal:
https://github.com/StanfordVLSI/dragonphy/blob/59089d659f54544086bb62c012f9d1c128826431/vlog/old_chip_src/jtag/jtag.sv#L15
Maybe we could handle this with an optional column in the Markdown file that specifies whether reset gating should be applied?
(I'm not sure if this automation belongs in JusTAG or DragonPHY, but thought I would add it as a task here to start.)

anybody got this issue ? it seems that Justag cannot work properly on my machine , I installed the latest JustTag v<IP_ADDRESS> verison
cd dragonphy2 -->
python make.py --view asic
--->
(myenv) master@icpc:~/work/mixed-signal/dragonphy2$ python make.py --view asic
######################################################
WARNING: The msdsl.circuit module is experimental!
######################################################
Failed to import libraries for results parsing.  Capabilities may be limited.
BUILD STATE DETECTED
BUILDING: jtag_config
BUILDING: jtag
b' M md/acore_intf.md\n M mflowgen.sh\n M setup.py\n M vlog/chip_src/jtag/jtag.sv\n?? 1.sh\n?? =1.0\n?? =1.0.0\n?? Genesis2/\n?? acore_intf.md\n?? install_pyvenv.sh\n?? make.py001back\n?? make.pybakc\n?? mflowgen/\n?? myenv/\n?? p37env/\n?? vlog/chip_src/jtag/raw_jtag_ifc_unq1.sv\n'
JTAG ID Code: 0x996cc141
['2574041409', '4', '/home/master/work/mixed-signal/dragonphy2/md/acore_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/cdr_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/dcore_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/error_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/hist_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/mdll_r1_top_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/prbs_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/sm_ffe_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/sram_multi_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/tx_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/txdata_intf.md', '/home/master/work/mixed-signal/dragonphy2/md/wme_intf.md', PosixPath('/home/master/work/mixed-signal/dragonphy2/vlog/pack/const_pack.sv')]
Traceback (most recent call last):
File "/home/master/work/mixed-signal/dragonphy2/myenv/bin/justag", line 8, in
sys.exit(main())
File "/home/master/work/mixed-signal/dragonphy2/myenv/lib/python3.8/site-packages/justag/JusTAG.py", line 93, in main
io_info = pandas.read_html(intf_html)[0]
File "/home/master/work/mixed-signal/dragonphy2/myenv/lib/python3.8/site-packages/pandas/io/html.py", line 1212, in read_html
return _parse(
File "/home/master/work/mixed-signal/dragonphy2/myenv/lib/python3.8/site-packages/pandas/io/html.py", line 1001, in _parse
raise retained
File "/home/master/work/mixed-signal/dragonphy2/myenv/lib/python3.8/site-packages/pandas/io/html.py", line 981, in _parse
tables = p.parse_tables()
File "/home/master/work/mixed-signal/dragonphy2/myenv/lib/python3.8/site-packages/pandas/io/html.py", line 257, in parse_tables
tables = self._parse_tables(self._build_doc(), self.match, self.attrs)
File "/home/master/work/mixed-signal/dragonphy2/myenv/lib/python3.8/site-packages/pandas/io/html.py", line 613, in _parse_tables
raise ValueError("No tables found")
ValueError: No tables found
Traceback (most recent call last):
File "make.py", line 158, in
main()
File "make.py", line 154, in main
graph.build()
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/graph/graph.py", line 399, in build
new_outputs[node_name] = (node.build(src_nodes, node.view), node.view)
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/graph/graph.py", line 333, in build
return vlog_gen(**config, filename=self.snk_path).generated_files
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/jtag.py", line 27, in init
self.justag(*justag_inputs, cwd=build_dir)
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/jtag.py", line 89, in justag
assert retcode==0, 'Error when calling justag.'
AssertionError: Error when calling justag.
and
=================================================================================
cd dragonphy2/build/mflowgen_dragonphy_top ---> make
.......
cd 5-info && if [[ -e mflowgen-check-postconditions.py ]]; then ./mflowgen-check-postconditions.py || exit 1; fi && cd ..
touch 5-info/.postconditions.stamp
mkdir -p 8-rtl/outputs && set -o pipefail && /home/master/work/mixed-signal/dragonphy2/mflowgen/mflowgen/scripts/mflowgen-letters -c -t rtl && chmod +x .mflowgen/8-rtl/mflowgen-run && cp -f .mflowgen/8-rtl/mflowgen-run 8-rtl && if [[ -e .mflowgen/8-rtl/mflowgen-debug ]]; then chmod +x .mflowgen/8-rtl/mflowgen-debug && cp -f .mflowgen/8-rtl/mflowgen-debug 8-rtl; fi && if [[ -e .mflowgen/8-rtl/mflowgen-check-preconditions.py ]]; then chmod +x .mflowgen/8-rtl/mflowgen-check-preconditions.py && cp -f .mflowgen/8-rtl/mflowgen-check-preconditions.py 8-rtl; fi && if [[ -e .mflowgen/8-rtl/mflowgen-check-postconditions.py ]]; then chmod +x .mflowgen/8-rtl/mflowgen-check-postconditions.py && cp -f .mflowgen/8-rtl/mflowgen-check-postconditions.py 8-rtl; fi && cd 8-rtl && if [[ -e mflowgen-check-preconditions.py ]]; then ./mflowgen-check-preconditions.py || exit 1; fi && ./mflowgen-run 2>&1 | tee mflowgen-run.log || exit 1 && cd .. && touch -c 8-rtl/outputs/*
    _     _

_ __  | |_  | |
| '__| | _| | |
| |    | |  | |
||     _| |_|
######################################################
WARNING: The msdsl.circuit module is experimental!
######################################################
Failed to import libraries for results parsing.  Capabilities may be limited.
Visiting cell_name=dragonphy_top
Found the following dependencies: ['const_pack', 'acore_debug_intf', 'tx_debug_intf', 'mdll_r1_debug_intf', 'input_buffer', 'analog_core', 'tx_top', 'digital_core', 'mdll_r1_top', 'mdll_inv']
Visiting cell_name=const_pack
Found the following dependencies: []
Visiting cell_name=acore_debug_intf
Found the following dependencies: ['const_pack']
Visiting cell_name=tx_debug_intf
Found the following dependencies: ['const_pack']
Visiting cell_name=mdll_r1_debug_intf
Found the following dependencies: ['mdll_pkg']
Visiting cell_name=mdll_pkg
Found the following dependencies: []
Visiting cell_name=tx_top
Found the following dependencies: ['const_pack', 'phase_interpolator', 'input_divider', 'hr_16t4_mux_top', 'qr_4t1_mux_top', 'div_b2', 'inv', 'output_buf_tx']
Visiting cell_name=hr_16t4_mux_top
Found the following dependencies: ['hr_4t1_mux_top']
Visiting cell_name=hr_4t1_mux_top
Found the following dependencies: ['hr_2t1_mux_top']
Visiting cell_name=hr_2t1_mux_top
Found the following dependencies: ['ff_c', 'mux']
Visiting cell_name=ff_c
Found the following dependencies: []
Visiting cell_name=mux
Found the following dependencies: []
Visiting cell_name=qr_4t1_mux_top
Found the following dependencies: ['ff_c', 'qr_mux_fixed', 'tx_inv']
Visiting cell_name=qr_mux_fixed
Found the following dependencies: []
Visiting cell_name=tx_inv
Found the following dependencies: ['INV_X4']
Visiting cell_name=div_b2
Found the following dependencies: ['ff_c_rn']
Visiting cell_name=ff_c_rn
Found the following dependencies: []
Visiting cell_name=inv
Found the following dependencies: []
Visiting cell_name=output_buf_tx
Found the following dependencies: ['tx_tri_buf', 'termination']
Visiting cell_name=tx_tri_buf
Found the following dependencies: ['TBUF_X4']
Visiting cell_name=digital_core
Found the following dependencies: ['const_pack', 'cdr_debug_intf', 'sram_debug_intf', 'dcore_debug_intf', 'dsp_debug_intf', 'prbs_debug_intf', 'wme_debug_intf', 'hist_debug_intf', 'error_tracker_debug_intf', 'tx_data_intf', 'channel_gpack', 'error_gpack', 'ffe_gpack', 'constant_gpack', 'ti_adc_reorder', 'avg_pulse_gen', 'adc_unfolding', 'mm_cdr', 'weight_manager', 'datapath_core', 'oneshot_multimemory', 'comb_comp', 'prbs_generator_syn', 'prbs_checker', 'error_tracker', 'histogram_data_gen', 'histogram_mux', 'histogram', 'output_buffer', 'jtag', 'tx_data_gen']
Visiting cell_name=cdr_debug_intf
Found the following dependencies: ['const_pack']
Visiting cell_name=sram_debug_intf
Found the following dependencies: ['const_pack']
Visiting cell_name=dcore_debug_intf
Found the following dependencies: ['const_pack', 'ffe_gpack', 'constant_gpack', 'cmp_gpack', 'channel_gpack']
Visiting cell_name=ffe_gpack
Found the following dependencies: []
Visiting cell_name=constant_gpack
Found the following dependencies: []
Visiting cell_name=cmp_gpack
Found the following dependencies: []
Visiting cell_name=channel_gpack
Found the following dependencies: []
Visiting cell_name=dsp_debug_intf
Found the following dependencies: ['const_pack', 'ffe_gpack', 'constant_gpack', 'cmp_gpack', 'channel_gpack']
Visiting cell_name=prbs_debug_intf
Found the following dependencies: ['const_pack']
Visiting cell_name=wme_debug_intf
Found the following dependencies: ['const_pack', 'ffe_gpack', 'channel_gpack']
Visiting cell_name=hist_debug_intf
Found the following dependencies: ['const_pack']
Visiting cell_name=error_tracker_debug_intf
Found the following dependencies: ['const_pack']
Visiting cell_name=tx_data_intf
Found the following dependencies: ['const_pack']
Visiting cell_name=error_gpack
Found the following dependencies: []
Visiting cell_name=ti_adc_reorder
Found the following dependencies: ['const_pack']
Visiting cell_name=avg_pulse_gen
Found the following dependencies: []
Visiting cell_name=adc_unfolding
Found the following dependencies: ['calib_sample_avg', 'calib_hist_measure']
Visiting cell_name=calib_sample_avg
Found the following dependencies: []
Visiting cell_name=calib_hist_measure
Found the following dependencies: []
Visiting cell_name=mm_cdr
Found the following dependencies: ['const_pack', 'mm_pd']
Visiting cell_name=mm_pd
Found the following dependencies: ['const_pack']
Visiting cell_name=weight_manager
Found the following dependencies: []
Visiting cell_name=datapath_core
Found the following dependencies: ['constant_gpack', 'ffe_gpack', 'error_gpack', 'cmp_gpack', 'channel_gpack', 'signed_buffer', 'buffer', 'signed_flatten_buffer_slice', 'comb_ffe', 'comb_comp', 'flatten_buffer', 'bit_aligner', 'flatten_buffer_slice', 'channel_filter', 'sliding_detector', 'detector_gpack']
Visiting cell_name=signed_buffer
Found the following dependencies: []
Visiting cell_name=buffer
Found the following dependencies: []
Visiting cell_name=signed_flatten_buffer_slice
Found the following dependencies: ['signed_flatten_buffer']
Visiting cell_name=signed_flatten_buffer
Found the following dependencies: []
Visiting cell_name=comb_ffe
Found the following dependencies: []
Visiting cell_name=comb_comp
Found the following dependencies: []
Visiting cell_name=flatten_buffer
Found the following dependencies: []
Visiting cell_name=bit_aligner
Found the following dependencies: []
Visiting cell_name=flatten_buffer_slice
Found the following dependencies: ['flatten_buffer']
Visiting cell_name=channel_filter
Found the following dependencies: []
Visiting cell_name=sliding_detector
Found the following dependencies: []
Visiting cell_name=detector_gpack
Found the following dependencies: []
Visiting cell_name=oneshot_multimemory
Found the following dependencies: ['const_pack', 'sram']
Visiting cell_name=sram
Found the following dependencies: ['sram_144_1024_freepdk45']
Visiting cell_name=prbs_generator_syn
Found the following dependencies: []
Visiting cell_name=prbs_checker
Found the following dependencies: ['prbs_checker_core']
Visiting cell_name=prbs_checker_core
Found the following dependencies: []
Visiting cell_name=error_tracker
Found the following dependencies: ['buffer', 'signed_buffer', 'flatten_buffer_slice', 'signed_flatten_buffer_slice', 'error_tracker_core']
Visiting cell_name=error_tracker_core
Found the following dependencies: ['sram']
Visiting cell_name=histogram_data_gen
Found the following dependencies: []
Visiting cell_name=histogram_mux
Found the following dependencies: []
Visiting cell_name=histogram
Found the following dependencies: ['histogram_core']
Visiting cell_name=histogram_core
Found the following dependencies: ['sram_small']
Visiting cell_name=sram_small
Found the following dependencies: ['sram_64_256_freepdk45']
Visiting cell_name=jtag
Found the following dependencies: ['raw_jtag_ifc_unq1', 'ffe_gpack', 'raw_jtag']
Visiting cell_name=raw_jtag_ifc_unq1
Found the following dependencies: []
Visiting cell_name=raw_jtag
Found no matches for cell_name=raw_jtag:
using view_order=['pack', 'chip_src']
using override={'sram': 'chip_src_freepdk45', 'sram_small': 'chip_src_freepdk45', 'tx_tri_buf': 'chip_src_freepdk45', 'qr_mux_fixed': 'chip_src_freepdk45', 'mdll_inv': 'chip_src_freepdk45', 'tx_inv': 'chip_src_freepdk45'}
Traceback (most recent call last):
File "gen_tcl.py", line 19, in
file_list += get_deps_asic(e['design_name'], process=e['adk_name'])
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/views.py", line 211, in get_deps_asic
deps += get_deps(
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/views.py", line 129, in get_deps
deps += get_deps(cell_name=submod, view_order=view_order, override=override,
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/views.py", line 129, in get_deps
deps += get_deps(cell_name=submod, view_order=view_order, override=override,
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/views.py", line 129, in get_deps
deps += get_deps(cell_name=submod, view_order=view_order, override=override,
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/views.py", line 102, in get_deps
impl_file = find_preferred_impl(
File "/home/master/work/mixed-signal/dragonphy2/dragonphy/views.py", line 51, in find_preferred_impl
raise Exception('Build failed due to a missing cell definition.')
Exception: Build failed due to a missing cell definition.
make: *** [Makefile:604: 8-rtl/.execstamp] Error 1


DX: do not use "empty" function

Inspired by @carusogabriel's talk at phpCE 2018.

Maybe one day we do it with phpstan/phpstan-strict-rules.


Clarifying the goals of the workshop and providing NEON-specific ideas

Hi everyone!
My name is Izzy and I am currently a first-season NEON field technician at Domain 03 in Gainesville, FL. In this post, I am (1) asking a question to clarify my understanding of our goals during the workshop, (2) providing some information regarding the NEON technician ground beetle ID process, and (3) adding a few ideas about how our workshop could speed up ground beetle identification for NEON technicians.
(1) Question: during this workshop, are we focusing specifically on how to quicken the NEON ground beetle identification process using AI/ML? Or are the workshop goals broader in scope, such as using ground beetle identification as a test-set to develop general methods of extracting information from images for the purpose of taxon identification? Both? I can imagine how these differently-focused goals might require different workshop strategies and end products.
(2) NEON technician summarized notes on the ground beetle ID process: As I don't yet have much experience IDing Carabids (AKA ground beetles, our beetle taxon of interest at NEON) in the lab, I asked some of my coworkers of varying skill levels about their experiences, in the interest of helping to shape our workshop goals. Here is a summary of what they said. I have included the full questions and answers at the bottom of this issue.

IDing the more common/easy-to-ID beetles takes approximately 5 min to 45 min
IDing the less common/harder-to-ID beetles takes approximately 45 min to 3 hrs
It is more difficult and takes longer to identify smaller beetles
Time taken to ID a beetle depends on the experience level of the NEON technician
It is sometimes required for the lead field technician and/or the taxonomist to re-ID beetles during the QC (quality check) process

(3) Some ideas/questions with the goal of quickening the NEON technician ground beetle identification process:

During the 'sorting' phase of the NEON lab technician's ground beetle processing duties, the technician separates the ground beetles (Family = Carabidae) from the rest of the insects/debris in the sample, prior to identifying the species. If it is not possible to used AI/ML to identify ground beetles down to the species level, it would still speed up the NEON technician process for an image-fed AI/ML algorithm to be able to quickly identify a beetle as belonging to Carabidae or not, particularly for tiny or uncommon beetles. Based on the comments from NEON technicians, I think it would even be useful for an AI/ML model to narrow down the identification such that technicians can save time by skipping the higher taxonomic keys and go straight to the genus level.

With the goal of NEON technician identification in mind, I don't feel too concerned about identification of extremely difficult-to-ID species that require sex-specific observation, dissection of beetle genitalia, or DNA-based identification. The technicians do not ID at this level of detail (from my understanding) and therefore in these cases, the expert taxonomist performing beetle ID QC would take the steps required to ID these special cases.

I do not have experience training AI/ML models. I am curious whether it would be possible/useful to create a model that combines different data types to perform the identification. For example, for the purpose of IDing NEON project beetles, would including the date (of sample collection), domain (e.g. Domain 03 in the Southeast), site (e.g. the Ordway-Swisher Biological Station), plot (e.g. plot 05), and trap (e.g. West trap) information along with the image data (and taxon/morphology data for the training set) improve the ability of the algorithm to identify beetles at the genus/species level? Would including this extra data obscure the biological meaning of the images, despite the relationship between the biology and the timing/location of observation? How about other NEON data collected simultaneously to the beetle samples, such as temperature, humidity, or wind speed at the collection site? Are there known associations between beetle diversity/abundance and other ecological data that could be used to inform our AI/ML model?

How can images best be taken to serve the AI/ML algorithm identification process (e.g. top, bottom, side view; how many images; how close-up of images)? How long should it take to image 1 beetle to still save time for the lab technician?

Here are the full comments from my NEON technician coworkers:
How long does it take you to ID the average beetle? Are there times when it takes a long time, and others shorter? Has it gotten a lot faster for you with more experience?

Asia Sawyer, third season NEON field technician: Usually, it can take anywhere from 15 min to 45 min to ID the more common beetles we get. The more difficult ones can sometimes take up to 2 hours depending on if they are smaller/harder to see specific parts, if I’m less familiar with that species, or if the key is very long. It can also take longer if you are less familiar with beetle anatomy because you then have to take extra time to look up what body part a key might be referring to. With more experience, it definitely goes faster, especially if you already know what family a beetle belongs to because you can skip the higher taxonomic keys and immediately go to the genus/species.
César Ortiz, second season NEON field technician: I would average time for an ID is around 15-30 minutes. It can be around 5 minutes for certain species, mostly the Pasimachus genus, since they are the most common and numerous we get. If beetles are very small or is a more uncommon species it takes longer, upwards of 45 minutes or an hour for one beetle. It definitely gets faster with experience.
Jordan Ehmke, NEON field ecology lead: it depends on the genus, for our more common genus it takes me about 5 minutes but for the tougher things it can take up to an hour. It definitely does get easier with practice and learning the keys, but can be variable based upon experience of the individual.
Keirsten Kaufmann, first season NEON field technician: It might take me 5 minutes to identify Pasimachus, they are nice and easy! It has gotten faster with experience and having someone knowledgeable in the beetle lab with me. More photo references in the keys would be helpful. On another note, a tiny carabid key would be helpful since some species have a small size, maybe there could be a specific guide for those tiny guys. I haven’t really attempted a difficult one but honestly it might take me a few hours whereas a lead like Jordan could identify it in less than 30 minutes.

Do you think the smaller beetles ever get missed because they’re so tiny, or do you think we generally catch all the carabids regardless of size?

Asia Sawyer, 3rd season NEON technician: If you’re referring to ID, then I think we generally catch all of the regardless of size since you can usually still see the enlarged trochanters on even the smaller beetles. However, I’m sure some get missed from general human error during the sorting process.
Keirsten Kaufmann, first season NEON field technician: The tiny ones can be hard to tell if it’s a carabid, I also imagine some are mistakenly put as bycatch or just thrown out entirely by mistake which could be avoided if the person looks through their waste a second or third time.

[question for field leads] Do you have to spend much time re-IDing beetles that were ID’d by less experienced NEON technicians?

Jordan Ehmke, NEON field ecology lead: I do have to re-ID some beetles if there is time to do so, but if not then the taxonomist is the ultimate last say for QC-ing the ID.

Wow, Izzy! Thanks for this insightful information. For question 1, we are hoping participants can lead a bit on the exact direction of the workshop. However, here is the motivation/scope/desired outcomes we posted to the application as a reminder to show what the organizing committee has been thinking of for the event:
Motivation
The National Ecological Observatory Network (NEON) collects an unprecedented multitude of ecological and environmental data on a continental scale through field sampling and remote sensing. As part of the field sampling, NEON collects, counts, and identifies biological specimens of environmental indicator species and those filling ecologically important roles. The underlying processing of specimens is often manual and time consuming and is limited to taxonomic identification and counts. This presents a unique opportunity with potentially long-lasting impact to explore the potential and limitations of AI/ML-driven automation for biodiversity data collection efforts, including by developing and utilizing multi-modal ML computer vision models that take advantage of imagery, remote sensing, and environmental data.
One taxonomic group especially ripe for proof-of-concept is beetles, one of the world's most diverse groups that serve important roles in pollination of plants and as indicator species providing early warning signals of environmental change. NEON collects ground beetles (Carabidae) from across the continental United States, Puerto Rico, and Alaska using pitfall traps. These specimens are then sorted and identified by NEON staff and other taxonomic experts in a painstaking, manual process that can take over a year and concludes with publishing counts of beetle species on NEON’s data portal. What if, rather than publishing counts of species, NEON captured and published images of beetles? Can we develop an automated process to more efficiently derive species counts from the imagery? Is it possible to use imagery to measure important characteristics (known as functional traits, such as body size) of the different beetles that are collected? This workshop offers an opportunity to develop a proof-of-concept to demonstrate how the application of computer vision techniques could transform how ground beetle community data are collected, and thus biodiversity data more generally. Moreover, the treasure trove of data products collected at NEON sites provides the opportunity to push multi-modal model development applied to computer vision in tackling this challenge.
Desired Outcomes
We aim to facilitate outcomes that address the potential of and need for ML-ready biological image datasets to extract information about biodiversity, including (but not limited to!) the following:
A prototype workflow for extracting trait and species identifications from images of NEON beetle specimens. Our goal is for this workflow to be reproducible, follow FAIR guiding principles, and understandable by biologists and computer scientists.
Publication of open data products containing species identifications and functional traits derived from beetle specimen images.
A peer-reviewed publication describing best practices for AI/ML ready biological specimen data and images. This information will be accompanied by a white paper to be presented to NEON with advice for how to move forward with efforts to make the Observatory’s data more AI/ML ready.
Scope
We are keeping the scope of possible projects focused on the extraction of species identifications and trait measurements from beetles to maximize the limited time we have in the workshop. That notwithstanding, we expect the event to connect people with domain science-focused goals, such as biologists interested in datasets that help answer biological questions, to people with ML-focused goals, such as ML researchers interested in domain science questions for which to develop algorithms and models.
We generally expect datasets curated at or for the event, as well as tools or methods developed, to satisfy FAIR principles, and where applicable also CARE principles.

@sydnerecord This is great, thank you for posting this information! Very helpful.

@iaviney, This is great background to understand the classification work that takes place at NEON!
Take a look at the What It Takes to Identify a Beetle bootcamp.
It would be great to see how your workflow feeds into the whole sampling/identifying/getting data scheme and learn about the typical tools you use for beetle ID and how those can be improved.

@JCGiron I will reply to your comment more thoroughly once I get to the hotel tonight, but here is a brief response. Once we have collected a sample from the field, we perform an 'Ethanol rinse' as soon as possible, usually on the same day. The Ethanol rinse involves pouring the sample (containing organisms and debris caught in propylene glycol) over a filter, rinsing the sample with DI water to remove propylene glycol, and then storing the sample in 95% Ethanol. At this point, any 'vertebrate bycatch' is removed, which could be small mammals, frogs, etc that got stuck in the trap along with the invertebrates. Then, when there are enough lab technicians available to do so, the invertebrates from the samples are 'sorted' to separate the ground beetles (Carabidae) from the rest of the invertebrates. From my understanding, the presence of enlarged trochanters is generally the feature used to identify a beetle as a Carabid, but I can ask my coworkers for more info on that. After that, again when enough lab technicians are available, the ground beetles are identified using dichotomous keys. I have asked our beetle lead (Jordan) if I can share some of the ID resources we use at D03 with you all.
I am thinking that we could use the ML tools to intervene at a couple points, potentially. Perhaps the AI/ML model could be useful during the invertebrate sorting process, to verify whether a beetle is a Carabid or not. I'm thinking this could be particularly useful for the tiny beetles, as those are harder to identify as Carabids given the smaller features. Additionally, if the model is accurate enough, perhaps it could also be used for the beetle identification point, so that technicians don't have to spend as much time going through the dichotomous keys. Ideally, the model could identify a beetle species from the image, but if not, it would also be useful if the model could narrow down the specimen to a taxonomic group above the species level to lessen the number of keys the technician has to go through.
That is very interesting to hear about the potential site-specific models/tools. I'm interested to hear more and talk more about this! Thank you for the info!

@sydnerecord Oh great! How would you like to use the keys? Are you thinking that those keys could somehow be incorporated into the computational process?

@iaviney I am hoping we can use them to ID some of the more common species (if the morphological traits can be seen dorsally). They could serve as the basis for an ID algorithm

@sydnerecord Cool! Would this algorithm also be AI/ML or more of some sort of branching decision maker based on if-then type statements? I'm sorry if this message isn't too clear, I'm not as familiar with some coding/algorithm terminology.


Added 64-bit HollowsHunter.

The 64-bit hollows_hunter was not specified, therefore it was added.

Thank you.
Looks like our CI has some issues at the moment:

the "get changed file" step failed so this has not been tested in CI properly yet
also the nuspec version needs to be updated which should have been caught by the linter (issue #212)

@mr-tz get-changed-files didn't fail. It relays on the linter failing if the nuspec is not updated (only looks for modified .nuspec). Thanks for fixing the linter issue! 😄

Ah, right, I see the filter on the action now :)
Thanks for the update here.


storage: High spontaneous latencies

We're currently facing issues with high latencies on GCS operations for 10-30 minutes a time, multiple times a day, causing timeouts on our APIs. Most issues surprisingly seem to happen with delete operations, even though they should be the easiest to process.

99th percentile

95th percentile

90th percentile

50th percentile (for comparison)

Our upload/delete code is similar to this:
func Upload(ctx context.Context, path string, src []byte) error {
	wc := s.bucket.Object(path).NewWriter(ctx)
	wc.ContentType = contentType
	wc.CacheControl = "max-age=31556926" // 1 year

	wc.Write(src)
	return wc.Close()
}

func Delete(ctx context.Context, path string) error {
	return s.bucket.Object(path).Delete(ctx)
}

Are you aware of anyone else having these troubles? Or do you have a solution for this?
I personally suspect these issues to stem from my request for #701, which - due to retrying on REFUSED_STREAM, causes those high latencies (tenfold even!). Could that be the case? And if so: Could it be solved somehow?

No one else has mentioned this problem (yet).
Could you instrument shouldRetry to see if your theory is correct?

@jba I just deployed our service with an instrumented shouldRetry function. I‘ll keep you posted. 🙂

@jba As already suspected it indeed is caused by spontaneous excessive amounts of REFUSED_STREAM errors. In our case 40% of uploads and 10% of all deletes fail that way, or 25 and 5 requests per second respectively.
The retrying that was added sadly does not solve the problem - it only prolongs it until the deadline runs out. For instance the above deletes run with a deadline of 25s, which is still seemingly easily exceeded.
Our applications are furthermore even running within the Google Container Engine by the way.
Do you have ideas as to why that is the case, or how it could be solved?
Could it be an infrastructure problem on your side? Something like your GCS load-balancer/front-proxy being overloaded?
Either way this is a very uncomfortable problem since sometimes 40% of all our requests fail to complete within an arguably generous deadline of 5s. 😕

Bringing it up internally. Will keep you posted.

We are seeing similar problems with uploads, high rate of failure that can spike up to 30% (but probably hovers around 1-5%).  Our code looks something like:
our timeout is long, 2min, but still see failures.
`
ctx, _ := context.WithTimeout(context.Background(), time.Duration(config.GcsTimedContextSec)*time.Second)
writer := object.NewWriter(ctx)
writer.ChunkSize = 0

for i := 0; i < config.GcsWriteRetries; i++ {
if _, err := writer.Write(body); err != nil {
log.WithFields(
logrus.Fields{
"reason":     err.Error(),
"uri":        uri,
"bodySize":   len(body),
"retries":    config.GcsWriteRetries,
"bucketName": config.HlsBucket,
"attempt":    i,
},
).Error(errGcsWrite)
continue
} else {
log.WithFields(
logrus.Fields{
"reason":     "wrote to gcs",
"uri":        uri,
"bodySize":   len(body),
"retries":    config.GcsWriteRetries,
"bucketName": config.HlsBucket,
"attempt":    i,
},
).Info(errGcsWrite)
}
return
}`

Is there any update or progress on this?  we are still seeing a very high level of uploads fail with REFUSED_STREAM, and very high latencies as suggested above.

@chasmastin We "solved" the issue in the interim, by keeping a running average of failed GCS requests. If that value is too high we forcefully instantiate a new connection pool.
You can do so by providing the GCS library with your own http.Transport using the option.WithHTTPClient() option each time, causing a new HTTP pool to be created and used.
This did not solve the underlying issue though of course.

We have seen somewhat similar things with the bigtable library. This uses GRPC.  We have been struggling with occasional latency for ~30 minutes where our upper percentiles timeout.  We often see this across hosts client, but have occasionally seen it against one particular host.
We have been heavily instrumenting the connection logic and the request logic. It is very hard to pinpoint what is actually going on. For bigtable, our fear was either large rows or hot keys but this does not appear to be the case. Our current main suspect is the GRPC connection. I wonder if we are seeing similar issues manifested in different API's.

We are seeing the same behavior in BigTable, in the graph below you can see the periods of high latency. We have a 1 minute timeout set, so it never exceeds 60s, but these periods of high latency are occurring fairly often. Not that all of the requests plotted here are BigTable reads, not writes.

Interesting fact: storage uses JSON/HTTP, while bigtable uses gRPC. So if these are related, they are at a deeper level—the Go networking stack, perhaps, or something in Google's network infrastructure.
It would be useful to know if these same issues crop up in other languages' clients.

Off-the-cuff observation: if the BigTable client maintains long-lived connections, then those spikes are probably the GFE killing connections every hour (WAI).

@jba At my company we solved the initial issue above by recreating the GCS client and subsequently its http.Transport whenever the error rate is >10% in the last 100 requests.
This simple heuristic triggers around 1-2 times per hour per instance in average, but solved the perceived issue entirely in the meantime.
But in fact we indeed have more issues like that with other services for which we didn't find any easy solution yet. For instance sometimes PubSub clients of our low-traffic services simply stop receiving any messages at all for up to multiple minutes at a time - far more than the client-side timeout of 60s.
(That's probably not related and I hope I'm not mistaken, but... Every single day at some time each database in our Spanner instances increase their respective CPU usage. Which is kinda understandable, since they're probably running some kind of DB compaction etc. But what's interesting is that each single day at some consistently random other time Spanner just "decides" to have a 3-10x (not a typo) increase in latency for a few minutes without any increase in OPS or CPU usage. Strange stuff. 😄)

At a high level, I feel like these issues are best discussed with the product teams via their support channels.  Latency is pretty hard to debug, and I don't think that the problems are specific to GoLang.  I think that we should have some good documentation about who to talk to about performance issues in a centralized place.  I do think that we ought to close this issue, since there isn't anything actionable from the client team.
@jadekler, we did some research on the Cloud Bigtable side.  There's an interesting client-side issue with a subtle interaction between connection pooling, low QPS clients, connection refreshes and retries-with-exponential-backoff.  That issue ought to be discussed off line, and in the context of performance testing and fixes.

I suspect this issue is another manifestation of https://github.com/golang/go/issues/27208, which we discovered in https://github.com/googleapis/google-cloud-go/issues/1211. In short, it's an http/2 bug, which has since been fixed and will arrive in Go 1.11.3.
I'm going to close this. If the problem appears again with Go 1.11.3, feel free to re-open.

FYI: The (supposed) fix for this issue was recently released in Go 1.11.4 instead. 🙂


[consumer/kafka] Disable Kafka consumer timeout

The Kafka consumer timeout is actually not needed, just adds complexity. So simplify by disabling the timeout and removing the code that handles the timeout.

Codecov Report

Merging #241 into master will increase coverage by 0.02%.
The diff coverage is 33.33%.

@@             Coverage Diff             @@
##             master    #241      +/-   ##
===========================================
+ Coverage     47.38%   47.4%   +0.02%
  Complexity     2533    2533
===========================================
  Files           608     608
  Lines         15981   15972       -9
  Branches       1196    1194       -2
===========================================
- Hits           7573    7572       -1
+ Misses         7955    7947       -8
  Partials        453     453

Impacted Files
Coverage Δ
Complexity Δ

...ify/heroic/consumer/kafka/KafkaConsumerModule.java
76.19% <ø> (+0.51%)
8 <0> (ø)
:x:

.../spotify/heroic/consumer/kafka/ConsumerThread.java
64% <33.33%> (+3.09%)
20 <2> (ø)
:x:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3568906...e6eaa9c. Read the comment docs.


CMake variable resolution, some heuristics on product naming

This contribution was made possible during a 'LogMeIn Inc.' event.
Description of Change

Resolve $SOME_CMAKE_VARIABLE stuff, recursively
heuristically apply ASDLIB -> LIBASD, while keeping original too in 'product' field(s)
some fixes, extensions to the regexes that handle version information
fixed infinite loop issue in #2257

Have test cases been added to cover the new functionality?
no.
Tested manually with libpng

I apologize about how long this has taken me to get back to. However, it is still not passing the basic test cases.  This is because the call to setVersion is passing in data that is not even remotely a version number.  Maybe we should be be filtering this via parseVersion(version, true).

Thank you for getting back at this, I will look at it and try that way tomorrow!

This PR was merged manually: https://github.com/jeremylong/DependencyCheck/commit/4ca7aa22ebbd11af42f01fbede5ee9c1378cffbc


Think about easier interface for KMF

Current interface

Might be possible to make especially the selection of T and E easier.

Fixed with #822


Implement stacks security and isolation model

Part of #609
What seems to be the problem?
Currently stacks are installed in a uniform way regardless of their intended persona or scenario.  This means that every stack has the same scope of permissions (cluster wide), which may not always be appropriate.
What could Crossplane do to help?
The Stack Manager should be updated to allow differing scopes of security and isolation for each stack being installed.  This means that some stacks would be cluster scoped and some stacks would be namespace scoped
The design for this will be fleshed out fully as part of #580.  This issue is tracking the implementation of that design.
What does it look like when we're done?

The security model described in #580 is implemented by the Stack Manager
Users can install cluster scoped stacks
Users can install namespace scoped stacks
Access to the installation of stacks can be constrained by the admin using RBAC
Documentation has been updated to explain the usage and examples for stacks of all scopes

Todo

[x] Update stack request CRDs to differentiate between types of Stacks
[x] Update the Stack Manager to create RBAC artifacts at the appropriate stack for the scope
[x] End to end testing to verify the installation and artifacts of all types of stacks
[x] Add documentation and examples that show how to use different types of stacks

Closed by #729 :tada:


added props to client config

Checklist

[x] Use a separate branch in your local repo (not master).
[x] Test coverage is 100% (or you have a story for why it's ok).
see issue #172

Coverage remained the same at 100.0% when pulling 55dd0c8b5ead579edc6fd97552426cdc492a34c9 on jstacoder:pass_props_through_client into 8be11a083307d99c7b13186aaf3bb227ec69b5e0 on google:master.


Crawlera 6592 rebranding

Continuation of https://github.com/zytedata/zyte-headless-proxy/pull/1

We have to continue using X-Crawlera-* headers instead of Zyte-Proxy-* at the moment.

what's state is this at @Gallaecio?. Do you plan to finish it or should it be picked by someone else in the Crawlera team?. We're now working heavily in the doc. Thanks!
cc @jpuigm

I plan to resume this shortly, this month.

Can we wrap up this work guys? @zloidemon @Gallaecio

Still 2nd on my To-Do list.


Openshift Admin Client

Openshift has a REST API: https://docs.openshift.org/latest/rest_api/openshift_v1.html#rest-api-openshift-v1
After looking at the Obsidian boosters, They use a maven plugin from he fabric8 team, that helps to create/build/deploy a project(booster) on Openshift.

https://github.com/fabric8io/fabric8-maven-plugin

https://github.com/fabric8io/kubernetes-client

I wonder if we should have a similar node equivalent

Currently in progress creating a POC here: https://github.com/bucharest-gold/openshift-client

Created another repo, https://github.com/bucharest-gold/nodeshift, that will do the actual "plugin" stuff.


DDS kind for switching tabs

yazi --debug output
N/A
Please describe the problem you're trying to solve
I had an issue opened for starship.yazi that pointed out that it wasn't updating when switching between tabs. When I was looking into it, I couldn't find a way to run the plugin when the tabs change.
Would you be willing to contribute this feature?

[X] Yes, I'll give it a shot

Describe the solution you'd like
If I'm not missing something, would it be possible to add a DDS kind when switching tabs so plugins can subscribe to that event?
Additional context
I can give it a shot if you point me in the right direction (haven't touched the code in a while).
Validations

[X] I have searched the existing issues
[X] The latest nightly build of Yazi doesn't already have this feature

@Rolv-Apneseth I opened a PR that could help implement this. It currently sends the index of the tab that was switched to as an event.
Do you think this could be enough for implementing this feature in your plugin? I'm not sure if the current directory is available from yazi's APIs as long as we know the current tab.

Oh nice, thanks. I believe the cwd is based on the current tab anyway but yeah seems to work perfectly for me, just subbed to the tab-switch event and it updates no problem.


fix: dfget couldn't download s3 directory correctly

Description
Something exception raised when using dfget to download a s3 directory.
Below is my usage:
dfget --recursive  --url "s3://my-bucket/my-dir" \
      --header "awsEndpoint: xxx" \
      --header ... \
      --output local_dir

I find the bug is that:
When download a s3 directory, client will call a s3 API ListObjectsV2, to check whether the url is a directory. And the API need a parameter prefix (used as prefix filter for s3 object ids, should not start with leading slash "/"). In dragonfly's s3 client code, the url will be parsed as request.URL and request.URL.path will be used as prefix(after add a trailing slash). Such path always have leading slash "/", make ListObjectsV2 API response wrong result.
Related Issue

Motivation and Context

Screenshots (if appropriate)
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation Update (if none of the other choices apply)

Checklist

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.

@piamo Please try to submit issues in English and fix lint.

@piamo Please try to submit the pull request in English and fix lint.

lint这个错误是啥原因啊，感觉没改几行code啊-_-||


Flush() in BulkIndexor is funky

This issue is really for @araddon
This works, but has some idiosyncrasies. For instance, in the bulk_test.go file for func TestBulkSmallBatch, if you do not "sleep" and only Flush(), the test fails. So it seems that in order to properly Flush, you have to wait for some unspecified duration of time so that the channel can receive and format the messages - at least I think this is what is happening.
I would not mind if the Flush method actually handled this wait time, but I don't really have any clue as to how to calculate a reasonable wait time.
Does this make sense?

Although fixed at some point, this problem seems to have resurfaced. In my tests, I want to be able to:

bulk index some data
flush the bulk indexer
refresh the index
search

I don't expect to be able to do this in production, but it is a reasonable expectation for testing.
I may well be doing something wrong, but it seems that without a (rather long) delay of between flushing and refreshing, this doesn't work.
Here is the code from the old version:
// Flush all current documents to ElasticSearch
func (b *BulkIndexer) Flush() {
	b.mu.Lock()
	if b.docCt > 0 {
		b.send(b.buf)
	}
	b.mu.Unlock()
	for {
		select {
		case <-wgChan(b.sendWg):
			// done
			return
		case <-time.After(time.Second * time.Duration(MAX_SHUTDOWN_SECS)):
			// timeout!
			return
		}
	}
}

and the current version:
// Flush all current documents to ElasticSearch
func (b *BulkIndexer) Flush() {
	b.mu.Lock()
	if b.docCt > 0 {
		b.send(b.buf)
	}
	b.mu.Unlock()
}

Just a note that this problem also hinders checking for errors at the end of a bulk index operation. Ideally, a developer would like to use a strategy like this:

start a bulk operation
push a bunch of data to the bulk operation
flush the bulk operation
check for errors
if errors, try again (the bulk operations would have to be idempotent)

However, flushing currently only ensures that the data has been passed to the write thread (if I understand correctly), not that it's been written and appropriate errors have been processed, so you don't know if you've gotten the errors yet.
At the very least, the flush operation should be fixed (ideally with some way to collect errors). I can provide a patch and merge request, but I want to make sure I understand the problem correctly. I would first try to fix the flush method in a way that is similar to the old code, and then attempt to provide an additional flush method that returns accumulated errors. Shall I proceed?


fix: Sluggish animation when selecting a different type

Details

Fixed Issues

$ https://github.com/Expensify/App/issues/49296
PROPOSAL: https://github.com/Expensify/App/issues/49296#issuecomment-2355042362
Tests

Mobile:

Launch New Expensify app.
Go to Search.
Tap on the dropdown.
Tap Chats.
Open the dropdown.
Select Invoices.
Verify the animation should be smooth when selecting a different type.

Desktop:

Launch New Expensify app.
Go to Search.
Tap on any canned type on left side.
Verify it works properly.

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/user-attachments/assets/f3f64dff-bb63-4b40-936b-5c35acc0ce14

Android: mWeb Chrome

https://github.com/user-attachments/assets/505a1d51-6958-418d-bfc2-c0d4816ff33b

iOS: Native

https://github.com/user-attachments/assets/4de4b554-f982-4001-a8da-31961234dc69

iOS: mWeb Safari

https://github.com/user-attachments/assets/9f36ac77-bfdb-4186-85d0-0fb0b7e09c3b

MacOS: Chrome / Safari

https://github.com/user-attachments/assets/ee7cbbb8-b03d-4326-8e07-6c5a22a59549

MacOS: Desktop

https://github.com/user-attachments/assets/2c69b457-d004-4fb9-bf9a-547b3763b71f

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

The reported performance regression can be ignored, see:

https://github.com/Expensify/App/issues/50141


Update to acme v02

Solves #7

Signed CLA

Maybe all we need to do is to bump the image in the helm chart?

You will need to bump both the image version as well as the manifests used.
There were a number of breaking changes between 0.2 and 0.3: https://cert-manager.readthedocs.io/en/latest/admin/upgrading/upgrading-0.2-0.3.html
In general, this repo should, on each minor release:

Ensure it is advising users to install the newest version of the Helm chart (e.g. helm install --version v0.3.0)

Ensure the Issuer and Certificate manifests recommended are up to date (e.g. update v01 to v02 in this case)

For this particular release, I think just those two points will be sufficient. I'll make sure there is a doc like the one linked above between each minor release, so at least it is possible to see what's breaking.

Thanks @munnerz. I'll merge this and update the chart version!


Support &, | for selectors with lambda's

Should work!
px.histogram(ar6.loc[isin(variable="Price|Carbon", region="World") & (lambda df: df[2050] < 200), 2050])

Fixed by #59


Incorrect debug assertion message for unaligned buffers

The debug assertion in check_alignment prints that it found alignment 1 << actual_align, but it should print that it found alignment 1 << actual_align.trailing_zeros().

Fixed by 6032bd77a845f4f46720e7e12a2374fd2588c099


Scheme eval is listed as a solver specific API

It's general but here it seems to be specific to the solver.

Right, it should be under General category.


Add support for Azure Regions

Preflight Checklist

[x] I have read the Contributing Guidelines for this project.
[x] I have searched the issue tracker for an issue that matches the one I want to file, without success.

Issue Details

Operating System:

Ubuntu 18.04, Node 14

Browser:

N/A, using CLI

Using Azure by setting the connection details in the .env file.

Expected Behavior
Expected to receive a report based on the Azure usage.
Actual Behavior
Error message.
/home/{redacted}/cloud-carbon-footprint/packages/common/src/Errors.ts:13
    super(message)
    ^
EstimationRequestValidationError: Not a valid region for this account
    at new EstimationRequestValidationError (/home/{redacted}/cloud-carbon-footprint/packages/common/src/Errors.ts:13:5)
    at validate (/home/{redacted}/cloud-carbon-footprint/packages/app/src/CreateValidRequest.ts:66:11)
    at Object.CreateValidFootprintRequest (/home/{redacted}/cloud-carbon-footprint/packages/app/src/CreateValidRequest.ts:134:3)
    at Object.<anonymous> (/home/{redacted}/cloud-carbon-footprint/packages/cli/src/cli.ts:54:29)
    at step (/home/{redacted}/cloud-carbon-footprint/packages/cli/src/cli.ts:52:23)
    at Object.next (/home/{redacted}/cloud-carbon-footprint/packages/cli/src/cli.ts:33:53)
    at /home/{redacted}/cloud-carbon-footprint/packages/cli/src/cli.ts:27:71
    at new Promise (<anonymous>)
    at __awaiter (/home/{redacted}/cloud-carbon-footprint/packages/cli/src/cli.ts:23:12)
    at Object.cli [as default] (/home/{redacted}/cloud-carbon-footprint/packages/cli/src/cli.ts:89:12)

To Reproduce

Configure for Azure
run yarn start-cli --startDate 2021-08-03 --region "AustraliaEast" --groupBy service --format csv --endDate 2021-08-04

Additional Information
The following code in CreateValidRequest.ts shows that the system only references AWS regions.
  if (region && !contains(region, values(configLoader().AWS.CURRENT_REGIONS))) {
    errors.push('Not a valid region for this account')
  }

Happy to contribute, but it may take me a while to grok the code in order to do it generically.  If already has pointers on how to how to access the config object through a common interface, please let me know.

Hey @nagytech! Good find - that's a little bit of tech debt we have from before we added in support for Azure and GCP. It's not one of our top priorities at the moment, so if you're willing to work on it at your leisure, that'd be awesome! For this, it'd be best however to use region constants from the various packages rather than the config object. Let us know if you plan on picking it up or have other questions!

Hi @nagytech @ericksod
I felt into the same issue. Is there any update on this ?
Seems like the west-europe or francecentral region fall into this issue, because it seems related to Azure Region

Pretty sure that I just hardcoded some values to get around this, but haven't touched the project in a long time.  Sorry I can't help further.

No problem @nagytech
This issue might be at the same time of #592

Hi @LvffY !
Yes, it is the same issue as #592 , which has been added to the project and planning board with more specific requirements. I'll close this one to avoid confusion.


sdl2 outdated. New version at 2.0.12

@alaughlin Sdl2 is outdated.

This package is now 7 days out of date. Its an important package for me (kiss-games). I want to take maintainership. I create a pullrequest if so.

It's yours @sdsddsd1


test_packageExtractWithUnicodePath fails on Windows

https://github.com/Cobertos/unitypackage_extractor/blob/b9019c795ab56d5aa6a984f5a76617eb92a6bd07/tests/test_testPackage.py#L62
Need to pass encoding="utf-8" to open(), since the file is UTF-8, but the default encoding on Windows isn't UTF-8 but region dependent, e.g. Windows-1252 or Windows-932 (on Japanese systems). Right now it'll read it with the wrong encoding because open() will default to the system/environment encoding setting, so the test fails.

Commit made to fix in #15

PRs will now run in Windows as well