NIFI-6851: Added FlowFileAcquisitionMethod to fetch/receive methods i…

…n ProvenanceReporter
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Adds two new methods to ProvenanceReporter enabling a way to capture how FlowFiles were fetch()ed or receive()ed in provenance events. Also updated provenance event persistence to include the new FlowFileAcquisitionMethod. See NIFI-6851 for additional rationale behind this change.
I have not updated any anything to use these new methods other than unit tests. This PR is just to contribute the feature. Processors can be updated either in this PR based on feedback or separately in another JIRA.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[ ] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[x] Have you verified that the full build is successful on both JDK 8 and JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

I've made all recommended changes or attempted to justify. If you'd like any more changes, let me know!

Thanks @m-hogue. I think there's something that was still missed in regards to the serialization/deserialization of the enum though. As it is, it stores a mapping of FlowFileAcquisitionMethod to a number. This number is no longer the ordinal. And when the event file is read, it reads in this mapping. However, it never does anything with the mapping.
Rather, it uses the mapping provided by FlowFileAcquisitionMethod. If the enumeration changes between version 1.11.0 and 1.12.0, for example, then when 1.12.0 reads the data, it will lookup value 1 according to definition in 1.12.0, which is similar to how it would work with the ordinal. Instead, if the FlowFileAcquisitionMethod is stored as a value of 1, it must lookup what the value of 1 meant in the mapping that was serialized, NOT the mapping that is available in FlowFileAcquisitionMethod. Otherwise, the mapping doesn't help, and version 1.12.0 would still parse the record incorrectly, as a result of the enumeration changing.
Does this make sense? If not, please let me know.

As it is, it stores a mapping of FlowFileAcquisitionMethod to a number

@markap14 : I see what you're saying. This behavior was actually copied from the existing implementation for event types, which i believe has this same problem:
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/EventIdFirstSchemaRecordWriter.java#L89
This means that the event type map and the acquisition type map both need to be serialized in the event header and read at deserialization time to know how it was serialized. Correct? If so, i can work on fixing that.

@m-hogue  I think the way that the data is serialized is correct for both event types and acquisition types. The problem that I see is in the deserialization. In the EventIdFirstSchemaRecordReader, we read the list of Event Types as they were when the data was serialized: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/EventIdFirstSchemaRecordReader.java#L119
Then, when we deserialize the data, we provide that list of Event Types to the LookupTableEventRecord (https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/EventIdFirstSchemaRecordReader.java#L143).  This is the important piece that I think this PR is missing.
LookupTableEventRecord determines the ProvenanceEventType like this, roughly (here, I've removed error handling, renamed variables for clarity, etc.):
// Get the integer value that was stored in the file.
final Integer eventTypeOrdinal = (Integer) record.getFieldValue(EventFieldNames.EVENT_TYPE);
// Get the name of the event. This is based on the List of Event Names that were stored in the file header.
String eventTypeName = eventTypes.get(eventTypeOrdinal);
// Get the ProvenanceEventType with that name.
ProvenanceEventType eventType = ProvenanceEventType.valueOf(eventTypeName);

However, what I believe this PR is doing is more like this:
// Get the integer value that was stored in the file.
final Integer acquisitionType = (Integer) record.getFieldValue(EventFieldNames.FLOWFILE_ACQUISITION_METHOD);
// Get the name of the acquisition type NOT from the List of Acquisition Types that were stored in the headers, but rather from the Enumeration (i.e., the current mapping of integer to name, rather than the mapping that makes sense for the data in the file) - this, I believe, is the problem.
String acquisitionName = FlowFileAcquisitionMethod.flowFileAcquisitionMethodNames().get(flowFileAcquisitionOrdinal);
// Get the acquisition method based on the name
flowFileAcquisitionMethod = FlowFileAcquisitionMethod.valueOf(acquisitionName);

@markap14 : You're totally right. I was misinterpreting how deserialization worked. I traced it through and made the change.  Thanks for taking the time to review!

@markap14 : I felt like it'd be useful to visualize the FlowFileAcquisition method in the provenance events in the UI, so i've update the PR to add that. Here's a screenshot showing it. Otherwise, i believe this PR is good to be merged unless there's anything else you see.


Machine win-6 is dead.

Is there an existing issue for this?

[X] I have searched the existing infra issues

Type of Request
infra task
Infrastructure Environment
LUCI, Github, Cocoon scheduler, Autosubmit, etc...
What is happening?
Machine win-6 is dead.

Steps to reproduce
Step 1:
Step 2:
..
Step n:
Expected results
I expect to see X when Y is finished.

Windows password is expired and required to reset.


BUG: On connections tab of settings, auth should use tenantId from the publishing profile!

Related to #6963
Publishing profiles now contain a tenantId.  We should use this when retrieving auth to work with the ABS channels on the connections tab.  Otherwise, this will not work for users with alternative tenants or personal accounts.

Shiproom - personal account scenario, which is important for R13

This should be fixed by #7554  - the tenantId is already in "new" publishing profiles, so we can apply it directly to the auth call.  If absent, we call back to the access token mechanism, which works for "legacy" profiles without the tenant specified.

Shiproom - make local accounts work is a P0 scenario - approved


Tag tweaks

Updated a few tags based on the official org structure spreadsheet (which only appeared recently)

Argh, wrong repo again sorry.


Include JDK 21 in CI build

https://jdk.java.net/21/

Coverage: 87.326%. Remained the same when pulling a8f048198f14d8fbc80cd1f0396d5825ea677fc6 on harawata:ci-on-jdk21 into e47fe38840da280f11219378fac9d8cd1b31d048 on mybatis:master.

Resolved by #2781

Thank you, @hazendaz !


MB-16393: Enable open telemetry collection in production

Jira ticket
Summary
This is entirely a change to how we deploy in production.

Messages

:book:
:link: MB-16393

  Generated by :no_entry_sign: dangerJS against f230a21c030d218890627894632987054fb76159


Fix a bug in swagger

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[x] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

/azp run automation - sdk

Can one of the admins verify this patch?

@yungezz - We have not released this version of API as  yet to public. Nor any of the SDK's have been generated using this version. It should be safe to check-in the change.

@yungezz - We have not released this version of API as yet to public. Nor any of the SDK's have been generated using this version. It should be safe to check-in the change.

thanks for confirmation. will merge it then.


add per-phone BLE RSSI cutoff

https://github.com/opentrace-community/opentrace-calibration

this initial code is to undergo more rigorous testing:
https://github.com/covidsafe/App-Android/commit/a0acbcc5c1aacb9dbb428f5c0c61d5cfb5bd3967


fix: add .keepOpen() method explanation and examples to chai-http challenges

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Related to https://github.com/freeCodeCamp/boilerplate-mochachai/pull/92

We got an email in the support inbox recently saying that none of the Learn tests for the Mocha Chai projects were working. The dev console on Learn threw CORS and network errors.
After digging into the issue, I found that chai-http automatically stops the server it's connected to after running its tests.
So what was happening is that, when the boilerplate on Replit is started, it runs the unit and functional tests automatically, and has a backdoor API endpoint for the tests on Learn to run against. But since chai-http would stop the server automatically, the project on Replit would stop, and all incoming Learn tests would fail.
I'm not sure why this wasn't an issue before, but these updated examples, and the updates to the boilerplate in https://github.com/freeCodeCamp/boilerplate-mochachai/pull/92, should fix things.
Here's a link to the completed project on Replit with these changes for testing<EMAIL_ADDRESS>

Hey @scissorsneedfoodtoo Does this close: https://github.com/freeCodeCamp/freeCodeCamp/issues/49634 ?

Thanks for linking to that existing issue, @ShaunSHamilton. I didn't see that earlier.
I'll take a closer look, drop a comment, and will close that issue if possible.


Adds a github action to support x64 performance testing using a sightglass

This github action allows performance testing using sightglass. The
action is triggered either via a workflow dispatch or with the comment
'/bench_x64', in a pull request. Once triggered the action will send
a request to a private repository that supports using a self-hosted runner
to do comparisons of "refs/feature/commit" vs "refs/heads/main" for
wasmtime. If the action is triggered via a comment in a pull request
(with '/bench_x64') then the commit referenced by the pull request is used
for the comparison against refs/head/main. If triggered via a workflow
dispatch the interface will request the commit to compare against
refs/head/main. The results of the performance tests, run via sightglass,
will be a table showing a percentage change in clock ticks in various stages
requried for executing the benchmark, namely instantiate, compiliation,
and execution. This patch is intended to be just a starting patch with much
to tweak and improve. One of the TODOs will be adding support for aarch64
.. currently this patch supports only x64. Note also that the logic for
actually doing the comparison and parsing the results occurs with the action
associated with the private repo and so this patch itself (though the trigger)
is fairly straight forward.

This patch is a replacement for #3740
Results currently look like this: https://github.com/jlb6740/wasmtime/pull/3#issuecomment-1179394254

This is looking quite nice! Would it be possible to add bytecodealliance/wasmtime-sightglass-benchmarking/.github/workflows/main.yml to this repository as well? I think it'd be ideal if we could have everything in this repository if we can.
Instead of a curl to send a repository dispatch to the private repository I'd imagine that we could instead push to a branch? For example the commit that wants to be benchmarked could be checked out as part of this workflow and then pushed to a special branch name in the private repository. The "actually run the benchmark" perf run would then trigger on this special branch name and do everything it's already doing today.
Review-wise I think it'd be best to be able to edit the workflow publicly in this repository rather than juggling hidden state elsewhere.

This is looking quite nice! Would it be possible to add bytecodealliance/wasmtime-sightglass-benchmarking/.github/workflows/main.yml to this repository as well? I think it'd be ideal if we could have everything in this repository if we can.

I would definitely like that as well, but how is it possible?

Instead of a curl to send a repository dispatch to the private repository I'd imagine that we could instead push to a branch? For example the commit that wants to be benchmarked could be checked out as part of this workflow and then pushed to a special branch name in the private repository. The "actually run the benchmark" perf run would then trigger on this special branch name and do everything it's already doing today.

@alexcrichton Would you like to see this refactor before merging this patch? Currently I am not sure how what you describe would work. The yml file that executes commands to download the correct sightglass, mainline, and patch, I would think needs to be associated with the private repo. That self-hosted runner executes the run steps in the github action yml so that needs to be part of the private repo's github actions. By putting logic here to push a patch to the private server I don't see yet what that really accomplishes. It introduces a patch on the private server but for what purpose? Currently the private server (that gets it's run steps from the yml) downloads the patch. The patch is never duplicated on any other branch, it just lives at its home of its original feature branch. Also, I am not sure how that would work when multiple requests at the same time?  In short, I don't yet see a way to accomplish the interesting steps in this repository alone. My thoughts are that while I completely agree from a review standpoint that it would be best to have all logic in one place, the private repository is that place and can be checked and has it's own review process. I don't think we have to sacrifice correctness. Just treat it like any other API .. when there is an issue just have to know which side to make the changes. From the wasmtime perspective, the curl statement sends a request to the private repository where the private repository is now a black box. If we want to scrutinize or update the logic the black box is doing, we just have to do it there.

Review-wise I think it'd be best to be able to edit the workflow publicly in this repository rather than juggling hidden state elsewhere.

Again, I do agree.

I would envision adding the workflow to this repository as:

The on: condition is only for branches that are named perf-test-*. This means it won't run for the main bytecodealliance repo
The workflow added to this repo would run on a custom runner. The custom runner is only added to the private repo. This is ok since it won't actually run in the public repo.
Otherwise the workflow in the private repo is basically the same, it checks out the branch whenever a push happens to perf-test-* and it could post results back to the PR when done.

Otherwise from this action added here the main difference would be instead of a curl a git push is done to a perf-test-* branch in the private repository.

Would you like to see this refactor before merging this patch?

Personally I would say yes. Having everything in one repo seems like a foundational shift relative to what you've currently got working which probably won't happen if we don't do it to start out with.

Also, I am not sure how that would work when multiple requests at the same time?

One idea would be for PR number N to push to perf-test-N so that way multiple PRs don't conflict.

the private repository is that place and can be checked and has it's own review process

The major downside, infrastructurally, is that you can't change atomically change the perf process in a PR to wasmtime. That means that the Wasmtime configuration to schedule a perf run must be kept in sync with the private repo's configuration. By having all the configuration in one place we can change everything at once in a single PR. Needing two PRs is not only cumbersome but runs the risk of breakage since the repos could get out of sync with each other.

@fitzgen @cfallin Hi guys .. had a good chat with @alexcrichton today and flushed out some concerns about the refactoring suggestion made. Its good to try the refactoring now. Since this is an anticipated patch, if there are any unanticipated roadblocks with the refactor we will look to merge this as is and address the refactor concern later, but we think the refactoring should work as expected. The idea is to have as much logic here as possible. Will start on that next week and hopefully have this ready for review the week after when @alexcrichton is back. I do anticipate that once the actions is updated there will be new comments and concerns to address before merger which is fine, so do anticipate that.

SGTM, really excited to get this up and running soon!

@alexcrichton Note, this latest patch is intended to address previous comments.

Looks great to me, thanks again for your patience!
I've got some minor nits below but it's fine to defer any or all of them to later if you'd prefer. Do you need me to configure any tokens or such in the repo before/after merging to have this work?

@alexcrichton No thanks for your patience! I think the refactoring works as imagined and is definitely an improved workflow as it combines everything in one patch and reduces logic. @alexcrichton @fitzgen @abrown I think this is ready to merge and iterate on. The only question I have is are the tokens workable. I believe they are, but of course those can be updated too.

Example output is here: https://github.com/jlb6740/wasmtime/pull/10
Note, the comment trigger /bench_x64 needs to be posted via the "review" comment box. It is not triggered by simply leaving a comment here in the "issue".


Update dependency docker.io/fluent/fluent-bit to v2 - abandoned

This PR contains the following updates:

Package
Update
Change

docker.io/fluent/fluent-bit
major
1.9.3 -> 2.1.1

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Renovate will not automatically rebase this PR, because other commits have been found.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, click this checkbox. ⚠ Warning: custom changes will be lost.

This PR has been generated by Renovate Bot.

Relevant release notes: https://github.com/fluent/fluent-bit/releases/tag/v2.0.0

Autoclosing Skipped
This PR has been flagged for autoclosing. However, it is being skipped due to the branch being already modified. Please close/delete it manually or report a bug if you think this is in error.


Flutter not rendering text after sitting idle

refs #57676
refs #55969
refs #64558
Our app uses flutter_background_fetch to send queued data to our API in the background.
We see missing image assets, icons and text on a regular basis. Force closing the app and restarting always fixes the issue.
[✓] Flutter (Channel stable, 1.22.0, on Mac OS X 10.15.6 19G2021, locale en-US)
• Flutter version 1.22.0 at /Users/mikerevoir/fvm/versions/stable
• Framework revision d408d302e2 (10 days ago), 2020-09-29 11:49:17 -0700
• Engine revision 5babba6c4d
• Dart version 2.10.0
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
• Android SDK at /Users/mikerevoir/Library/Android/sdk
• Platform android-30, build-tools 29.0.3
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b4-5784211)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 12.0.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 12.0.1, Build version 12A7300
• CocoaPods version 1.9.3
[✓] Android Studio (version 3.6)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin version 45.1.1
• Dart plugin version 192.8052
• Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b4-5784211)

@dukemike
ccan you provide a minimal reproducible code sample
that doesn't rely on 3rd party plugins like background_fetch
if you believe the issue is caused by said plugin I recommend you to open an issue in the dedicated github

@dukemike
Please provide a code sample as requested above that shows the issue you are facing.
Thanks.

@darshankawar unfortunately I'm not able to reproduce the issue at will. I don't remember if this issue started to occur before or after adding that plugin.

@dukemike we'll atleast need reproducible use case that could possibly show the issue you are facing.

This is almost certainly a duplicate of https://github.com/flutter/flutter/issues/64558

@dnfield Completely agree. We received one crash log on iOS that showed that the app was attempting to make a metal call while it was in the background.

Closing this as duplicate of 64558 per this comment.


Tiles and dates obtained from WcsRequest do not correspond to each other

Problem statement
I call req=WcsRequest(...), then I want to make additional filtering of tiles before actual downloading. So I use req.get_tiles() and req.get_dates() and produce from them indexes_to_dowload for the next call req.save_data(data_filter=indexes_to_download).
But it turned out that get_tiles() and get_dates() lists do not correspond to each other.

Generally get_tiles() and get_dates() in opposite direction. I.e. I need to reverse one of them to make them correspond to each other.
But if request was done with parameter time_difference and bounding box lies on intersection of tiles, then things go even worse. Then I get lists of different sizes, i.e.len(req.get_tiles()) != len(req.get_dates())

In this case it is even difficult to understand what indexes do I need to provide to save_data: either from get_dates() or get_tiles(). I guess that I need to use indexes of get_dates(), because it seems that get_dates() in correspondence with get_download_list(), at least by length.

So eventually I need to write a code for matching between lists of dates and tiles.
Minimal reproducing code (the worst case)
def main(INSTANCE_ID):
    intersection_of_tiles_wgs84 = [5.62788, 51.62314, 6.44577, 51.05245]
    betsiboka_bbox = BBox(bbox=intersection_of_tiles_wgs84, crs=CRS.WGS84)

    req = WcsRequest(layer='BANDS-S2-L1C',
                     bbox=betsiboka_bbox,
                     time=('2019-05-01', '2019-06-01'),
                     time_difference=datetime.timedelta(minutes=60),
                     resx='10m', resy='10m',
                     instance_id=INSTANCE_ID)

    tiles = req.get_tiles().tile_list
    dates = req.get_dates()
    for tile, date in zip(tiles, dates):
        print(tile['properties']['date'], tile['properties']['time'],
              ' | ',
              date)
    print('Number of tiles: ', len(tiles))
    print('Number of dates: ', len(dates))

Output
2019-05-31 10:56:59  |  2019-05-01 10:56:39
2019-05-31 10:56:54  |  2019-05-03 10:46:39
2019-05-31 10:56:50  |  2019-05-06 10:56:35
2019-05-31 10:56:45  |  2019-05-08 10:46:44
2019-05-31 10:56:41  |  2019-05-11 10:56:40
2019-05-31 10:56:39  |  2019-05-13 10:46:39
2019-05-28 10:47:03  |  2019-05-16 10:56:35
2019-05-28 10:47:00  |  2019-05-18 10:46:44
2019-05-28 10:46:58  |  2019-05-23 10:46:39
2019-05-28 10:46:48  |  2019-05-26 10:56:34
2019-05-28 10:46:45  |  2019-05-28 10:46:44
2019-05-28 10:46:44  |  2019-05-31 10:56:39
Number of tiles:  72
Number of dates:  12

Default code I would like to write
Here is the code, that I would expect to work correctly, but it doesn't.
req = WcsRequest(**any_parameters)

# filter before download
idxs = []
for i, (tile, date) in enumerate(zip(req.get_tiles(), req.get_dates())):
    if need_to_download(tile, date):
        idxs.append(i)

req.save_data(data_filter=idxs)

Is it a bug or maybe I haven't found a proper way of doing it?
Anyway, thanks for the great API !:)

I believe that get_dates() returns only one date per orbit (or datatake) where get_tiles() returns the meta-data for all the tiles in the orbit.
So you need to decide, what you would like to achieve and use the appropriate function. You can also take into account that orbit takes 100 minutes with Sentinel-2 and take this into account when setting time buffers.

@gmilcinski thanks for the answer!
get_dates() returns one date per orbit only in those particular example. If I remove time_difference, then get_dates() will return the same list as get_tiles(), but reversed. And if I will set time_difference to 10 seconds, then it will return 24 dates.
Inconsistency with documentation
get_dates
I am just read once more thoroughly documentation for get_dates method. At the end it states:

Most recent acquisition being first in the list.

But it can be seen from reproducing code that the first date in the list is the oldest date.
get_tiles
Documentaiton

Returns iterator over info about all satellite tiles used for the OgcRequest

It is not quite clear what does it mean all satellite tiles used for the request. Eventually it filters meta information for all tiles out of date range, but it fails to filter meta information with respect to time_difference parameter. Both parameters were used for the OgcRequest.
My intention
I believe that get_tiles intended to provide meta information for the tiles that will be downloaded.
I want to obtain meta information for the tiles that will be downloaded, how can I achieve it?

Well a little inconsistency at the end of the method OgcImageService.get_dates().
Problem code
    class OgcImageService(OgcService):
        # ...
        def get_dates(self, request):  # from documentation supposed to return recent first
            # ...
            return OgcService._filter_dates(dates, request.time_difference)

    class OgcService:
        def _filter_dates(dates, time_difference):  # from documentation supposed to return oldest first
            #...
            sorted_dates = sorted(dates)  # oldest dates first
            # filter with respect to time_difference, no change order
            return separate_dates

It seems that there is no important code that depends on assumption that get_dates should return recent acquisitions first.
Anyway I suggest to clarify documentation on get_tiles() method and add a new one method get_filtered_tiles() that will return meta information consistent with get_dates() and get_download_list() methods.
I think it is a very natural wish to save meta information together with tile. But in current state of API to do it you need to duplicate code of _filter_dates(..) method what potentially leads to errors in client code.
Is it worth to make a pull request on it?

Hi @pisarik,
I checked the code and I can confirm the following:

There is a mistake in the documentation of get_dates method. The method will always return dates from least recent to most recent.
Documentation about get_tiles could have amore detailed explanation.

Explanation, why get_tiles and get_dates return different number of items is the following:
A user-specified bounding box is likely to overlap with multiple Sentinel-2 tiles that either have completely the same acquisition timestamp or timestamps that differ for a few minutes. Therefore Sentinel Hub service will collect data from multiple tiles with the same timestamp and join them into a single image that will be provided through the Python package. This means that one part of an image could be obtained from one tile and the other part from another.
Hence get_tiles returns info about all tiles from which Sentinel Hub service will collect some data from. On the other hand, get_dates will return acquisition dates of returned images.
The time_difference parameter is used to tell Sentinel Hub service that it can join even acquisitions that differ for the specified amount of time (e.g. few minutes or hours) into a single one.
We have just started working on adding support for new Sentinel Hub API in this package. So we can update this part of the documentation as well.
Regarding your suggestion about get_filtered_tiles I don't think it makes sense as there will usually be more tiles involved than the number of images returned by the service. The service also doesn't return information if some of these tiles haven't been used at all.

Dear @AleksMat,
Thank you very much for thorough explanation!
Merging several tiles into a single image
Are you sure that tiles with timestamps withing time_difference will be merged into a single image? Actually this parameter almost doesn't have explanation in documentation. The only place is 8th example and there is nothing said about merging tiles into one single image.
As I could understand from example explanation and code documentation [1] [2], if several tiles happened to be in time_difference interval, then all tiles will be rejected except the oldest one.
If they really merge in a single image, could you please provide an idea of merging? Namely, how conflicts do resolved in case of overlap of non-blank areas from several tiles?
Obtaining meta information for the downloaded images
I think you missed my point.
Suppose we are in good case (bbox belongs only to one tile). I want to save a meta information of tile (cloud coverage, path on aws, etc.) along with images. How can I get this meta information?
Proper question for all cases
How can I get a list of meta information for the tiles that were used to produce a downloaded image?

Closing due to long-term inactivity


Uri for home url always forced to end in /

TIL that if you have a System.Uri that is for the root, e.g. a path of string.Empty, it will always render out from .ToString() as "https://mydomain.com/". Note the trailing slash.
Apparently this is to adhere to the HTTP spec, see link from the SO post.all equivalent. https://stackoverflow.com/a/46980376/725626
The spec is basically saying no slash or with slash on root is the same, but you should normalize to with the /.
My issue though is that may not be the canonical URL you'd want for a site that doesn't use trailing slashes, and I wonder if parsers of Schemas are respecting the spec...
What do you think? More just opening a discussion than logging a bug.

https://mydomain.com/ and https://mydomain.com are equivelant for search engines like Google. Trailing slashes only matter for paths. I asked the same question some years ago.


Error: "cannot create a file when that file already exists"

Checklist

[x] I'm asking a question
[x] I've looked through the README and FAQ for similar questions
[x] I've searched the bugtracker for similar questions including closed ones

Question

I have the latest version of youtube-dl (2019.07.16). Even in previous versions I would frequently get this error when downloading playlists from youtube. "cannot create a file when that file already exists", though no such files were there. It was a fresh download. I'm left with a file ending with TEMP in the filename, then the playlist download process is stopped because of that error. Any way to fix that? Thanks

Post complete verbose log clearly demonstrating the issue.


Blend Micro no visibility when creating a BLEPeripheral

Hi,
Brilliant library, used it to make many a HID device. I've recently tried to get the test app from the library working via the Arduino IDE.
I can get HID devices to be recognised by my iPhone (iOS9.3.3) no problem however when trying to use just BLEPeripheral.h to declare a BLE device it isn't recognised by any devices on iOS or Mac.
I'm using 1.0.3 RBL AVR Boards.
The code uploads correctly and the Serial outputs that it's initialised however I still can't see them.
Any suggestions appreciated.

Very sorry, this issue is down to myself. I've since discovered that iOS doesn't show LE connections. Apologies.

@withgallantry I would suggest using the LightBlue app from the app store for testing.
More info. in the "Make: Bluetooth" book if you are interested. (http://makebluetooth.com).


Make it jaunty on the client

Currently, we add a random amount of jaunt to the yellow box using styled-component with SSR, then we rehydrate this behaviour on the client. But the random value is different on the client, so styled-components creates a different class and Next complains (admittedly only in development) that there's a mismatch between client and server.

If we add the jauntiness client-side, we can circumvent this warning (and get a bit of animation).
Not sure if we need to consider the mismatch as a real issue, but if this (adding-client-side-with-animation) approach is preferred, we don't have to worry about whether it is or not.

@davidpmccormick do we want to take people who prefer reduced movement into account for this?

Closing as there's not an obvious user need and it raises more questions than it answers.


Run at least one random test with ImperasDV enabled

This task cannot make progress until a Member with an ImperasDV license joins the effort.

Closing this task as it is redundant due to #227.


Add EXOOfflineAddressBook

Pull Request (PR) description
Add DSC resource EXOOfflineAddressBook
This Pull Request (PR) fixes the following issues

This change is

@malauter looks like you will need to manually add the Get-OfflineAddressBook stub to the Generic stub file for the tests to go through

Codecov Report

Merging #380 into Dev will increase coverage by 0.78%.
The diff coverage is 97.91%.

@@            Coverage Diff            @@
##              Dev    #380      +/-   ##
=========================================
+ Coverage   89.72%   90.5%   +0.78%
=========================================
  Files          83      95      +12
  Lines        8807   10215    +1408
  Branches       10      10
=========================================
+ Hits         7902    9245    +1343
- Misses        895     960      +65
  Partials       10      10

Impacted Files
Coverage Δ

...OfflineAddressBook/MSFT_EXOOfflineAddressBook.psm1
97.91% <97.91%> (ø)

...s/MSFT_EXOSharingPolicy/MSFT_EXOSharingPolicy.psm1
97.82% <0%> (ø)

...esOrganization/MSFT_EXOOnPremisesOrganization.psm1
98.11% <0%> (ø)

...AvailabilityConfig/MSFT_EXOAvailabilityConfig.psm1
89.47% <0%> (ø)

...PartnerApplication/MSFT_EXOPartnerApplication.psm1
98% <0%> (ø)

...FT_EXOPolicyTipConfig/MSFT_EXOPolicyTipConfig.psm1
97.59% <0%> (ø)

...Relationship/MSFT_EXOOrganizationRelationship.psm1
96.12% <0%> (ø)

...ces/MSFT_EXORemoteDomain/MSFT_EXORemoteDomain.psm1
75.19% <0%> (ø)

...AccessRule/MSFT_EXOActiveSyncDeviceAccessRule.psm1
97.77% <0%> (ø)

...EmailAddressPolicy/MSFT_EXOEmailAddressPolicy.psm1
97.97% <0%> (ø)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95d48f5...e2152fa. Read the comment docs.


Can't copy ColorSchemes/Light.xaml into my app because ResourceAliasExtension class is internal

(thanks for your work on this excellent library!)
Describe the bug
"Custom color schemes" documentation recommends creating a copy of Light.xaml and/or Dark.xaml to customize. However, Light.xaml uses AdonisUI.Helpers.ResourceAliasExtension is internal, so I can't reference it from outside the AdonisUI dll. This means I effectively can't have a customised copy of Light.xaml in my own app.
To Reproduce
Steps to reproduce the behavior:

Copy AdonisUI/ColorSchemes/Light.xaml from Github into your application
Try to compile
See error:   LightThemeColors.xaml(18, 90): [MC3064] Only public or internal classes can be used within markup. 'ResourceAliasExtension' type is not public or internal.

Additional context
Workaround is to create a copy of AdonisUI/Helpers/ResourceAliasExtension.cs in my own project - but would be nicer not to have to (i.e. if ResourceAliasExtension was a public class).

I see this is kind of misleading, sorry for that. The ResourceAliasExtension should be either public or the docs should be more clear about how to create custom color schemes.
I'll try to explain it a little more detailed here.
If you want to create a whole new color scheme you can create a new ResourceDictionary file and add a new Color for each key in AdonisUI.Colors and a new Brush (e.g. SolidColorBrush) for each key in AdonisUI.Brushes. Don't forget to use AdonisUI's keys for all those resources, e.g.:
<Color x:Key="{x:Static adonisUi:Colors.AccentColor}">#0BAC08</Color>
<!-- all colors go here -->

<SolidColorBrush x:Key="{x:Static adonisUi:Brushes.AccentBrush}" Color="{DynamicResource {x:Static adonisUi:Colors.AccentColor}}"/>
<!-- all brushes go here -->

If you want to use a shipped color scheme and customize only some resources you can create a new ResourceDictionary file and make it include the theme you want to derive from in its ResourceDictionary.MergedDictionaries. Afterwards, you can create colors as you like and assign the color keys you want to override.
<ResourceDictionary xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
                    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
                    xmlns:local="clr-namespace:AdonisUITest"
                    xmlns:adonisUi="clr-namespace:AdonisUI;assembly=AdonisUI">

    <ResourceDictionary.MergedDictionaries>
        <ResourceDictionary Source="pack://application:,,,/AdonisUI;component/ColorSchemes/Light.xaml"/>
    </ResourceDictionary.MergedDictionaries>

    <Color x:Key="{x:Static adonisUi:Colors.AccentColor}">Green</Color>
    <!-- additional colors or brushes go here -->

</ResourceDictionary>

The colors and brushes you did not specify are then taken from the included color scheme. You can include this ResourceDictionary in your App.xaml instead of the light and dark color schemes or in addition to them if you want to switch between those.
So now regarding your question: The ResourceAliasExtension is only used for Layer0BorderColor (and the associated brush). You can now either leave this color as it is by including the light color scheme and simply not overriding it. Or you assign a new value to it like mentioned:
<Color x:Key="{x:Static adonisUi:Colors.Layer0BorderColor}">#404040</Color>

Hope this helps :)

Yes that does help, thanks!
I didn't think to use the Adonis Light.xaml as a MergedDictionary inside my own ColorScheme. That's much better than copying the whole thing.


Make wandb checkpoint logging compatible with wandb model registry

What does this PR do?
This PR modifies wandb_logger.py to instantiate model checkpoints as type "model" instead of ".pt", to allow compatibility with W&B's new model registry feature, which MosaicML and WandB are cross-promoting with a demo/webinar next week.
Before submitting

[x] Have you read the contributor guidelines?
[ ] Is this change a documentation change or typo fix? If so, skip the rest of this checklist.
[ ] Was this change discussed/approved in a GitHub issue first? It is much more likely to be merged if so.
[x] Did you update any related docs and document your change?
[x] Did you update any related tests and add any new tests related to your change? (see testing)
[x] Did you run the tests locally to make sure they pass?
[x] Did you run pre-commit on your change? (see the pre-commit section of prerequisites)

@eracah

LGTM. Just curious: do you have a link to a wandb run that used this?
Why yes I do


Polygon-SDK accepts transactions with an invalid gas limit as valid transactions

Polygon-SDK accepts transactions with an invalid gas limit as valid transactions
Description
During our internal testing we notice that if we submit a transaction with an invalid gas limit (this was done on purpose) to a blockchain created using Polygon-SDK the transaction is accepted by the blockchain and added to the tx pool.
Later when the method writeTransactionsruns the blockchain will check the gas limit and if the value is invalid the transaction will be silently discarded.
This poses a problem, because the transaction is silently discarded applications that use the blockchain (like metamask) will not be able to receive a confirmation for this transaction meaning that the transaction will be "forever pending".
This exact use case doesn't happen in an ethereum test network like the Rinkeby Test Network. The ethereum blockchain validates the gas limit before adding the transaction to the tx pool and returns an appropriate error to the application interacting with the blockchain (like metamask).
Attached is a video that shows this test using the Rinkeby Test Network and a blockchain created using the Polygon-SDK.
Your environment

OS and version: Windows 10 (WSL Ubuntu 20.04.3 LTS
version of the Polygon SDK: develop branch at commit 7fe8091967102192539618fd786f226d54dd312b

Steps to reproduce
Video that shows the use cases in the issue description https://1drv.ms/u/s!AvhuC05QeOJntn9-aiUsXaKgKqZN
Expected behaviour
We believe the Polygon-SDK should behave similar to the ethereum blockchain and return an error when we submit an invalid transaction.
Actual behaviour
The Polygon-SDK accepts a transaction with an invalid gas limit as a valid transaction unlike the ethereum network.
Logs
Logs from Polygon-SDK:
2021-12-15T16:23:52.952Z [INFO]  polygon.consensus.ibft.acceptState: we are the proposer: block=71149
2021-12-15T16:23:52.952Z [ERROR] polygon.consensus.ibft: failed to write transaction: transaction's gas limit exceeds block gas limit
2021-12-15T16:23:52.952Z [INFO]  polygon.consensus.ibft: picked out txns from pool: num=0 remaining=0

Proposed solution
Maybe do something similar to go-ethereum where they have the check in validateTx, screenshot from go-ethereum:

Hey @AzulPretoBranco,
Thank you for reaching out.
So far what you've described is normal behavior - the transaction is accepted into the TxPool of the Polygon SDK but is discarded when it is attempted to be applied.
This is because block limits are enforced during transaction execution when the block is being built, and not at the point where you're adding the transaction - which may or may not succeed for whatever reason.
The reason block limits are not checked in transaction addition is because these are consensus specific - the TxPool shouldn't care about whether or not the transaction can be executed if it passes the basic checks to be in the pool in the first place.
Apps like Metamask, or any web3 providers in general query the transaction by hash or attempt to get the receipt of the transaction after submitting it, and update their own status according to the responses.
I'll sync with the rest of the team so we can find an optimal solution to this pending transaction status in MM.
I'll keep you posted on this, moving this to a discussion since it's not necessarily an issue with the Polygon SDK, but rather a feature request 👍


Track upstream changes (update to 23.1.x, fix usage of plist_options)

I confirmed locally that the cockroach service is started with the correct --store=type=mem,size=8GB parameter and that a local bin/environmentd works against it.
Plus, this get's rid of an annoying homebrew warning about plist_options that would show up whenever running any brew command (upstream fixed this by removing usage of that deprecated feature).

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 12 committers have signed the CLA.:white_check_mark: aljoscha:x: celiala:x: rail:x: ZhouXing19:x: lidorcarmel:x: natelong:x: renatolabs:x: thadc23:x: michae2:x: kpatron-cockroachlabs:x: THardy98:x: jenngeorgeYou have signed the CLA already but the status is still pending? Let us recheck it.


Datasource Entries dimensions missing

This PR fixes the issue #14 . Now, we will split into two different types, the Datasources and Datasources Entries, and, for the last type, we get the dimensions as well.

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


quadsing: quadrature for integrals with (fractional power like) singalarities at endpoints

Hi.
mpmath's quad is surprisingly bad at integrating something like
quad(lambda x: x**(-99999999/mpf(100000000)), [0, 1])
I added the function function quadsing which can handle such integrals.
The function works reasonably well for power-like singularities x**e with
complex exponent e at endpoints.
Greetings
Timo
Edit: The check was not successful because 3.10-dev could not build gmpy as dependency...

@thartmann15, you can cherry-pick from #601 to fix gmpy2 problem.

Hi. I have cherry-picked the fix.

Please add more tests to cover remaining lines.

more code coverage tests added

@thartmann15, could you please fix merge conflicts?

@thartmann15, I'm closing this.  Feel free reopen the pr if you wish to continue this work.


test: add unit tests for util functions

@Tylertron1998 are you happy now?

give me coverage stats or riot.

For util.ts:
Basically all that is feasible to unit test, has been unit tested. Some else cases and branches aren't covered yet but idk how they would even be covered.


feat: make auto dependent on other columns

Using the same approach for auto setting as for sn-table. Calculating the independent widths (pixels, percentage, fit to content), subtracting the total width of them, then dividing the remaining width between the auto columns. This is only applicable when the psuedo dimension is deciding the width.
Now also calculating the width at the same time, meaning they are stored in an array. This means we don't have to memoize the function anymore

:tada: This PR is included in version 2.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


clone for BodyResReady

So it can be used in enums with the other message body structs.

Can we just remove this type entirely? Everything it does is a noop.

I'd say it's probably best to keep it because we represent all other types of Cassandra messages.


cannot convert model to tflite

when I try to converto to tflite, occur below errors
$ tflite_convert --output_file=./model1/test.tflite --keras_model_file=./model1/186-0.0481.hdf5
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:96: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:1253: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:96: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
2019-02-12 18:00:17.730635: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/lite.py:636: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.convert_variables_to_constants
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/graph_util_impl.py:245: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
2019-02-12 18:00:18.207014: I tensorflow/core/grappler/devices.cc:53] Number of eligible GPUs (core count >= 8): 0 (Note: TensorFlow was not compiled with CUDA support)
2019-02-12 18:00:18.207141: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session
2019-02-12 18:00:18.210189: E tensorflow/core/grappler/grappler_item_builder.cc:636] Init node dense/kernel/Assign doesn't exist in graph
Traceback (most recent call last):
File "/usr/local/bin/tflite_convert", line 10, in
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/tflite_convert.py", line 442, in main
app.run(main=run_main, argv=sys.argv[:1])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/tflite_convert.py", line 438, in run_main
_convert_model(tflite_flags)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/tflite_convert.py", line 191, in _convert_model
output_data = converter.convert()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/lite.py", line 500, in convert
**converter_kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/convert.py", line 442, in toco_convert_impl
input_data.SerializeToString())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/convert.py", line 205, in toco_convert_protos
"TOCO failed. See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: TOCO failed. See console for info.
2019-02-12 18:00:19.028566: F tensorflow/core/framework/function.cc:1640] Check failed: GetOpGradFactory()->insert({op, func}).second Duplicated gradient for MapAccumulate
Aborted (core dumped)
How can I fixed it?
somebody has any idea?

Please fill the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Along with the template, please provide as many details as possible to find the root cause of the issue. It would be great if you can provide a small code to reproduce the error. Thanks!

Python version : Python 2.7.6
Bazel version : Build label: 0.21.0
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.4) 4.8.4
(Don't use GPU)
This is my environment information.

I used test example code --> (part of keras) https://www.tensorflow.org/lite/convert/python_api
and occur error like this
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:1253: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/base_layer_utils.py:123: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
2019-02-13 11:33:51.428806: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:96: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/lite.py:636: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.convert_variables_to_constants
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/graph_util_impl.py:245: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
2019-02-13 11:33:51.991970: I tensorflow/core/grappler/devices.cc:53] Number of eligible GPUs (core count >= 8): 0 (Note: TensorFlow was not compiled with CUDA support)
2019-02-13 11:33:51.992069: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session
2019-02-13 11:33:51.995173: E tensorflow/core/grappler/grappler_item_builder.cc:636] Init node dense/kernel/Assign doesn't exist in graph
Traceback (most recent call last):
File "./tensorflow/lite/examples/simple_model_test/make_model.py", line 26, in
tflite_model = converter.convert()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/lite.py", line 500, in convert
**converter_kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/convert.py", line 442, in toco_convert_impl
input_data.SerializeToString())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/convert.py", line 205, in toco_convert_protos
"TOCO failed. See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: TOCO failed. See console for info.
2019-02-13 11:33:52.824627: F tensorflow/core/framework/function.cc:1640] Check failed: GetOpGradFactory()->insert({op, func}).second Duplicated gradient for MapAccumulate
Aborted (core dumped)

Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.19.0-25-generic x86_64)
Tensorflow build from source

@adppark Please check tested build configurations. I think this error may be due to bazel version 0.21.0. Could you check the version of TF by print(tf.version). Thanks!

Python 2.7.6 (default, Nov 13 2018, 12:45:42)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
print(tf.version)
<module 'tensorflow._api.v1.version' from '/usr/local/lib/python2.7/dist-packages/tensorflow/_api/v1/version/init.pyc'>

Is this right?

when I try to with bazel version 0.15.0
printed like this
release@VDBS1109:~/tensorflow$ ./configure
WARNING: Running Bazel server needs to be killed, because the startup options are different.
WARNING: ignoring http_proxy in environment.
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.15.0 installed.
Please upgrade your bazel installation to version 0.19.0 or higher to build TensorFlow!
Which version is correct?

@adppark Were you able to install tensorflow? Could you execute
import tensorflow as tf
print tf.version
If you have TF1.12, then Bazel 0.15.0 is supported and if you are installing recent versions >TF1.12, then Bazel 0.19.0 or Bazel 0.21.0 works. Thanks!

@jvishnuvardhan
I already installed tensorflow.
I tried build with Bazel version 0.19.0, 0.21.0, 0.15.0..
Bur always occur error message same as..
$ bazel version
INFO: Invocation ID: 65ccb62c-d398-46bc-8535-f4d2f1c072cb
Build label: 0.22.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Jan 28 12:58:08 2019<PHONE_NUMBER>)
Build timestamp:<PHONE_NUMBER>
Build timestamp as int:<PHONE_NUMBER>
$ python
Python 2.7.6 (default, Nov 13 2018, 12:45:42)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
**>>> import tensorflow as tf

print tf.version
1.12.0**

WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:1253: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/base_layer_utils.py:123: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
2019-02-15 13:25:49.564196: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:96: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/lite.py:636: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.convert_variables_to_constants
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/graph_util_impl.py:245: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
2019-02-15 13:25:50.157624: I tensorflow/core/grappler/devices.cc:53] Number of eligible GPUs (core count >= 8): 0 (Note: TensorFlow was not compiled with CUDA support)
2019-02-15 13:25:50.157737: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session
2019-02-15 13:25:50.160530: E tensorflow/core/grappler/grappler_item_builder.cc:636] Init node dense/kernel/Assign doesn't exist in graph
Traceback (most recent call last):
File "./make_model.py", line 26, in
tflite_model = converter.convert()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/lite.py", line 500, in convert
**converter_kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/convert.py", line 442, in toco_convert_impl
input_data.SerializeToString())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/convert.py", line 205, in toco_convert_protos
"TOCO failed. See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: TOCO failed. See console for info.
2019-02-15 13:25:50.973720: F tensorflow/core/framework/function.cc:1640] Check failed: GetOpGradFactory()->insert({op, func}).second Duplicated gradient for MapAccumulate
Aborted (core dumped)

@gargn @jvishnuvardhan  Can the issue be because of using Python2.7? Python2.7 will retire in 2020 and seeing the warnings quoted below, I think it's because of that only.

WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:1253: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/base_layer_utils.py:123: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
2019-02-15 13:25:49.564196: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:96: calling init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/lite.py:636: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.

Can you try using the latest TF version? If that doesn't work, can you provide your model or a minimal model that produces the same error?

having this issue with this docker container tensorflow/tensorflow:latest-py3
docker run -it --rm -v $PWD:/tmp -w /tmp tensorflow/tensorflow:latest-py3 tflite_convert --outut_file=/tmp/foo.tflite --keras_model_file=/tmp/da19.hdf5

Traceback (most recent call last):
  File "/usr/local/bin/tflite_convert", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/lite/python/tflite_convert.py", line 442, in main
    app.run(main=run_main, argv=sys.argv[:1])
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/lite/python/tflite_convert.py", line 438, in run_main
    _convert_model(tflite_flags)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/lite/python/tflite_convert.py", line 122, in _convert_model
    converter = _get_toco_converter(flags)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/lite/python/tflite_convert.py", line 109, in _get_toco_converter
    return converter_fn(**converter_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/lite/python/lite.py", line 370, in from_keras_model_file
    keras_model = _keras.models.load_model(model_file)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/saving.py", line 232, in load_model
    raise ValueError('No model found in config file.')

@danilaplee Please file a new GitHub issue which describes your problem in detail and provides instructions on how to reproduce it.

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

lol tensorflow converter doesnt work and will never work !!! why ? simple because it has never worked to begin with. evidence of this is shown here = zero solutions are shown , zero 100% solution to the issue and above all just waste of time


RN mod update for ksp 1.2.2

change compatible version since the mods all still work

Can you change those "ksp_version_min": "1.2" to "ksp_version_min": "1.2.0"

@politas done

I can't see any reason for those to fail, given the scope of the changes, so I'll merge this.


Dishonored 2 won't boot to menu

Dishonored 2 launches with a black screen then freezes.
Software information
Dishonored 2, default settings
System information

GPU: NVIDIA GeForce GTX 1050
Driver: Vulkan Beta 516.89
DXVK version: dxvk-master-2fa628fce600b6573e4250ab66c77b6c310caf77
OS: Windows 10 21H2

Log files

d3d11.log: Dishonored2_d3d11.log
dxgi.log: Dishonored2_dxgi.log

Updated to latest Vulkan beta driver and it solved the issue.


Profile for 2019 Rav4 TSS2 Limited

Can you create a profile for my car i can work with you on testing and fine tuning

What's the main reason you want an accel profile? I think once I work on fine tuning it, it should be better for all cars

Closed for inactivity


Bump govuk_tech_docs from 1.8.3 to 2.0.11

Brings in the latest govuk_tech_docs gem which includes GOVUK Frontend 3.0 which brings in changes to make its components and styles WCAG 2.1 compliant.

the CHANGELOG entry for the GOVUK Frontend 3.0 release
all changes to govuk_tech_docs this brings in

This is a partial replacement for https://github.com/alphagov/notifications-tech-docs/pull/92 which tried to bump govuk_tech_docs to 2.0.5 but was blocked by https://github.com/alphagov/tech-docs-gem/issues/133.
Also: the branch name is now incorrect. Sorry.

Slightly delayed by this: https://github.com/alphagov/tech-docs-gem/issues/163

Slightly delayed by this: alphagov/tech-docs-gem#163

This issue is now resolved so moved this pull request out of [WIP].


Pagination is broken if there is only one page

[X] I have searched the Issues to see if this bug has already been reported
[X] I have tested the latest version

Steps to reproduce

Render pagination component with this props:

<Pagination
  currentPage={1}
  totalPages={1}
  showIcons
  nextLabel=""
  previousLabel=""
/>

Current behavior
There is no 1 between the arrows, just two buttons (previous and next)

Expected behavior
1 is shown between the arrows, similarly to the case when we have 2 and more pages.

Context
I would like to show the Pagination component with only one page when user has a small amount of data.
The problem is in range function, where this check is contained
if (start >= end) {
    return [];
}

I would assume it should be
if (start > end) {
    return [];
}

but not sure whether this breaks the expected behaviour from your side. If you are fine I can create the PR.
Thanks,
Sergii

I think maybe this needs to be currentPage < 1 also
For this
  currentPage={0}
  totalPages={0}
  showIcons
  nextLabel=""
  previousLabel=""
/> ```

<img width="147" alt="pagination-0" src="https://github.com/themesberg/flowbite-react/assets/27743176/07fdbfbf-3c2a-48d2-8420-f8e089edc0ba">

we wouldn't want them to be able to click previous if no pages are available

we wouldn't want them to be able to click previous if no pages are available

Agreed, but this doesn't mean we do not show the pages, even if its just one.

@tulup-conner I've raised a PR to fix this

I have the same issue. Just waiting for the fix.
Thanks!


Remove crochet_ast dep from crochet_types

This is necessary in order for us to add inferred_type fields to AST nodes in crochet_ast, otherwise there'd be a cyclic dependency between crates which isn't allowed.

Codecov Report

Merging #231 (372e76d) into main (d974051) will decrease coverage by 0.19%.
The diff coverage is 87.50%.

@@            Coverage Diff             @@
##             main     #231      +/-   ##
==========================================
- Coverage   93.80%   93.61%   -0.20%
==========================================
  Files          32       33       +1
  Lines        4119     4133      +14
==========================================
+ Hits         3864     3869       +5
- Misses        255      264       +9

Impacted Files
Coverage Δ

crates/crochet_ast/src/expr.rs
59.37% <ø> (ø)

crates/crochet_infer/src/infer_pattern.rs
86.90% <50.00%> (ø)

crates/crochet_types/src/lit.rs
71.42% <60.00%> (ø)

crates/crochet_ast/src/lit.rs
91.30% <77.77%> (ø)

crates/crochet_infer/src/util.rs
96.15% <77.77%> (-0.77%)
:arrow_down:

crates/crochet_types/src/type.rs
80.61% <83.33%> (+0.61%)
:arrow_up:

crates/crochet_ast/src/prim.rs
42.85% <85.71%> (-42.86%)
:arrow_down:

crates/crochet_types/src/prim.rs
85.71% <85.71%> (ø)

crates/crochet_dts/src/parse_dts.rs
83.22% <87.50%> (ø)

crates/crochet_codegen/src/d_ts.rs
80.14% <90.90%> (ø)

... and 8 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Metrics are not collected

Hi,
I have installed the metrics exporter on my Lightsreamer instance but nothing is collected and I receive this error message.:
2022-03-08 12:38:10,602 WARN c.l.u.m.j.JMXMetricsCollector [Thread-21] Error while querying mbeans
java.lang.reflect.UndeclaredThrowableException: null
at com.sun.proxy.$Proxy26.getStatus(Unknown Source) ~[?:?]
at com.lightstreamer.utility.metrics.jmx.JMXMetricsCollector.lambda$inititalize$4(JMXMetricsCollector.java:326) ~[lightstreamer-utility-metricsexporter-adapter-0.1.0.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: javax.management.AttributeNotFoundException: No such attribute: Status
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:81) ~[?:?]
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206) ~[?:?]
at javax.management.StandardMBean.getAttribute(StandardMBean.java:372) ~[?:?]
I am using the latest verison of Lightstreamer docker image:7.2.2

Hi Mojtaba Karimi,
It seems that your Lightstreamer license does not support the full JMX interface feature; or it is disabled through configuration.
Please could you check in the 'License' tab of the Monitoring Dashboard if the 'JMX Management API' feature is enabled?
However, I believe that the message is not completely clear as to the reasons for the error; I think a change is needed to provide a more specific message.
Thank you,
Giuseppe

Hi Mojtaba Karimi,
It seems that your Lightstreamer license does not support the full JMX interface feature; or it is disabled through configuration. Please could you check in the 'License' tab of the Monitoring Dashboard if the 'JMX Management API' feature is enabled?
However, I believe that the message is not completely clear as to the reasons for the error; I think a change is needed to provide a more specific message.
Thank you, Giuseppe

Hello Giuseppe,
Thanks for your response. you were right despite the fact that we are using the enterprise license, this feature is disabled. is there any way to enable it?

Hi Mojtaba Karimi,
It seems that your Lightstreamer license does not support the full JMX interface feature; or it is disabled through configuration. Please could you check in the 'License' tab of the Monitoring Dashboard if the 'JMX Management API' feature is enabled?
However, I believe that the message is not completely clear as to the reasons for the error; I think a change is needed to provide a more specific message.
Thank you, Giuseppe

Hello Giuseppe, Thanks for your response. you were right despite the fact that we are using the enterprise license, this feature is disabled. is there any way to enable it?

The 'JMX Management API' is an optional feature and can be enabled or not based on the list of features included in the enterprise license purchased. If not, a license upgrade will be required.
However, if you are interested in testing the export of the Lightstreamer server metrics to prometheus, you can use a test server running the DEMO license type.

Hi Mojtaba Karimi,
It seems that your Lightstreamer license does not support the full JMX interface feature; or it is disabled through configuration. Please could you check in the 'License' tab of the Monitoring Dashboard if the 'JMX Management API' feature is enabled?
However, I believe that the message is not completely clear as to the reasons for the error; I think a change is needed to provide a more specific message.
Thank you, Giuseppe

Hello Giuseppe, Thanks for your response. you were right despite the fact that we are using the enterprise license, this feature is disabled. is there any way to enable it?

The 'JMX Management API' is an optional feature and can be enabled or not based on the list of features included in the enterprise license purchased. If not, a license upgrade will be required.
However, if you are interested in testing the export of the Lightstreamer server metrics to prometheus, you can use a test server running the DEMO license type.

Our production license is expiring soon so we might try to add this feature in our next purchase.


No extension metadata for mapped superclass

I have a single table inheritance setup, where my base entity has the Timestampable fields mapped like this:
<?xml version="1.0" encoding="UTF-8"?>

<doctrine-mapping xmlns="http://doctrine-project.org/schemas/orm/doctrine-mapping"
                  xmlns:gedmo="http://gediminasm.org/schemas/orm/doctrine-extensions-mapping"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                  xsi:schemaLocation="http://doctrine-project.org/schemas/orm/doctrine-mapping
                                      http://doctrine-project.org/schemas/orm/doctrine-mapping.xsd">

    <entity name="MyProject\Common\Entity\Page\AbstractPage"
        inheritance-type="SINGLE_TABLE"
        repository-class="MyProject\Common\Repository\PageRepository"
        table="page">

        <id name="id" column="id" type="integer">
            <generator strategy="AUTO" />
        </id>

        <discriminator-column name="type" type="pagetype" length="25" />
        <discriminator-map>
            <discriminator-mapping value="rootpage" class="MyProject\Common\Entity\Page\RootPage" />
            <discriminator-mapping value="contentpage" class="MyProject\Common\Entity\Page\ContentPage" />
        </discriminator-map>

        <!-- TIMESTAMPABLE FIELDS -->
        <field name="createdAt" column="created_at" type="datetime">
            <gedmo:timestampable on="create"/>
        </field>
        <field name="updatedAt" column="updated_at" type="datetime">
            <gedmo:timestampable on="update"/>
        </field>
        <!-- /TIMESTAMPABLE FIELDS -->
    </entity>

</doctrine-mapping>

When trying to persist a child class, the timestampable fields are left empty.
Example child class mapping:
<?xml version="1.0" encoding="UTF-8"?>

<doctrine-mapping xmlns="http://doctrine-project.org/schemas/orm/doctrine-mapping"
                  xmlns:gedmo="http://gediminasm.org/schemas/orm/doctrine-extensions-mapping"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                  xsi:schemaLocation="http://doctrine-project.org/schemas/orm/doctrine-mapping
                                      http://doctrine-project.org/schemas/orm/doctrine-mapping.xsd">

    <mapped-superclass name="MyProject\Common\Entity\Page\RootPage">
    </mapped-superclass>

</doctrine-mapping>

It appears there is a check in the ExtensionMetadataFactory which blocks retrieving the config if the object is a mapped superclass: https://github.com/Atlantic18/DoctrineExtensions/blob/v2.4.x/lib/Gedmo/Mapping/ExtensionMetadataFactory.php#L74
When I remove that check, everything seems to be working. Is there any reason why this check needs to be here?

Well, I never imagined that it was possible to use MappedSuperclasses other than extending it. Seems like inheritance mapping allows to use it in discriminator map. You are the first probably to use it this way without actually extending it. You can submit a pull request for this change to 2.4.x branch, did not thought doctrine allows such combination

As it turns out, my mapping was "wrong". My base class cannot be a mapped superclass, Doctrine doesn't allow inheritance mapping there. My leaf classes were mapped superclasses, but that was actually a mistake by me. When I converted them to entities instead of mapped superclasses, everything worked fine.
I did actually look at the source code. Removing the mentioned check caused some unit tests for Tree to error. Since everything can be solved by adjusting my mapping, I suggest we defer or ignore this issue.

yes I agree, because it is not a recommended way anyway to do the mapping. closing


Fix SyntaxError error for non-utf8 coded files

There is a bug in Python 3.4, 3.5, and 3.6 which generates a SyntaxError when the coding declaration in the header (e.g. # -*- coding: latin-1 -*-) is read by the Python interpreter.  The issue occurs when the files have unix (line-feed) line endings and the coding is not "utf-8".  It seems to be related to Python issues 20731, 20844, 27797, and 32809.  This PR provides a workaround to that issue.

Would it be better to convert these to utf-8?  They are all Python 3 test files, so the source encoding won't matter as much - they will be converted properly.
I'd prefer to do this, but your other PR for automated testing relies on this so I don't want to make changes until we discuss it.  Thanks.

Yes, converting the files to utf-8 would be better and I have done so.
I will update PR 697 to align with this change.
(By the way, @mkleehammer, great to have you back!)


Qptuna models

Dear developers,
I was wondering if it is already possible of utilizing the Qptuna models from here: https://github.com/MolecularAI/Qptuna .
I was also wondering that if I were to use staged learning (RL in particular) using the output of transfer learning. How should I implement the agent?
I think it should be like this, if I had an output of TL.model from the TL step, but I am not really sure of it:
[parameters]
Reinvent
prior_file = "priors/reinvent.prior"
agent_file = "TL.model"
Thanks in advance,
Wout

Hi,
yes, a Qptuna scoring component is supported as mentioned in Table 2 of our paper.
The agent and prior in a fresh RL start would be the same, in your case the model after TL.  In subsequent stages (staged learning) the agent would be the last checkpoint (chkpt_file) you have created.
Cheers,
Hannes.

So if I understand correctly both my prior and agent should be the same after the transfer learning mode? So it becomes more like this:
Reinvent
prior_file = "TL.model"
agent_file = "TL.model"

Yes.  The prior is used as a reference or regularization term if you will.  Its network is unaffected by RL.  The agent network undergoes parameter changes as per RL reward strategy.

Okay, thank you!


isEmpty function bug

isEmpty function is not work for number.
Expected return false
Actual return true
Code from Git

Actual bundled code from npm package

Online Testing result

It has already been solved with commit 6d42fc7a107b6e47784dd90e5395a2697c664093
I can confirm that the version on the repo works as intended
import { isEmpty } from "radash";

console.log(isEmpty(0)); // true
console.log(isEmpty(1)); // false

The version on NPM has not yet been update though

Thanks for bringing this up with all the details @5kahoisaac ! It has been fixed on the main branch but not yet published (as @matteosacchetto pointed out), will be published this weekend with a lot of other little fixes.


feat: add provisional, raw_rand, etc to mgmt canister

This adds deposit_cycles, provisional_create_canister_with_cycles, provisional_top_up_canister, raw_rand, and updates canister_status to return a record

hmm, why is the if-ref/test (linux-stable) checking out to release-0.14
  /usr/bin/git checkout --progress --force -B release-0.14 refs/remotes/origin/release-0.14
  Switched to a new branch 'release-0.14'

when .github/workflows/ic-ref.yml has been updated to spec: '0.14.0'

Because that's the pull_request_target CI job which doesn't take any changes from .github. We do that for cloned repos, and probably should disable it when the repo is dfinity/agent-rs.


Update Set-SafeLinksPolicy.md

Updating documentation to reflect recent change in cmdlet params.
Rename: IsEnabled to EnableSafelinksForEmail
Remove duplicate parameters for: DoNotTrackClicks and DoNotAllowClickThrough in SafelinksPolicy
TrackClicks and AllowClickThrough: will not be changed or updated

Docs Build status updates of commit e918395:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

@JohanFreelancer9

@joscurtis, how prevalent are these changes? Just yesterday, I dumped all available cmdlets/parameters in an E5 tenant, and those parameters you removed are still available to me. And, the EnableSafeLinksForEmail parameter is not available to me.

Changes have been checked in and are deploying. They will reach SDF environment by Monday 03/14 and 100% WW by Friday 03/18.

From: Chris Davis @.>
Sent: Thursday, March 10, 2022 12:30 PM
To: MicrosoftDocs/office-docs-powershell @.>
Cc: Josh Curtis @.>; Mention @.>
Subject: Re: [MicrosoftDocs/office-docs-powershell] Update Set-SafeLinksPolicy.md (PR #9074)
@joscurtishttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjoscurtis&data=04|01|Joshua.Curtis%40microsoft.com|b325e123fee643623c0f08da02bb9c3f|72f988bf86f141af91ab2d7cd011db47|1|0|637825302060089844|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=7A87c2PDdxmFm%2BiR%2BJBZYa%2F7XLiOh%2FrIlGCsUwnjArE%3D&reserved=0, how prevalent are these changes? Just yesterday, I dumped all available cmdlets/parameters in an E5 tenant, and those parameters you removed are still available to me. And, the EnableSafeLinksForEmail parameter is not available to me.
—
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Foffice-docs-powershell%2Fpull%2F9074%23issuecomment-1064316674&data=04|01|Joshua.Curtis%40microsoft.com|b325e123fee643623c0f08da02bb9c3f|72f988bf86f141af91ab2d7cd011db47|1|0|637825302060089844|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=OBRpzaZfWNuzbQTivqahRYWKTkgWg%2BxRZm8plADk7Gg%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FARFTIVAXLOWQOPGSYFPUJNDU7IWRTANCNFSM5QKH4TJQ&data=04|01|Joshua.Curtis%40microsoft.com|b325e123fee643623c0f08da02bb9c3f|72f988bf86f141af91ab2d7cd011db47|1|0|637825302060089844|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=6R%2FaPgo64EYQd7CSkc5QXqJNrxw4YzQB5eQ0kmYgzp0%3D&reserved=0.
Triage notifications on the go with GitHub Mobile for iOShttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04|01|Joshua.Curtis%40microsoft.com|b325e123fee643623c0f08da02bb9c3f|72f988bf86f141af91ab2d7cd011db47|1|0|637825302060089844|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=ITStCHQNKzLiwqNx3SZiA%2FPkbxtuJgzlbUeAZebWZZM%3D&reserved=0 or Androidhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04|01|Joshua.Curtis%40microsoft.com|b325e123fee643623c0f08da02bb9c3f|72f988bf86f141af91ab2d7cd011db47|1|0|637825302060089844|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=lXqTtzXuq0tjJQxd7LWwv04HUXEMyQSdWJuDJSpDQb0%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>

Docs Build status updates of commit dbac79b:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md
:x:Error

Details

exchange/exchange-ps/exchange/index.yml
:warning:Warning

Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md

Line 0, Column 0: [Error: MAML2Yaml_LoadFileFailed]

Failed to load file: W:/ikac-s/exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md:93:(753) '```yaml
Type: Boolean
Parameter Sets: (All)
Ali...'
 Invalid yaml: unknown key Applies to, please try to use PlatyPS validate the content

exchange/exchange-ps/exchange/index.yml

Line 112, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-HostedConnectionFilterPolicy'.
Line 124, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-MalwareFilteringServer'.
Line 125, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-MalwareFilterPolicy'.
Line 126, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-MalwareFilterRule'.
Line 140, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Update-SafeList'.
Line 169, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-ImapSettings'.
Line 170, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-MailboxCalendarConfiguration'.
Line 234, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-OwaVirtualDirectory'.
Line 255, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-DatabaseAvailabilityGroupNetwork'.
Line 257, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Start-DatabaseAvailabilityGroup'.
Line 304, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-SafeLinksPolicy'.
Line 305, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-SafeLinksRule'.
Line 475, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-HybridMailflow'.
Line 609, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-MailboxAutoReplyConfiguration'.
Line 703, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-InboundConnector'.
Line 712, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-TransportAgent'.
Line 713, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-TransportConfig'.
Line 714, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-TransportServer'.
Line 715, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-TransportService'.
Line 716, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-X400AuthoritativeDomain'.
Line 765, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-PublicFolderMigrationRequest'.
Line 766, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-PublicFolderMoveRequest'.
Line 901, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-TransportRule'.
Line 928, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-MailboxAuditBypassAssociation'.
Line 950, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Start-ComplianceSearch'.
Line 999, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-DataClassification'.
Line 1113, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-HoldCompliancePolicy'.
Line 1114, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-HoldComplianceRule'.
Line 1115, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-ManagedContentSettings'.
Line 1116, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-ManagedFolder'.
Line 1121, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-RetentionComplianceRule'.
Line 1260, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-ThrottlingPolicyAssociation'.
Line 1317, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Update-SiteMailbox'.
Line 1350, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.New-UMAutoAttendant'.
Line 1421, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-Group'.
Line 1424, Column 5: [Warning: xref-not-found] Cross reference not found: 'ExchangePowerShell.Set-MailUser'.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit dad6bc8:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 18de605:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 24aba1c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 85f1914:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 439ecab:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Most of the changes are now live, so I'm publishing. Discrepancies:

The IsEnabled parameter is still available. I added it back with the following description:

This parameter has been deprecated and is no longer used.
Use the EnableSafeLinksForEmail parameter instead.

The MakeBuiltInProtection and RecommendedPolicyType parameters were missing. I added them with the same existing "Don't use this" description from New-SfafeLinksPolicy.

Docs Build status updates of commit b289e5a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Set-SafeLinksPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


No write access with ActiveDirectory.ps1

Script seems to run and the csv output seems to be correct.
However script fails to write the data back to ITGlue AD.
Active Directory flex asset configuration file found!
No flexible asset id was found... creating a new flexible asset.
New-ITGlueFlexibleAssets : {"errors":[{"status":401,"title":"Unauthorized","detail":"Unauthorized resource
access","source":{"pointer":"id"}}]}
At C:\jobs\ITGlueAPI\ActiveDirectory.ps1:338 char:41

...         $api__output_data = New-ITGlueFlexibleAssets -data $api__body

                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,New-ITGlueFlexibleAssets

Get-itglueusers and get-itglueorganizations returns valid data.
The org_id and flexible_asset_type_id from config.psd1 have been confirmed to be correct based on the id in the urls in ITGlue (US version)
Note this is a 2016 AD environment.

In another 2016 environment today I was able to successfully update the AD flexible asset type so this can be closed.


Erase all available simulators when prelaunching

This should hopefully resolve issues on CI where the iOS Simulator doesn't boot when switching between Xcode 6 and Xcode 7. /cc @segiddins

:+1: once CI runs