[tool_crash] FileSystemException: Failed to decode data using encoding 'utf-8', null

Command
flutter --no-color run --machine --track-widget-creation --device-id=334075bc --start-paused --dart-define=flutter.inspector.structuredErrors=true lib\main.dart

Steps to Reproduce

...
...
...

Logs
FileSystemException: Failed to decode data using encoding 'utf-8', null
#0      _File._tryDecode (dart:io/file_impl.dart:615:7)
#1      _File.readAsStringSync (dart:io/file_impl.dart:624:7)
#2      ForwardingFile.readAsStringSync (package:file/src/forwarding/forwarding_file.dart:99:16)
#3      ErrorHandlingFile.readAsStringSync.<anonymous closure> (package:flutter_tools/src/base/error_handling_io.dart:218:22)
#4      _runSync (package:flutter_tools/src/base/error_handling_io.dart:590:14)
#5      ErrorHandlingFile.readAsStringSync (package:flutter_tools/src/base/error_handling_io.dart:217:12)
#6      AndroidPlugin._getSupportedEmbeddings (package:flutter_tools/src/platform_plugins.dart:210:53)
#7      AndroidPlugin._supportedEmbeddings (package:flutter_tools/src/platform_plugins.dart:158:49)
#8      AndroidPlugin._supportedEmbeddings (package:flutter_tools/src/platform_plugins.dart)
#9      AndroidPlugin.toMap (package:flutter_tools/src/platform_plugins.dart:152:30)
#10     _extractPlatformMaps (package:flutter_tools/src/flutter_plugins.dart:361:40)
#11     _writeAndroidPluginRegistrant (package:flutter_tools/src/flutter_plugins.dart:375:53)
#12     injectPlugins (package:flutter_tools/src/flutter_plugins.dart:1164:11)
<asynchronous suspension>
#13     FlutterProject.ensureReadyForPlatformSpecificTooling (package:flutter_tools/src/project.dart:378:5)
<asynchronous suspension>
#14     FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:1648:7)
<asynchronous suspension>
#15     FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1350:27)
<asynchronous suspension>
#16     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#17     CommandRunner.runCommand (package:args/command_runner.dart:212:13)
<asynchronous suspension>
#18     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:348:9)

[✓] Flutter (Channel stable, 3.16.5, on Microsoft Windows [版本 10.0.22631.3296], locale zh-CN)
    • Flutter version 3.16.5 on channel stable at D:\flutter SDK\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 78666c8dc5 (3 months ago), 2023-12-19 16:14:14 -0800
    • Engine revision 3f3e560236
    • Dart version 3.2.3
    • DevTools version 2.28.4
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn

[✓] Windows Version (Installed version of Windows is version 10 or higher)

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at D:\androidSDK
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = D:\androidSDK
    • Java binary at: D:\androidStudio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-10027231)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at C:\Users\Administrator\AppData\Local\Google\Chrome\Application\chrome.exe

[✗] Visual Studio - develop Windows apps
    ✗ Visual Studio not installed; this is necessary to develop Windows apps.
      Download at https://visualstudio.microsoft.com/downloads/.
      Please install the "Desktop development with C++" workload, including all of its default components

[!] Android Studio (version unknown)
    • Android Studio at D:\androidStudio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    ✗ Unable to determine Android Studio version.
    • android-studio-dir = D:\androidStudio
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-10027231)

[✓] IntelliJ IDEA Ultimate Edition (version 2019.3)
    • IntelliJ at D:\idea\IntelliJ IDEA 2019.3.5
    • Flutter plugin version 51.0.1
    • Dart plugin version 193.7547

[✓] IntelliJ IDEA Ultimate Edition (version 2023.3)
    • IntelliJ at D:\idea2023\IntelliJ IDEA 2023.3.4
    • Flutter plugin version 78.1.1
    • Dart plugin version 233.13763.5

[✓] Connected device (4 available)
    • V1981A (mobile)   • 334075bc • android-arm64  • Android 12 (API 31)
    • Windows (desktop) • windows  • windows-x64    • Microsoft Windows [版本 10.0.22631.3296]
    • Chrome (web)      • chrome   • web-javascript • Google Chrome 123.0.6312.86
    • Edge (web)        • edge     • web-javascript • Microsoft Edge 123.0.2420.65

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 2 categories.

Flutter Application Metadata
Type: app
Version: 1.0.0+1
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: stable
Creation framework version: 78666c8dc57e9f7548ca9f8dd0740fbf0c658dc9
Plugins
amap_flutter_location-3.0.0
permission_handler-11.3.1
permission_handler_android-12.0.5
permission_handler_apple-9.4.4
permission_handler_html-0.1.1
permission_handler_windows-0.2.1

Closing this as a duplicate of https://github.com/flutter/flutter/issues/65021


Qt support

It looks like that theming is currently not supported for Qt apps, at least outside of KDE. I think one easy way to support it is by generating a theme for gtk2, which is compatible with qt:
qt = {
  enable = true;
  platformTheme = "gtk";
  style.name = "gtk2";
};

Possibly related: #142


Bench and upgrade to zstd 1.4.8

Development Task
Zstd introduce performance improvement since 1.4.7. We need to benchmark and upgrade to the latest version: https://github.com/facebook/zstd/releases We are currently on 1.4.4: https://github.com/tikv/rust-rocksdb/blob/master/librocksdb_sys/Cargo.toml#L15

related PR: https://github.com/tikv/rust-rocksdb/pull/610

done


Missing parentheses Error

I installed the plugin and tried to run Run :checkhealth clipboard-image  but got the following error.
  health#clipboard-image#check
  ========================================================================
    - ERROR: Failed to run healthcheck for "clipboard-image" plugin. Exception:
       function health#check, line 21
       Vim(call):E107: Missing parentheses '(': health#clipboard-image#check()

My environment is below.

NVIM v0.5.1
macOS Monterey 12.6

It's in Japanese, but I'll post a picture.

Do you have any solutions?
thank you

Hello, I'm also receiving this error. Using (NVIM v0.9.0 nightly), macOS Ventura 13.2.1


avoid circular deps caused within api.api.v1

Closes #3691
‼
note that we will need to update fidesplus accordingly if we move urn_registry.py as is proposed here currently!
‼
Description Of Changes
The root cause here is that the __init__.py of our api.api.v1 module brings in a lot of dependencies via ADMIN_ROUTER. this means we need to be judicious when to depend on code within api.api.v1 so as to keep it as self-contained as possible. in this case, the immediate cause of the problem is that postgres_setup.py depends on our connectors module, and our connectors module was bringing in api.api.v1 via some utilities (namely consent_util which brought in endpoint.utils.)
it seems like an unambiguously good update to move our api.api.v1.endpoint.utils up a few levels into api.util.endpoint_utils, alongside other API related utilities that are not only used by code within api.api.v1, per the general principle above. i've made that change in https://github.com/ethyca/fides/pull/3692/commits/5bee96312ab1331df7337db1d9c42f9095454d02, and that does help us resolve the initial error here.
but we quickly hit another error (stack trace below), this time because our fides_connector (specifically the fides_client it is highly dependent on), which is also initialized by the connectors module, relies directly on the urn_registry that's within api.api.v1. the solution to this feels less straightforward to me -- i see 3 potential paths forward but none seem like an obvious best choice:

we could move urn_registry out of api.api.v1, ~but that doesn't feel right, because it's pretty strictly related/coupled to the code in that module~ -- @ThomasLaPiana also suggested this, and given some more thought, this feels like it might be the right path forward. my proposed change here is to move it to a common.api.v1 module/directory, so that it's clear it refers to our api.v1 endpoints/code.
we could update the __init__.py of api.api.v1 to not bring in so many dependencies via the ADMIN_ROUTER, but this feels like a tall order and larger scope than what we're doing here. i'm not even sure it's desirable, it may lead to other problems.
we could move fides_connector (and fides_client) out of the connectors module, or at least not have it initialized as part of the module's __init__.py, so that it's not initialized with postgres_setup.py. this feels simpler and better, it's not crazy to have the fides_connector special-cased, since it's a pretty special connector in how self-referential it is. but where's the better spot for it so that we're sure it gets properly initialized on normal app runtime/startup, but it's not causing dep issues?
we could update fides_client to not rely on the urn_registry and instead just use string literals for its URL/URN references. this would be simple, but it feels like it makes that code materially worse, and it just doesn't feel like the right solution here - constants are good :)

new error with the update contained in https://github.com/ethyca/fides/pull/3692/commits/5bee96312ab1331df7337db1d9c42f9095454d02:
Traceback (most recent call last):
  File "/fides/tests/ops/integration_tests/setup_scripts/postgres_setup.py", line 15, in <module>
    from fides.api.service.connectors.sql_connector import PostgreSQLConnector
  File "/fides/src/fides/api/service/connectors/__init__.py", line 23, in <module>
    from fides.api.service.connectors.fides_connector import (
  File "/fides/src/fides/api/service/connectors/fides_connector.py", line 19, in <module>
    from fides.api.service.connectors.fides.fides_client import FidesClient
  File "/fides/src/fides/api/service/connectors/fides/fides_client.py", line 9, in <module>
    from fides.api.api.v1 import urn_registry as urls
  File "/fides/src/fides/api/api/v1/__init__.py", line 8, in <module>
    from .endpoints.admin import ADMIN_ROUTER
  File "/fides/src/fides/api/api/v1/endpoints/admin.py", line 7, in <module>
    from fides.api.db.database import configure_db, reset_db
  File "/fides/src/fides/api/db/database.py", line 17, in <module>
    from fides.api.db.seed import load_default_resources, load_samples
  File "/fides/src/fides/api/db/seed.py", line 12, in <module>
    from fides.api.api.v1.endpoints.saas_config_endpoints import (
  File "/fides/src/fides/api/api/v1/endpoints/saas_config_endpoints.py", line 58, in <module>
    from fides.api.util.connection_util import validate_secrets
  File "/fides/src/fides/api/util/connection_util.py", line 40, in <module>
    from fides.api.service.privacy_request.request_runner_service import (
  File "/fides/src/fides/api/service/privacy_request/request_runner_service.py", line 25, in <module>
    from fides.api.graph.analytics_events import (
  File "/fides/src/fides/api/graph/analytics_events.py", line 16, in <module>
    from fides.api.task.task_resources import TaskResources
  File "/fides/src/fides/api/task/task_resources.py", line 17, in <module>
    from fides.api.service.connectors import (
ImportError: cannot import name 'BigQueryConnector' from partially initialized module 'fides.api.service.connectors' (most likely due to a circular import) (/fides/src/fides/api/service/connectors/__init__.py)

Code Changes

[x] moved api.api.v1.endpoint.utils.py to api.util.endpoint_utils.py and updated references
[x] figure out something for above problem with fides_connector.py / fides_client.py when running postgres_setup.py --> proposed update here is to move urn_registry.py to common.api.v1 rather than api.api.v1

Steps to Confirm

[x] nox -s quickstart and nox -s dev -- shell postgres should both work fine
[x] general smoke testing of application with fides_env(test) (just given the number of files touched) - privacy request execution and system management still worked fine.

Pre-Merge Checklist

[ ] All CI Pipelines Succeeded
Documentation:

[ ] documentation complete, PR opened in fidesdocs
[ ] documentation issue created in fidesdocs

[x] Issue Requirements are Met
[ ] Relevant Follow-Up Issues Created
[x] Update CHANGELOG.md
[ ] For API changes, the Postman collection has been updated

@ThomasLaPiana @pattisdr @galvana i've made some progress but my brain's a bit stuck here as i wrap up my day - wrote up where i am and the problem as i see it.
maybe there's a totally different approach here that's either simpler or preferred - i went down this path and didn't really look back!

@ThomasLaPiana @pattisdr @galvana i've made some progress but my brain's a bit stuck here as i wrap up my day - wrote up where i am and the problem as i see it.
maybe there's a totally different approach here that's either simpler or preferred - i went down this path and didn't really look back!

proposed update is to move urn_registry.py to common.api.v1 rather than api.api.v1

I'm just saying this out loud...the majority of these issues were caused specifically by the admin router due to it's need to directly modify the database. While we probably could have avoided most of these changes by directly moving that endpoint into main.py or something, I still feel these refactors still actually make the code more navigable 🙂
Thanks for the changes here @adamsachs !

I'm just saying this out loud...the majority of these issues were caused specifically by the admin router due to it's need to directly modify the database. While we probably could have avoided most of these changes by directly moving that endpoint into main.py or something (like 4 PRs ago, haha), I still feel these refactors meaningfully contribute to the code being more navigable 🙂
Thanks for the changes here @adamsachs !

ha - i also made that observation @ThomasLaPiana , thank you for calling it out here! i felt a bit hesitant to move it and "special-case" the endpoint for lack of consistency, but i think you're right to make a case for it. i will say that even that router imports from from fides.api.api.v1.endpoints import API_PREFIX so we'd need some further adjustments along the lines of what i've done in this PR, but it does feel like a good change to simplify the dependency tree longer-term!


Incorrect sha256 in README.md

This is a great project. Thanks for putting this together.
I believe that the sha256 value for 0.0.7 which is listed in the README.md is incorrect. I believe that the actual value is bf7ecd386740328f96c343dca095a63b93df7f86f8d3e1e2e6ff46e400880077. The corrected
http_archive(
    name = "mypy_integration",
    sha256 = "bf7ecd386740328f96c343dca095a63b93df7f86f8d3e1e2e6ff46e400880077",  # for 0.0.7
    strip_prefix = "bazel-mypy-integration-{version}".format(version = mypy_integration_version),
    url = "https://github.com/thundergolfer/bazel-mypy-integration/archive/{version}.zip".format(
        version = mypy_integration_version,
    ),
)

Thanks for pointing this out, I'll give it a look now.

Thanks 🙂


Feature/form submit handlers

Adds support for onSubmitError  and onSubmitSuccess handlers to the ResetPasswordForm and the ChangePasswordForm.
Fixes #168

Thanks @Tweety-FER , I can confirm that I can now set those handlers.  I did notice an inconsistency that we should handle.  In our docs we say that the handlers can be used to pass a different error back, but that's not true in this PR, it just passes back the original error because it's bound.
I'm referring to this example in docs/api.md:
class LoginPage extends React.Component {
  onFormSubmit(e, next) {
    // e.data will contain the data mapped from your form.
    console.log("Form submitted", e.data);

    // To return an error message, call next() as:
    // next(new Error('Something in the form is wrong.'));

    // Or if you want to change the data being sent, call it as:
    // next(null, { myNewData: '123' });

    // If you call next without any arguments,
    // it will simply proceed processing the form.
    next();
  }

  render() {
    return <LoginForm onSubmit={this.onFormSubmit.bind(this)} />;
  }
}

@robertjd Right you are! Changed the behaviour to use the passed error if it exists, otherwise it uses the old error.


High Availability - Critical - Provider High Availability

If a provider connector fails after receiving a request, a transfer process needs to be picked up by another provider connector instance. This is specially important when deploying to Kubernetes, since pods can be restarted at any time (e.g. VM autoscale).
A TransferProcessStore backed by an external storage is required to achieve this. In addition a transfer process leasing mechanism to orchestrate the assignment of transfer processes to provider connector instances needs to be in place as well.
A CosmosDB-based solution is partially implemented in EDC. This solution requires two stored procedures which available in test resources but not packaged for runtime (#1, #2). Finally this implementation is missing lease expiration logic to allow a leased transfer process to be taken by another provider connector instance after a timeout.
Suggested fixes and improvements:

Provide working implementation of the Cosmos stored procedures in a source folder together with terraform files for deployment.
Implement lease expiration logic to allow a second provider connector instance to take over a transfer process on failure of a first provider connector instance.
Provide a working sample for this solution.

the lease expiration has been implemented, it just has not been ported to the TPS yet.

@marcgs this would be a great hardening task


Accessing localstorage via try/catch

Accessing localstorage in IE and Edge can crash the whole application:

The recommended way is to always access localstorage via try/catch;
This problem will fix issue #51.

Thanks for this. The next version of this lib will use session storage by default. This means, localStorage will be an opt-in.


Update to rustler 0.22

I used the PR #12 from @scrogson as a starting point and updated the package to rustler 0.22. Even if the PR is not going to be merged, maybe some other can use it.

Thanks for the PR! :tada:


İs it possible to check if aos is already initialized?

This is:

Question

Specifications

AOS version:2.3.4

Detailed Description
I'd like to check if aos is already initialized. because i only use it on spesific pages and dont want to reinitialize it again
Possible Solution

You could instead just add a <script> to a JS file with the AOS init instructions but don't add it to your other pages where it isn't necessary.

I happen to have something similar. In a WordPress site I conditionally place the AOS script on pages where it's been used and one general framework.js where all settings are put of all scripts.
What I used to make sure the plugin itself is even callable is this:
// Initialize Animate-On-Scroll
  if (typeof AOS !== 'undefined') {
    AOS.init()
  }

So AOS.init() only runs when the library is available. Hope this helps! Tested with latest version.


New API

HSLuv have a great potential in a design systems, where designer may define base colors or even a set of hue values and the system will take care of rest. To achieve this, we should provide a set of tools for native HSLuv color manipulation.
The first commit contains mocks for future API which I want to discuss (second hardest thing in a computer science). In general it looks like:
@use "hsluv/tools" as hsluv;
@import "hsluv/hsluv";

button {
  $baseColor: hsluv.color(10deg, 80%, 60%);
  $backgroundColor: hsluv.contrast-darken($baseColor, 'AA');
  $borderColor: hsluv.rotate($backgroundColor, 10deg);

  color: hsluv($baseColor);
  background-color: hsluv($backgroundColor);
  border: 1px solid hsluv($borderColor);
}

I still not sure if it's a good idea to have @use and @import in one file, maybe we may consider to add hsluv.to-hsluv() and hsluv.to-hpluv() functions to be able to just @use. Naming of hsluv.color() can be confusing as for me, an alternative is something like hsluv.create()
Also, color contrast functions accepts both numeric ratio and AA and AAA values which will contain a small reduction to satisfy automated color contrast checkers (some colors after conversion to RGB may produce a little bit lower contrast ratio)
@boronine @apexskier what do you think?

Hi @Ptico sorry for the delay.
I just looked at the Sass color module here:
https://sass-lang.com/documentation/modules/color
I'm not really impressed with their API, there's too many functions that are poorly defined. Some things just don't make sense, what happens when you adjust lightness to be above 100% or below 0%? Presumably it clips.. The API suggests using scale to adjust by percentages, but this seems even worse, for example if you scale lightness by 50% for two different colors you might have a huge change in one color and a tiny change in another color, so what is 50% supposed to mean intuitively? At this point it seems easier to just use a color picker.
I think we should make a smaller, lower level API that gives users an opportunity to do EXACTLY what they want, relying on the sass:math module. Once we have that we can consider adding high-level helper functions.
At minimum, it seems that we need functions to extract HSLuv values from an existing color, e.g. their official brand color:

get-hue(#abc123) -> deg
get-saturation(#abc123) -> %
get-lightness(#abc123) -> %
get-hpluv-saturation(#abc123) -> % (this only works with a subset of colors, we can probably use @error to deal with this)

(Or perhaps just implement get-hsluv and get-hpluv functions instead?)
The user can then use sass:math to manipulate these values and plug them back into the hsluv contructor to get new colors.
For contrast, the minimum implementation would be something like this:

get-contrast-ratio($lightness1 %, $lightness2 %)
get-darker($lightness %, $ratio) -> % (throws error when out of bounds)
get-lighter($lightness %, $ratio) -> % (throws error when out of bounds)

Along with these functions we can provide documentation with some minimum contrast ratios defined by some standards (AA, AAA etc.). I don't see much value in adding these to the API since these are only minimum values, not exact targets. Or maybe provide them as constants?
I don't think we should expose an internal representation of HSLuv to the public API unless we really have to. If we can get away with using native Sass colors everywhere it would be simpler for the user.
What are your thoughts? There's a lot going on in this discussion, so I'd be happy to focus on the minimal merge-able implementation here and delay some stuff (e.g. contrast) until the next PR.

I agree with minimizing an API, though it requires some work on documentation to show users an examples of how to work with sass:math and hsluv
For the rest of new API, the main idea is to work with hsluv as a first class sitizen. It will be useful for designers who prefer to use HSLuv as a main color space, as well as for principal frontend engineers who will build a design systems on top of it.
An internal representation is nothing more than (h:, s:, l:) map. The problem with a SASS color is that we have to round an RGB values to the nearest 0-255 whole number and each conversion from/to RGB loses the color information, so if we define a base color, then do a lot of transformations through the styles, we may have pretty unexpected results.
Also, manipulating an rgb colors with this library makes no sense for me, it's just duplicating what we already have with sass:color and almost doesn't add any value. However, we can always translate an existing sass color with hsluv.create(#123abc)

I'm very out of date on SASS syntax, but I have a question about your example:
Why do you need the hsluv() "calls" when using the color variables? I'd expect:
button {
  $baseColor: hsluv(10deg, 80%, 60%);
  $backgroundColor: hsluv.contrast-darken($baseColor, 'AA');
  $borderColor: hsluv.rotate($backgroundColor, 10deg);

  color: $baseColor;
  background-color: $backgroundColor;
  border: 1px solid $borderColor;
}

As is, I see two problems:

I think people would commonly forget to convert back to a sass color (creating broken css)
The utilities can't be used with other color representations (because they expect the internal ('h', 's', 'l') representation) (hsluv.contrast-darken(#ff0000, 'AA'); won't work)

I'd prefer if the hsluv.color/hsluv.create function and ('h', 's', 'l') representation was internal/private to the hsluv implementation. (This assumes it can translate a value of any sass color representation, as implied in @Ptico's last comment.)
e.g.
@function get-hue($color) {
  $raw-hsluv: create($color);
  @return map-get($raw-hsluv, 'h');
}
// usage: hsluv.get-hue(#ff0000)

@function contrast($color, $op: 'darken', $ratio: 'AA') {
  $raw-hsluv: create($color);
  $h: map-get($raw-hsluv, 'h');
  $s: map-get($raw-hsluv, 's');
  $l: map-get($raw-hsluv, 'l');
  // ...
  @return hsluv($raw-hsluv)
}

I agree with much being said, but I'm a little overwhelmed by how many issues are discussed simultaneously. I'm going to try to summarize our discussion here. Seems that we have 3 proposals (ignoring function names for now):
A. Non-magic minimal version, 6 functions added
$brandColor: #abc123;
$brandH: hsluv.get-hue($brandColor);
$brandS: hsluv.get-saturation($brandColor);
$brandL: hsluv.get-lightness($brandColor);
$compH: ($brandH + 180deg) % 360;

.banner {
    color: hsluv($brandH, $brandS, $brandL);
    background-color: hsluv($compH, $brandS, hsluv.get-darker($brandL, hsluv.AA));
    border: 1px solid hsluv($compH, $brandS, hsluv.get-darker($brandL, hsluv.AA * 1.5));
}

B. Magic version exporting HSLuv internal representation, ~17 functions added
$brandColor: hsluv.create(#abc123);
$compColor: hsluv.rotate($brandColor, 180deg);

.banner {
    color: hsluv.hex($brandColor);
    background-color: hsluv.hex(hsluv.contrast-darken($compColor, 'AA'));
    border: 1px solid hsluv.hex(hsluv.contrast-darken($compColor, 5.5));
}

C. Magic version hiding HSLuv internal representation, ~16 functions added
$brandColor: #abc123;
$compColor: hsluv.rotate($brandColor, 180deg);

.banner {
    color: $brandColor;
    background-color: hsluv.contrast-darken($compColor, 'AA');
    border: 1px solid hsluv.contrast-darken($compColor, 5.5);
}

My preference is option A, then option C, then option B. I agree with @apexskier's arguments to choose option C over option B. @Ptico raises an issue with option C regarding rounding, but can we not avoid rounding if we use CSS rgb(...) constructor rather than hex?

Hi @boronine @apexskier
My main point actually is to operate HSLuv kind of natively, because I don’t see any value added by this library in operating rgb colors
Would you mind to have a quick text or voice chat somewhere like discord or gitter to better explain an idea?

@Ptico I'd prefer async communication. Can you explain your idea here? I understood your proposal to be option B above. Are you proposing something different? If so can you draft a list of functions you're proposing?

I feels like I have a small language barrier to properly explain an idea, but i'll try:
Let's say, we have a team of designer and frontend developer who wants to create a design system for an upcoming web-app from scratch. Buttons, modals, typography, etc.
The designer have to choose one or two main (brand) colors, then also some special, like for errors, dangerous actions, accents, etc. Every color will have a lot of deviations in terms of lightness and saturation and they might be able to quickly add an additional ones. An RGB always was a bad choice because for the majority of people three numbers in a range from 0 to 255 means nothing unless pasted to color picker. It's hard to mix three colors in your head. It's also hard to say what is the difference between two colors in terms of lightness and saturation.
And here is where HSL-kind of colors shines. I know that the range between 120-140deg is a green(ish) color, 80% is pretty saturated and 50% is bright enough (you know this better than me). So even if we already have some brand colors defined in RGB it's simplier to just convert them to HSLuv once and then communicate with this format instead of rgb.
Then we might create some UI patterns, lets say button. We have picked a main color for button, then we might want to have some accent in a form of darker border and to make it great-looking we want a text to be not just white, but bright shade of main color with an accessible contrast. We also want to describe CSS rules just once instead of duplicating the code for each color but make them looks uniform in terms of visual brightness. Here we come to HSLuv
From the designer's perspective the process might look like this:
Designer installs some HSLuv color picker to his favourite design tool, converts existings colors to HSLuv and draws the UI elements. To create a nice button border he just move the L slider to make it darker, and to draw the danger button he just Cmd+C/Cmd+V the main one and move the H slider somewhere to 10deg. Easy, handy, happy days!
Then, the frontend developer should implement this design in SCSS. The process might looks similar to the designer's one (my suggestion) or the designer have to convert all the base colors back to the RGB (where 10deg, 80%, 50% becomes #de3442) gives them to the frontent developer forcing him to continously extract HSLuv values from RGB and likely converting them back and forth loosing the precision and making designer unhappy (and no, using the rgb() constructor will not help, because it's just slightly different representation of the same values).
So, my point is to give the frontend developers a tool for a native HSLuv flow where designers and developers communicate the same language and perform the same operations. Also, in an HSLuv native flow, internal representation is easier to debug because the developer can see the same values as in designs

I could try to create a real world example and test different approaches there

Designer installs some HSLuv color picker to his favourite design tool, converts existings colors to HSLuv and draws the UI elements. To create a nice button border he just move the L slider to make it darker, and to draw the danger button he just Cmd+C/Cmd+V the main one and move the H slider somewhere to 10deg. Easy, handy, happy days!

Ok I can imagine a tool like that.

Then, the frontend developer should implement this design in SCSS. The process might looks similar to the designer's one (my suggestion) or the designer have to convert all the base colors back to the RGB (where 10deg, 80%, 50% becomes #de3442) gives them to the frontent developer forcing him to continously extract HSLuv values from RGB and likely converting them back and forth loosing the precision and making designer unhappy

I don't understand this part. The designer can provide HSLuv values and the frontend developer can use hsluv-sass to plug them into SCSS, no? Why would the frontend developer need to convert colors back and forth?

my point is to give the frontend developers a tool for a native HSLuv flow where designers and developers communicate the same language and perform the same operations

The current solution to this problem is for designers to use SASS + hsluv-sass and then pass the .scss file to the frontend developer.
Are you suggesting making some kind of app? I'm not opposed to it though I probably won't have time to contribute. If you want to take a shot at it, I'd happily put it under the https://github.com/hsluv org and link to it from the homepage. If you want to discuss it further perhaps the best place is to open an issue under the main repo: https://github.com/hsluv/hsluv/issues
For this comment thread let's go back to the API question. My summary of the current proposals in this comment above: https://github.com/hsluv/hsluv-sass/pull/9#issuecomment-854775270
Please let me know which option you choose or if you have another option, please make a list of functions or a code sample to demonstrate it.

And here is where HSL-kind of colors shines. I know that the range between 120-140deg is a green(ish) color, 80% is pretty saturated and 50% is bright enough (you know this better than me).

Definitely - this is the clear value of HSLuv and the main point of the project.

Then, the frontend developer should implement this design in SCSS. The process might looks similar to the designer's one (my suggestion) or the designer have to convert all the base colors back to the RGB (where 10deg, 80%, 50% becomes #de3442) gives them to the frontent developer forcing him to continously extract HSLuv values from RGB and likely converting them back and forth loosing the precision and making designer unhappy (and no, using the rgb() constructor will not help, because it's just slightly different representation of the same values).

forcing him to continously extract HSLuv values from RGB and likely converting them back and forth

This point doesn't make sense to me, as I think @boronine also mentioned.
If the developer is able to deal with the built-in sass color values, they should never need to extract HSLuv or RGB values. The only place that needs to happen is inside the hsluv module. My concern with the original proposal in the thread is that you are extracting something. It's not as bad as extracting individual h, s, and l number values, but it is extracting the (h, s, l) map value when you could use color values directly.

loosing the precision

I agree, the loss of precision is the main downside to my suggestion, but I personally don't think it's going to be a huge deal. Designing for the web already means designers already need to be aware of many different factors that cause loss of color fidelity - OS/browser handling, different monitors, etc. I think the value of having a color be a color, and usable in a property directly, is high.

Let's say, we have a team of designer and frontend developer who wants to create a design system for an upcoming web-app from scratch. Buttons, modals, typography, etc.

I like this example, but I think we should also consider non-greenfield projects, which are very common.
Say a team embedded in a large company wants to start using HSLuv, but needs to embed within an existing SASS library where they have a large set of colors already defined as hex colors (e.g. $primary: #212e61. It'd be nice for them to do something like $primary-shadow: hsluv.darken($primary, 0.2) instead of $primary-shadow(hsluv.create($primary), 0.2).
For both cases, I think introducing a new datatype is going to cause unnecessary boilerplate and verboseness as a codebase grows due to the need to convert when declaring, using, and potentially modifying existing colors.

By the tool I mean hsluv-sass itself

I don't understand this part. The designer can provide HSLuv values and the frontend developer can use hsluv-sass to plug them into SCSS, no? Why would the frontend developer need to convert colors back and forth?

Lets take a look at our examples in a context of HSLuv native flow:
The designer gives us a "danger" color: 10deg, 80%, 50%
Option A:
$dangerH: 10deg;
$dangerS: 80%;
$dangerL: 50%;
$dangerTextL: hsluv.contrast-lighten($dangerL, hsluv.AA);
$dangerBorderL: $dangerL - 20;

.dangerButton {
  background-color: hsluv($dangerH, $dangerS, $dangerL);
  color: hsluv($dangerH, $dangerS, $dangerTextL);
  border-color: hsluv($dangerH, $dangerS, $dangerBorderL);
}

No need for RGB conversions of course, but imagine having 10 different HSLuv colors.
Option C:
$dangerColor: #de3442;

.dangerButton {
  background-color: $dangerColor;
  color: hsluv.contrast-lighten($dangerColor, hsluv.AA);
  border-color: hsluv.darken($dangerColor, 0.2);

Here we:
a) Forced to operate with SASS colors, so the designer would have to convert everything to RGB
b) The only value hsluv-sass adds here is a color-contrast functions, because other operations may be done with sass:color and besides accessability there is no need for HSLuv at all
Option B
$dangerColor: hsluv.create(10deg, 80%, 50%);

.dangerButton {
  background-color: hsluv($dangerColor);
  color: hsluv(hsluv.contrast-lighten($dangerColor, hsluv.AA));
  border-color: hsluv(hsluv.darken($dangerColor, 20%));
}

There is a downside of requirement to apply hsluv() every time we need to assign an HSLuv color to actual CSS, but for a much larger codebases IMO it's much easier to operate with HSLuv colors instead of separate HSLuv values or RGB colors

For option C, this is what I'm suggesting:
$dangerColor: hsluv.create(10deg, 80%, 50%);

.dangerButton {
  background-color: $dangerColor;
  color: hsluv.contrast-lighten($dangerColor, hsluv.AA);
  border-color: hsluv.darken($dangerColor, 0.2);
}

No need to convert to RBG.

b) The only value hsluv-sass adds here is a color-contrast functions, because other operations may be done with sass:color and besides accessability there is no need for HSLuv at all

HSLuv adds additional value in that its functions operate on a different geometry than sass:color. I also think interoperability with sass colors is a big benefit.

I like this example, but I think we should also consider non-greenfield projects, which are very common

Non-greenfield projects are common, the partial usage of HSLuv in them is not, but anyway we already have an API for this kind of usage.
Think of new (optional) API as of tool for the new projects which are going to use HSLuv in all stages of a design pipeline and would not mix SASS colors with HSLuv.

No need to convert to RBG.

We will need to implement native API internally anyway, would it work if we will let users choose which one they will use?
Like:
@use 'hsluv/native'; // For native operations
//or
@use 'hsluv/tools'; // For SASS color operations

It will not require too much effort because hsluv/native will be a backend for hsluv-tools

I like that idea

I think the value offered by saving a few keystrokes will be destroyed by wasting time reading though this bloated API, trying to understand what "native" means, trying to understand what happens when you go out of bounds, or getting compilation errors and giving up on hsluv-sass completely. Instead of giving the user 30 different wrappers for hsluv function, it would be better to just explain to the user the meaning of H, S, and L. After they understand it, they never have to reference the documentation again.
I did some tests trying to use rgb and rgba to avoid rounding, but Sass always rounds these to integers so these formats are indeed equivalent to hex. However the difference between two off-by-one hex colors is imperceptible, so I believe rounding is a non-issue in practice. Unless we can come up with a practical scenario where rounding is problem, all "native" API options should be rejected, leaving us with minimal API (option A) and sass:color clone (option B).

Ok, here is another suggestion: let’s have a minimal set of tools, like color contrast and component extraction and collection of helpers in an external, third-party package. Would it be fair to mention it in a readme?

@Ptico before we consider making another repo, why don't we take stock of the discussion.
You mention 3 things:

color contrast
component extraction (rgb -> hsluv)
helper functions (sass:color clone)

I think all of us in this discussion agree that 1 and 2 would add value to this repo. Am I right? I propose we implement those first, release them as v2.1.0, then open another issue for helper functions, summarizing the relevant discussion from this thread. Let's make incremental progress.
At the end of the day I think @apexskier should make the call as the original creator of this port. I will merely provide my opinions.

@boronine yeah, that’s what I meant. Also, I’m thinking about to update my company’s design system and then make another proposal based on a code extraction from the real usecase

@Ptico ah okay, I misread your comment. Great that we agree!

I’m thinking about to update my company’s design system and then make another proposal based on a code extraction from the real usecase

This sounds like a great idea!

Have there been any updates on this pull request? I am very interested in having more SASS hsluv methods to work with in generating color schemes for my projects, particularly the ability to create an hsluv color object and HSL properties from a HEX color. Cool project!

@badfeather I think RGB/hex -> HSLuv conversion can be valuable to expose. When generating palettes from scratch there is not much need for it, but when working around a color specified elsewhere, e.g. a brand color, it may be useful. I would make a separate issue for it.

@boronine Yes, exactly. It would be very useful to take a brand's colors and convert them to HSLuv colors in order to generate shades. I will create a separate issue.


cloud provider Amazon option - cluster creation fail

What kind of request is this (question/bug/enhancement/feature request):
bug?
Steps to reproduce (least amount of steps as possible):
Create a new ec2 cluster and use cloud provider = "Amazon"
Result:
Error:
This cluster is currently Provisioning; areas that interact directly with it will not be available until the API is ready.
[workerPlane] Failed to bring up Worker Plane: Failed to verify healthcheck: Failed to check https://localhost:10250/healthz for service [kubelet] on host [<IP_ADDRESS>]: Get https://localhost:10250/healthz: dial tcp <IP_ADDRESS>:10250: connect: connection refused, log: F1226 18:08:56.949847 31560 server.go:262] failed to run Kubelet: could not init cloud provider "aws": ClusterID tags did not match: "c-7cjkr" vs "dev01"
Other details that may be helpful:
-> If I edit the cluster and choose the option none in cloud provider the cluster creation works
-> Rancher instance profile role is aws admin, so I think its is not aws permission issue
Environment information
HA Installation
Component	Version
Rancher	v2.1.3
User Interface	v2.1.17
Helm	v2.10.0-rancher5
Machine	v0.15.0-rancher1-1
Cluster information
1 - Master
Active | tmaster1|<IP_ADDRESS> | etcd Control Plane | v1.12.3 17.3.2 | 0.3/2 Cores | 0/7.4 GiB | 2/110 |
1 - worker
Active|tworker1|<IP_ADDRESS>|Worker	v1.12.3 17.3.2 0.5/2 Cores	0.1/7.4 GiB	8/110

I build a new cluster to get more info from logs:
I1227 20:51:39.837329   23114 aws.go:1025] Building AWS cloudprovider
I1227 20:51:39.837356   23114 aws.go:987] Zone not specified in configuration file; querying AWS metadata service
F1227 20:51:40.216036   23114 server.go:262] failed to run Kubelet: could not init cloud provider "aws": ClusterID tags did not match: "c-xvdtx" vs "dev01"

echo kubelet --v=2 --hostname-override=master1 --cloud-provider=aws --kubeconfig=/etc/kubernetes/ssl/kubecfg-kube-node.yaml --client-ca-file=/etc/kubernetes/ssl/kube-ca.pem --address=0.0.
0.0 --resolv-conf=/etc/resolv.conf --anonymous-auth=false --authentication-token-webhook=true --read-only-port=0 --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bi
n --volume-plugin-dir=/var/lib/kubelet/volumeplugins --pod-infra-container-image=rancher/pause-amd64:3.1 --cluster-dns=<IP_ADDRESS> --fail-swap-on=false --root-dir=/var/lib/kubelet --tls-ciph
er-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES
_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --cadvisor-port=0 --cgroups-per-qos=True --enforce-node-allocatable= --allow-privileged=true --cluster-domain=cluster.local --register-w
ith-taints=node-role.kubernetes.io/controlplane=true:NoSchedule

The prerequisites for Amazon cloud provider are listed here: https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/cloud-providers/#amazon
The log says the ClusterID tags do not match, can you check what ClusterID tag is set on the resources mentioned in the docs?

thanks @superseb ! that is true.....I didnt put the tags in the subnets and security groups in the first try. I fixed it now, but still no success with my current cluster and same error when create a new cluster and enable aws as cloud provider.
NEW CLUSTER c-hkcq4:
[rancher-f4fb5f5c6-vwpcc] 2018/12/28 12:43:02 [INFO] 2018/12/28 12:43:02 http: multiple response.WriteHeader calls
[rancher-f4fb5f5c6-r247s] 2018/12/28 12:42:56 [ERROR] cluster [c-hkcq4] provisioning: [workerPlane] Failed to bring up Worker Plane: Failed to verify healthcheck: Failed to check https://localhost:10250/healthz for service [kubelet] on host [<IP_ADDRESS>]: Get https://localhost:10250/healthz: dial tcp <IP_ADDRESS>:10250: connect: connection refused, log: F1228 12:43:36.748220   18636 server.go:262] failed to run Kubelet: could not init cloud provider "aws": ClusterID tags did not match: "c-hkcq4" vs "dev01"
[rancher-f4fb5f5c6-r247s] 2018/12/28 12:42:56 [ERROR] ClusterController c-hkcq4 [cluster-provisioner-controller] failed with : [workerPlane] Failed to bring up Worker Plane: Failed to verify healthcheck: Failed to check https://localhost:10250/healthz for service [kubelet] on host [<IP_ADDRESS>]: Get https://localhost:10250/healthz: dial tcp <IP_ADDRESS>:10250: connect: connection refused, log: F1228 12:43:36.748220   18636 server.go:262] failed to run Kubelet: could not init cloud provider "aws": ClusterID tags did not match: "c-hkcq4" vs "dev01"
[rancher-f4fb5f5c6-mshdw] 2018/12/28 12:43:57 [INFO] 2018/12/28 12:43:57 http: multiple response.WriteHeader calls
[rancher-f4fb5f5c6-r247s] 2018/12/28 12:43:14 [INFO] 2018/12/28 12:43:14 http: multiple response.WriteHeader calls
[rancher-f4fb5f5c6-r247s] 2018/12/28 12:43:22 [ERROR] ProjectRoleTemplateBindingController p-xjzpv/creator-rt-ffjhv [mgmt-auth-prtb-controller] failed with : roleTemplate.management.cattle.io "rt-ffjhv" not found
[rancher-f4fb5f5c6-r247s] 2018/12/28 12:43:22 [ERROR] ProjectRoleTemplateBindingController p-pqcz8/creator-rt-fzrf6 [mgmt-auth-prtb-controller] failed with : roleTemplate.management.cattle.io "rt-fzrf6" not found
[rancher-f4fb5f5c6-r247s] 2018/12/28 12:43:22 [ERROR] ProjectRoleTemplateBindingController p-dd4xc/creator-rt-fzrf6 [mgmt-auth-prtb-controller] failed with : roleTemplate.management.cattle.io "rt-fzrf6" not found
[rancher-f4fb5f5c6-r247s] 2018/12/28 12:43:25 [INFO] 2018/12/28 12:43:25 http: multiple response.WriteHeader calls
[rancher-f4fb5f5c6-vwpcc] W1228 12:43:43.869114       7 reflector.go:341] github.com/rancher/rancher/vendor/github.com/rancher/norman/controller/generic_controller.go:144: watch of *v3.Node ended with: too old resource version: 5435615 (5439806)
[rancher-f4fb5f5c6-vwpcc] 2018/12/28 12:43:46 [INFO] 2018/12/28 12:43:46 http: multiple response.WriteHeader calls
[rancher-f4fb5f5c6-r247s] W1228 12:43:40.084877       7 reflector.go:341] github.com/rancher/rancher/vendor/github.com/rancher/norman/controller/generic_controller.go:144: watch of *v3.Node ended with: too old resource version: 5439136 (5439834)
[rancher-f4fb5f5c6-mshdw] 2018/12/28 12:44:34 [ERROR] ProjectRoleTemplateBindingController p-xjzpv/creator-rt-ffjhv [cluster-prtb-sync] failed with : couldn't get role template rt-ffjhv: roleTemplate.management.cattle.io "rt-ffjhv" not found
[rancher-f4fb5f5c6-mshdw] 2018/12/28 12:44:34 [ERROR] ProjectRoleTemplateBindingController p-dd4xc/creator-rt-fzrf6 [cluster-prtb-sync] failed with : couldn't get role template rt-fzrf6: roleTemplate.management.cattle.io "rt-fzrf6" not found
[rancher-f4fb5f5c6-mshdw] 2018/12/28 12:44:34 [ERROR] ProjectRoleTemplateBindingController p-pqcz8/creator-rt-fzrf6 [cluster-prtb-sync] failed with : couldn't get role template rt-fzrf6: roleTemplate.management.cattle.io "rt-fzrf6" not found
[rancher-f4fb5f5c6-mshdw] 2018/12/28 12:44:40 [INFO] 2018/12/28 12:44:40 http: multiple response.WriteHeader calls
[rancher-f4fb5f5c6-mshdw] 2018/12/28 12:44:51 [INFO] 2018/12/28 12:44:51 http: multiple response.WriteHeader calls
[rancher-f4fb5f5c6-vwpcc] 2018/12/28 12:44:17 [INFO] 2018/12/28 12:44:17 http: multiple response.WriteHeader calls

CURRENT CLUSTER c-5mf6j:
[rancher-f4fb5f5c6-mshdw] W1228 12:52:26.468466       7 reflector.go:341] github.com/rancher/rancher/vendor/github.com/rancher/norman/controller/generic_controller.go:144: watch of *v3.Node ended with: too old resource version: 5440512 (5441678)
[rancher-f4fb5f5c6-r247s] 2018/12/28 12:51:44 [ERROR] ClusterController c-5mf6j [cluster-provisioner-controller] failed with : [workerPlane] Failed to bring up Worker Plane: Failed to verify healthcheck: Failed to check https://localhost:10250/healthz for service [kubelet] on host [<IP_ADDRESS>]: Get https://localhost:10250/healthz: dial tcp <IP_ADDRESS>:10250: connect: connection refused, log: F1228 12:51:59.310500   31898 server.go:262] failed to run Kubelet: could not init cloud provider "aws": ClusterID tags did not match: "c-5mf6j" vs "dev01"
[rancher-f4fb5f5c6-vwpcc] W1228 12:52:02.753388       7 reflector.go:341] github.com/rancher/rancher/vendor/github.com/rancher/norman/controller/generic_controller.go:144: watch of *v1.Endpoints ended with: too old resource version: 5428500 (5429463)

TAGS:
aws ec2 describe-tags --filters "Name=resource-id,Values=xxxxxxxxxxxxxx" "Name=resource-type,Values=subnet"
{
    "Tags": [
        ...
        ...
        {
            "ResourceType": "subnet",
            "ResourceId": "xxxxxxx",
            "Value": "shared",
            "Key": "kubernetes.io/cluster/c-5mf6j"
        },
        {
            "ResourceType": "subnet",
            "ResourceId": "xxxxxxx",
            "Value": "shared",
            "Key": "kubernetes.io/cluster/c-hkcq4"
        }
    ]
}

aws ec2 describe-tags --filters "Name=resource-id,Values=xxxxxxxxxxxxxx" "Name=resource-type,Values=security-group"
{
    "Tags": [
        ...
        ...
        {
            "ResourceType": "security-group",
            "ResourceId": "sg-0892da2dc7fd5031e",
            "Value": "shared",
            "Key": "kubernetes.io/cluster/c-5mf6j"
        },
        {
            "ResourceType": "security-group",
            "ResourceId": "sg-0892da2dc7fd5031e",
            "Value": "shared",
            "Key": "kubernetes.io/cluster/c-hkcq4"
        }
    ]
}

Note: I'll not post here the node tags, because looks fine! and its created by rancher automatically.

When I create new nodes the provision process tag new nodes with:
kubernetes.io/cluster/c-5mf6j -> owned
but should tag with:
kubernetes.io/cluster/dev01 -> owned
What I'm doing wrong? if I tag manually the new nodes with kubernetes.io/cluster/dev01, I'm able to change the cloud provide from none to aws.

After the I changed the cluster provider from none to aws I see that the output from "rancher node list" changed too from "master-us-east-1b-m5-large1" (pool name format) to "ip-10-53-103-111.ec2.internal" (aws format).
is it expected?

When using EC2 node template driver, the tag on the instances is set to kubernetes.io/cluster/<CLUSTERID>=owned. If you want to have more flexibility on this, you can use the custom cluster functionality and manually tag the instances with the correct tags and then add them to the cluster (which can still be configured with the k8s cloud provider amazon to get desired cloud provider functionality).
Let me know if you have any other questions on this.


Add shock-capturing for nonconservative terms in 1D and 3D for the TreeMesh

Right now, shock capturing for nonconservative terms using a blending with FV is not implemented in 1D and 3D (but in 2D).

Shock-capturing for nonconservative terms on TreeMesh has been implemented in both 1D #1133 and 3D #900, so this can be closed.


HOTFIX: use .equals to compare HostInfo in StreamsMetadataState

Hotfix for patching this in older branches

cc @mjsax

Retest this please.

Wow, all tests passed on the first try! cc @mjsax

Merged to 2.5 and cherry-picked to 2.4.
It's fixed in trunk and 2.6 via #8900


New version: FreeParameters v0.1.1

Registering package: FreeParameters
Repository: https://github.com/climate-machine/FreeParameters.jl
Created by: @charleskawczynski
Version: v0.1.1
Commit: e88288e86a4b83f7ccdb5864ba763aea72d21f24
Reviewed by: @charleskawczynski
Reference: https://github.com/climate-machine/FreeParameters.jl/issues/21#issuecomment-584375791
Release notes:

Provides support for working with FreeParameter's interactively.

close


[fix] changes for GitHub actions PR CI for matching oneDAL Nightly-build

Description
Fixes to return CI to green as Nightly-build was changed in oneapi-src/oneDAL#2865 and oneapi-src/oneDAL#2920

Checklist to comply with before moving PR from draft:
PR completeness and readability

[ ] I have reviewed my changes thoroughly before submitting this pull request.
[ ] I have commented my code, particularly in hard-to-understand areas.
[ ] I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
[ ] Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
[ ] I have added a respective label(s) to PR if I have a permission for that.
[ ] I have resolved any merge conflicts that might occur with the base branch.

Testing

[ ] I have run it locally and tested the changes extensively.
[ ] All CI jobs are green or I have provided justification why they aren't.
[ ] I have extended testing suite if new functionality was introduced in this PR.

Performance

[ ] I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
[ ] I have provided justification why performance has changed or why changes are not expected.
[ ] I have provided justification why quality metrics have changed or why changes are not expected.
[ ] I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

Still a problem with kneighbors and stability, this probably will need to be deselected and a ticket made for investigation.

/intelci: run

/intelci: run

/intelci: run


Simplifying Design

Hi, I want to customize and simplify the existing design with removing XVS, XHS, CLK_IN, XMASTER, ADDR0 and becuase I'm gonna use one sensor per device and thinking they are connected to each other but I'm not sure about sometings and have some questions. Also I don't want to use QWIIC. I'm asking questions assuming a few things, please correct me if I am wrong.
1- What XVS and XHS are doing ? If there aren't multiple sensor which need to be synchronized, are they necessary ? And what is the connection between XVS-XHS and Master-Slave ? I assume it will be used to synchronize the master with the slaves.
2- There is 24 MHz built-in oscillator.  When I look at the PCB picture in v1.4, I see that there is no external oscillator connected, so I assume that it can work without an extra clock source. How does it work if an external clock signal is given from the CLK_IN pin ?
3- According to the datasheet, multiple clock sources can be used, 24, 27, 37.125, 72 and 74.25 MHz. What are the advantages and disadvantages of higher clock speeds ? Can we count the FPS increase as one of them or does this only happen when using 8-lane instead of 4-lane ? Also, how many of these other clock speeds can the Raspberry Pi 5 handle ? lf I use a different clock source that the Raspberry Pi 5 can handle, all I need to do is change the part of the driver code related to the clock signal ?
4- What is the Master-Slave situation and what do XMASTER, ADDR0 and ADDR1 pins ?(Also I'm talking about i2c adress table on the board, what is default position, 0x1A) Again assuming that they are related to the use of multiple sensors, are they necessary if I'm gonna use one sensor and if there is no slave, am I the Master ?
5- If I don't use any of these pins I mentioned, will it work ?
I asked some things just to be sure. Thanks in advance for your answers.

XVS and XHS are output or input sync signals, you can remove that.
I'm not quite sure what is your question here, the sensor require clock signal to work, so there is a oscillator on v1.4 board providing 24Mhz signal, you can send external 1.8V clock signal to that pin if you want to remove it.
I don't know, this is more for supporting systems that are already using those crystal in the systems so you can save one oscillator on the board. It does not change the sensor readout speed, the only thing you need to do is adjust the driver code to initialize the sensor with correct PLL settings.
I'm not clear about what are you asking, but these are not necessary for single sensor usage. The master option is for XHS/XVS clock output/input selection, which if not use then you can ignore them.
You can leave XHS,XVS, Master Open and pull the AD0 and AD1 to GND to fix I2C address to 0x1A.

Thank you @will127534, you helped a lot. Maybe my questions were a bit confusing and not clear enough, but I got the answers I needed.
In the 2nd question, what I wanted to ask was what happens when an external oscillator is connected when there is already an internal oscillator. However, this question is just out of curiosity.
In the 4th question, I thought they had a relationship because the XMaster and ADD0-ADD1 pins were positioned close in the schematic, but now I realise that this is wrong.
Thank you again for your answers. I am currently working on a different sensor similar to the IMX585 based on your work.

In the 2nd question, what I wanted to ask was what happens when an external oscillator is connected when there is already an internal oscillator. However, this question is just out of curiosity.

There is a resistor on the board to switch between the two options, so you either do external or internal with the HW.


Improve memory usage and performance for the ioc api

Be able to load classes dinamically and not all at one at the begening for the components.
Profile the api to see how much memory is using and how fast it is, and improve the api in general by creating cache maps for must common tasks.

I detect that the memory consuption was a littel high beacouse of the fields and methods creation when calling the getDeclaredFields() and getDeclaredMethods() in the classes upon dependency injection. it was fix by creating a cache for the components clases, the memory consuption drop down to the half on the test project.


MatrixVT exps

Hi,
replace "bev_depth_lss_r50_256x704_128x128_20e_cbgs_2key_da_ema.py" with "MatrixVT",
after 32 epochs' training,
got AP 0.3243 and NDS 0.4091 on val dataset.
What can i do to improve, Thank you.

depth aggregation is not supported in MatrixVT backbone, so set self.backbone_conf['use_da'] = True didn't work.

Besides, multi-frame (2key) is currently not supported in MatrixVT, sweep idx is set to 0 during VT. see here. You can add multi-frame support into this file.

@ZRandomize
Hi,
Thank you for the great work about MatrixVT.
I am willing to add the functionality of multi-frame (2key) to MatrixVT in order to reproduce the result of the paper of MatrixVT, but I'm not sure how to do it.
Multi-frame (2key) creates two BEV features (current frame and previous one) and concatenates them, and then passes the concatenated feature to a detector. Is my understanding right?

yes, indeed. BTW, there are also RNN-based methods like VideoBEV.

@ZRandomize
Thank you for your quick response!
I'll check VideoBEV, too.


Fix LaTeX warning and R CMD check error for test packages

Resolve the following WARNING from the report.
LaTeX errors when creating PDF version.
This typically indicates Rd problems.
LaTeX errors found:
! Font TS1/cmr/m/n/10=tcrm1000 at 10.0pt not loadable: Metric (TFM) file not fo
und.
<to be read again>
relax
l.18 \item[Title]\AsIs{Hagrid’s giant spider}

Also fix this ERROR:
* checking PDF version of manual without hyperrefs or index ... ERROR

https://gitlab.irstea.fr/in-wop/airGRiwrm/-/issues/53 seems relevant.


Add a JS minifier Step

Right now we don't have any minifiers for JavaScript. minify-html comes with minify-js, but minify-js is known to have correctness issues: https://github.com/privatenumber/minification-benchmarks.

github.com/tdewolff/minify is built in Go but comes with Python bindings, but lacks wheels in its CI. Best play may be to make a pull request to add wheels, since the repository is actively maintained.

Another option could be to make PyO3 bindings to https://github.com/swc-project/swc; however, this project is much broader than our need, and could be a huge challenge to wrap for our needs.

See https://github.com/tdewolff/minify/issues/599 and https://github.com/tdewolff/minify/pull/600. Looks like Golang is very difficult to bind to Python, so may consider working on swc bindings with maturin.

We should be able to move forward with tdewolff-minify; though wheels won't be available for every platform, the most important by far will be covered along with several others. Mac users may need to use Docker or a VM as a workaround.


Stop using signals to cache objects

Using the signal handlers to cache objects and clear the cache is way too complicated and is even worse when you want to cache misses. It's also very hard to reason about and has been a source of some pain in the past.
The strategy I've been using internally lately is much, much simpler:
@classmethod
def get(cls, name):
    key = cls._cache_key(name)
    obj = cache.get(key)
    if obj == EMPTY:  # Some sentinel value
        return None  # or raise cls.DoesNotExist
    if obj:
        return obj
    try:
        obj = cls.objects.get(name=name)
        cache.add(key, obj)
    except cls.DoesNotExist:
        cache.add(key, EMPTY)
        return None  # or raise
    return obj

    def save(self, *a, **kw):
        cache.delete(self._cache_key(self.name))
        # ...

This works great when, like in waffle, most access is by one particular attribute, in this case the names. This doesn't impact the user API at all, only the internals of some methods.
It's a little more complex with the sub-keys for user and group lists on flags, but I think by being clever with the _cache_key and including the modified datetime, we can make that invalidation automatic and easier.

I feel bad about this, but here's what I'm about to do.
In Flag.get(), what's actually going to get cached is an augmented copy of the instance .__dict__ with the IDs from the two many-to-many relationships as lists (e.g. _user_ids and _group_ids). Then the restore-from-cache path is f = Flag(); f.__dict__.update(cached_dict), and two new properties (e.g. Flag.user_ids) get added.
Then, for the user/group checks, we only need check that the user ID is in the flag.user_ids list, or that the intersection of the user's group IDs and flag.group_ids is not empty. I'll do something here to make sure that when we check multiple flags, we don't select the user group IDs every time.
The "all" caches will cache lists of these dicts, instead of whatever they have now.
Finally, signal handlers will be set up on post_save, pre_delete, post_delete, and m2m_changed, for the following:

post_save: update the instance cache and the all cache
pre_delete: update the instance cache with the EMPTY value
post_delete: update the all cache
m2m_changed: update the instance cache and the all cache

The assumption is that changing flags is very rare compared to checking them, so we'd much rather incur these queries predictably on change, instead of on access.
If any of that sends people screaming for the hills... Or if it seems I've forgotten anything here, let me know.

I'll also update the admin to not use the queryset delete—or to at least overwrite all the caches if it does.

The assumption is that changing flags is very rare compared to checking them, so we'd much rather incur these queries predictably on change, instead of on access.

+1

Is there any update on this feature? When this feature is complete the 0.10.2 should be released?


Update to use Sprockets 3.0

3.0 is still in beta. https://rubygems.org/gems/sprockets/versions

Okay. we will wait!

Sprockets 3 seems to've been released.

Done.


Thanks though


(agent-scheduler): Enable exactOptionalPropertyTypes

This PR would introduce a breaking change and versions 2.1+ do not permit any breaking changes. We may revisit this PR when breaking changes are allowed.


Source containers

Merging feature branch into master
Maintainers will complete the following section:

[ ] Commit messages are descriptive enough
[ ] "Signed-off-by:" line is present in each commit
[ ] Code coverage from testing does not decrease and new code is covered
[ ] JSON/YAML configuration changes are updated in the relevant schema
[ ] Changes to metadata also update the documentation for the metadata
[ ] Pull request includes link to an osbs-docs PR for user documentation updates

Release notes:

new API method is created: create_source_container_build. This method triggers source container image builds which results into images which contains SRPMs from given koji N-V-R


ERROR: Mob_typer return code 1

Hello!
I'm running mob_recon on a Salmonella incomplete assembly:
mob_recon --infile ~/suganda_data/assemblies/contigs/SRR1556083_contigs_renamed.fasta --outdir ~/suganda_data/mob_suite/test -d /home/johnsont/public/mob_suite_dbs --run_typer

Without the --run_typer flag, everything seems to run perfectly. With the --run_typer flag, I receive the following error:

Traceback (most recent call last):
File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 4410, in get_value
return libindex.get_value_at(s, key)

File "pandas/_libs/index.pyx", line 44, in pandas._libs.index.get_value_at
File "pandas/_libs/index.pyx", line 45, in pandas._libs.index.get_value_at
File "pandas/_libs/util.pxd", line 98, in pandas._libs.util.get_value_at
File "pandas/_libs/util.pxd", line 83, in pandas._libs.util.validate_indexer
TypeError: 'str' object cannot be interpreted as an integer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/panfs/roc/groups/1/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/mob_suite/mob_typer.py", line 513, in
main()

File "/panfs/roc/groups/1/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/mob_suite/mob_typer.py", line 373, in main
input_seq = args.infile )

File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/mob_suite/mob_host_range.py", line 89, in getLiteratureBasedHostRange
repliconsearchdict = findHitsInLiteratureDBbyReplicon(replicon_names,plasmid_lit_db)

File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/mob_suite/mob_host_range.py", line 59, in findHitsInLiteratureDBbyReplicon
db_hit_indices=[i for i in range(0, plasmid_lit_db.shape[0]) if plasmid_lit_db.iloc[i, :]["Replicon"] == replicon_name]

File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/mob_suite/mob_host_range.py", line 59, in
db_hit_indices=[i for i in range(0, plasmid_lit_db.shape[0]) if plasmid_lit_db.iloc[i, :]["Replicon"] == replicon_name]

File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/pandas/core/series.py", line 871, in getitem
result = self.index.get_value(self, key)

File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 4418, in get_value
raise e1

File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 4404, in get_value
return self._engine.get_value(s, k, tz=getattr(series.dtype, "tz", None))

File "pandas/_libs/index.pyx", line 80, in pandas._libs.index.IndexEngine.get_value
File "pandas/_libs/index.pyx", line 90, in pandas._libs.index.IndexEngine.get_value
File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Replicon'
2020-05-28 13:37:13,910 ERROR: Mob_typer return code 1 [in /home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/mob_suite/mob_recon.py:193]
Traceback (most recent call last):
File "/home/johnsont/millere/.conda/envs/mob_suit/bin/mob_recon", line 10, in
sys.exit(main())
File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/mob_suite/mob_recon.py", line 835, in main
database_dir=database_dir))
File "/home/johnsont/millere/.conda/envs/mob_suit/lib/python3.7/site-packages/mob_suite/mob_recon.py", line 194, in run_mob_typer
raise Exception("MOB_typer could not type {}".format(plasmid_file_abs_path))
Exception: MOB_typer could not type /home/johnsont/millere/suganda_data/mob_suite/test/plasmid_653.fasta

I'm not sure what the issue is, but I'm wondering if it has something to do with my version of python? The anaconda version that I originally tried to use could not solve the MOB-suite conda environment (python3/3.7.1_anaconda) so I installed python version 3.7.6 in my MOB-suite env first and then the conda package:
# load python3/3.7.1_anaconda
module load python3

# create MOB-suite env
conda create --name mob_suit
source activate mob_suit
conda install python==3.7.6
conda install -c bioconda mob_suite

Any suggestions would be very much appreciated!
Many thanks!
Liz

Hello,
I recommend updating to MOB-suite v. 3.0.0. That issue isn't reproducible with me in a test environment on my computer using some test data. So it could be specific to your input data or environment. MOB-suite v. 3 is a major rewrite of the software and part of that is better error propagation. So I highly recommend switching and trying again. Could you let me know if the error persists in the new version? If so, it would be great to have the input file that you are using.
Cheers,
James

My apologies. I totally missed that the default conda package wasn't version 3.0.0. I've now updated MOB-suite to v.3.0.0 and it's working perfectly!
Thank you so much for your help and prompt reply!
Liz


chore: explorer feedback consistency

Updates icons (sync with the new website)
Updates link structure
Adjust typography
Adjust spacing
Match new layout (almost)

📷 Demo

We should change the a to use Link from the UI lib? @pedronauck

Would be better to use it, of course


 We work on the dev branch and then add that functionality to master when it is stable. Your issue may be fixed on dev and there is no need for this issue, just wait and it will eventually be merged to master.
All Issue sections MUST be completed to help us determine the actual problem and find its cause
-->

Expected Behavior

Actual Behavior
**SyntaxError: Missing parentheses in call to 'print' zo 18 sep 2016 21:33:54 CEST Pokebot  Stopped. Press any button or wait 20 seconds to continue.     File "pokecli.py", line 68     print "Run following command to get latest update: pip install -r requirements.txt --upgrade"**
Steps to Reproduce
bash run.sh
Other Information
OS:
mac osx 10.11.6
Branch:
master
Git Commit:
`commit fef76945022210f4663c091b55750c57684026ec
Merge: a8ee312 5d3148e
Author: Stuart Travers<EMAIL_ADDRESS>Date:   Sat Sep 17 08:34:23 2016 +1000
Merge pull request #5493 from Gobberwart/dev_release_ivcp`

Python Version:

Any other relevant files/configs (eg: path files)

Have you upgraded the components needed with pip according to a newly created file called "requirements.txt"?

A good guess is that you are using python3. The print statement has to be with parenthesis there.
Could you check first your python version.
Check for the running virtualenv and pip and afterwards also the ./bin/* Directory inside your app dir for correct python interpreter.

If you are running python3, do sudo apt-get remove python3 && sudo apt-get install python

@VBQL would be better do do that it in virtualenv via pip like I was already saying.
Also on MacOS I don't think there is apt installed :-)

Ahem, yes the python upgrade was the issue, indeed 3.7. I assumed wrongly, that it would still be compatible...Had fun downgrading. Thanks for the help. As for apt MacOS has homebrew (not installed by default)

Wonderful.
Indeed python3 is not fully compatible to python2. There exists of cours also a package to load the deprivated print statement to python3.
Glad to help.
#closed


Try to fix macOS wheel build

Follow up of https://github.com/kylebarron/pydelatin/pull/37.
Note: Somehow I can't run the workflows locally anymore, so this is kind of a shot in the dark.

Try to add glm to the path. I needed to do that locally, so this might also work on the runner.
Use macos-13 instead of macos-latest, because macos-latest uses M1 processors and I'm not sure if x86 wheels can be built.

Can you temporarily set the yaml to
# Only run on new tags starting with `v`
on:
  push:

so that we can test the wheel building on this branch?

I guess since I don't check for a tag, you should also comment out the release part while testing on this branch

I'm not sure why but the wheels aren't building on CI in this PR. Are the actions running on your fork?

Yes they are running here 👍

The build seems to be successful, so I reverted the temporary changes.

Ok, I force pushed the v0.2.8 tag again, so wheels are attempting to build from main now

Looks like 0.2.8 is on pypi: https://pypi.org/project/pydelatin/0.2.8/#files
Can you try that out?

Works great, thank you!
pip install --no-cache pydelatin
Collecting pydelatin
  Downloading pydelatin-0.2.8-cp312-cp312-macosx_11_0_arm64.whl.metadata (8.9 kB)
Requirement already satisfied: numpy in /Users/patrick/Library/Application Support/hatch/env/virtual/off-grid/iGlVZ8gk/off-grid/lib/python3.12/site-packages (from pydelatin) (2.1.0)
Downloading pydelatin-0.2.8-cp312-cp312-macosx_11_0_arm64.whl (170 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 171.0/171.0 kB 4.1 MB/s eta 0:00:00
Installing collected packages: pydelatin
Successfully installed pydelatin-0.2.8


Merge labels from KafkaCluster resource to child objects

Q
A

Bug fix?
no

New feature?
no

API breaks?
no

Deprecations?
no

Related tickets
fixes #X, partially #Y, mentioned in #Z

License
Apache 2.0

What's in this PR?

Update util.MergeLabels to a variadic Function to merge multiple map of labels.
Why?

It could be great that "child" objects (eg. pods, services, cm, pvc) inherit labels from KafkaCluster resource.
Additional context

This docker image is available if you want to test the feature
sebdoido/kafka-operator:0.8.1-4
helm upgrade -i kafka-operator --namespace=dbre banzaicloud-stable/kafka-operator --set operator.image.repository=sebdoido/kafka-operator --set operator.image.tag=0.8.1-4

Checklist

[ ] Implementation tested
[ ] Error handling code meets the guideline
[ ] Logging code meets the guideline
[ ] User guide and development docs updated (if needed)

To Do

[ ] If the PR is not complete but you want to discuss the approach, list what remains to be done here

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

We really like the idea that the child objects can inherit labels from the KafkaCluster resource. Although we think it should be optional. We recommend to add a new field to the kafkacluster_types something like ClusterLabels and if that one is set the operator should place the labels in all resources. What do you think?

Thanks for your comment.
You're right, i'm going to continue to work on this PR on that way, adding a flag to feature-flip this behaviour.

Hi there,
image sebdoido/kafka-operator:0.8.1-5 is available if you want to test the latest commit.
Need to add this flag to your manifest propagateLabels: true
Example:
config/samples/simplekafkacluster.yaml
11:  propagateLabels: true

From a quick look, it appears you need to do a make generate manifests and commit the result.
The make manifests will update the CRD in config/base/crds, and it will have to be manually added to the copy in the helm chart.

Thanks, i updated the pr

Thanks, good catch

I see a few other places where we should probably do that too

Let me know if you want to I complete some stuff through this PR ?

Don't worry about that, for this I'd say just make sure the helm chart is updated.

Hi there
any chance to get this PR merged?
Thanks

Sure will check it today

Thanks for the contribution @sebdoido


Azure Functions ModuleNotFoundError for installed packages

Hello,
I'm attempting to get an Azure Function up and running and can't seem to get deployment figured out. I've determined that Kudu is not available for Linux Function Apps, so deploying via most of the Kudu package options is not possible.
Azure Pipelines seems to be a great option for deployment and I'm able to successfully build and deploy to my Azure Function from Pipelines. Within pipelines (build phase) I have an "install application dependencies" step that properly installs all packages from my requirements.txt file.
Despite the build and release processes running smoothly, when I run my application it errors out with the following:
Exception: ModuleNotFoundError: No module named 'pyodbc'
Stack:   File "/usr/local/lib/python3.6/site-packages/azure/functions_worker/dispatcher.py", line 230, in _handle__function_load_request
func_request.metadata.entry_point)
File "/usr/local/lib/python3.6/site-packages/azure/functions_worker/loader.py", line 66, in load_function
mod = importlib.import_module(fullmodname)
File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 941, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/site/wwwroot/HttpTrigger/init.py", line 7, in
import pyodbc
This occurs both for the pyodbc module as well as the requests module. I suspect it would occur with any 3rd party module.
I originally tried a thinned down requirements.txt file, which had the same error. My current file is this:
adal==1.2.1
asn1crypto==0.24.0
astroid==2.1.0
atomicwrites==1.3.0
attrs==19.1.0
autopep8==1.4.4
azure==4.0.0
azure-applicationinsights==0.1.0
azure-batch==4.1.3
azure-common==1.1.18
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.5
azure-datalake-store==0.0.44
azure-eventgrid==1.2.0
azure-functions==1.0.0b4
azure-graphrbac==0.40.0
azure-keyvault==1.1.0
azure-loganalytics==0.1.0
azure-mgmt==4.0.0
azure-mgmt-advisor==1.0.1
azure-mgmt-applicationinsights==0.1.1
azure-mgmt-authorization==0.50.0
azure-mgmt-batch==5.0.1
azure-mgmt-batchai==2.0.0
azure-mgmt-billing==0.2.0
azure-mgmt-cdn==3.1.0
azure-mgmt-cognitiveservices==3.0.0
azure-mgmt-commerce==1.0.1
azure-mgmt-compute==4.4.0
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==1.4.0
azure-mgmt-containerregistry==2.7.0
azure-mgmt-containerservice==4.4.0
azure-mgmt-cosmosdb==0.4.1
azure-mgmt-datafactory==0.6.0
azure-mgmt-datalake-analytics==0.6.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-datamigration==1.0.0
azure-mgmt-devspaces==0.1.0
azure-mgmt-devtestlabs==2.2.0
azure-mgmt-dns==2.1.0
azure-mgmt-eventgrid==1.0.0
azure-mgmt-eventhub==2.4.0
azure-mgmt-hanaonazure==0.1.1
azure-mgmt-iotcentral==0.1.0
azure-mgmt-iothub==0.5.0
azure-mgmt-iothubprovisioningservices==0.2.0
azure-mgmt-keyvault==1.1.0
azure-mgmt-loganalytics==0.2.0
azure-mgmt-logic==3.0.0
azure-mgmt-machinelearningcompute==0.4.1
azure-mgmt-managementgroups==0.1.0
azure-mgmt-managementpartner==0.1.0
azure-mgmt-maps==0.1.0
azure-mgmt-marketplaceordering==0.1.0
azure-mgmt-media==1.0.0
azure-mgmt-monitor==0.5.2
azure-mgmt-msi==0.2.0
azure-mgmt-network==2.6.0
azure-mgmt-notificationhubs==2.0.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-policyinsights==0.1.0
azure-mgmt-powerbiembedded==2.0.0
azure-mgmt-rdbms==1.7.1
azure-mgmt-recoveryservices==0.3.0
azure-mgmt-recoveryservicesbackup==0.3.0
azure-mgmt-redis==5.0.0
azure-mgmt-relay==0.1.0
azure-mgmt-reservations==0.2.1
azure-mgmt-resource==2.1.0
azure-mgmt-scheduler==2.0.0
azure-mgmt-search==2.0.0
azure-mgmt-servicebus==0.5.3
azure-mgmt-servicefabric==0.2.0
azure-mgmt-signalr==0.1.1
azure-mgmt-sql==0.9.1
azure-mgmt-storage==2.0.0
azure-mgmt-subscription==0.2.0
azure-mgmt-trafficmanager==0.50.0
azure-mgmt-web==0.35.0
azure-nspkg==3.0.2
azure-servicebus==0.21.1
azure-servicefabric==<IP_ADDRESS>
azure-servicemanagement-legacy==0.20.6
azure-storage-blob==1.5.0
azure-storage-common==1.4.0
azure-storage-file==1.4.0
azure-storage-queue==1.4.0
beautifulsoup4==4.7.1
bs4==0.0.1
certifi==2018.11.29
cffi==1.12.2
chardet==3.0.4
colorama==0.4.1
cryptography==2.6.1
dict2xml==1.5
idna==2.8
isodate==0.6.0
isort==4.3.9
json2xml==3.0.0
lazy-object-proxy==1.3.1
lxml==4.3.1
mccabe==0.6.1
more-itertools==7.0.0
msrest==0.6.6
msrestazure==0.6.0
oauthlib==3.0.1
pep8==1.7.1
pluggy==0.9.0
py==1.8.0
pycodestyle==2.5.0
pycparser==2.19
PyJWT==1.7.1
pylint==2.2.2
pyodbc==4.0.26
pytest==4.4.1
python-dateutil==2.8.0
pytz==2019.1
requests==2.19.0
requests-oauthlib==1.2.0
rope==0.14.0
six==1.11.0
soupsieve==1.8
urllib3==1.24.1
wrapt==1.11.1
xmltodict==0.11.0
Can anyone help me understand what I'm missing here?

Thank you for opening this issue! We are routing it to the appropriate team for follow up.

@scott-nordicwi you should create an issue in the Function repo, it will get faster and you will reach the right people directly:
https://github.com/Azure/azure-functions-python-worker/issues
@asavaritayal FYI

I was able to get a build that imports these packages by switching over from the Template style to the YAML code. I used the following YAML code:
jobs:

job: Build
pool:
vmImage: ubuntu-16.04
steps:

task: UsePythonVersion@0
displayName: "Setting python version to 3.6 as required by functions"
inputs:
versionSpec: '3.6'
architecture: 'x64'
bash: |
if [ -f extensions.csproj ]
then
dotnet build extensions.csproj --output ./bin
fi
python3.6 -m venv worker_venv
source worker_venv/bin/activate
pip3.6 install setuptools
pip3.6 install -r requirements.txt
task: ArchiveFiles@2
displayName: "Archive files"
inputs:
rootFolderOrFile: "$(System.DefaultWorkingDirectory)"
includeRootFolder: false
archiveFile: "$(System.DefaultWorkingDirectory)/build$(Build.BuildId).zip"
task: PublishBuildArtifacts@1
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/build$(Build.BuildId).zip'
name: 'drop'

Hello,
I have got the same error below while running the code in the Azure function App.
Exception: ModuleNotFoundError: No module named 'azure.common'
Stack:   File "/usr/local/lib/python3.6/site-packages/azure/functions_worker/dispatcher.py", line 230, in _handle__function_load_request
func_request.metadata.entry_point)
File "/usr/local/lib/python3.6/site-packages/azure/functions_worker/loader.py", line 66, in load_function
mod = importlib.import_module(fullmodname)
File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 941, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/site/wwwroot/fun/init.py", line 3, in <module
Can anyone tell me what's the issue here?


docker-compose grpc error

gRPC client-server connection is broken when running with docker-compose
When a client tries to dial server TLS handshake error from <IP_ADDRESS>:34824: tls: oversized record received with length 21536
we need to fix containers setup

issue only exist on docker-compose containers


Repeat calls of fastLinkJS cause IllegalStateException on Windows and JDK 17+

How to reproduce

Clone https://github.com/nartamonov/scalajs-bug-repr
Enter SBT console
fastLinkJS (should execute without errors), do not exit SBT console
Make any changes to Main.scala
fastLinkJS again, it throws exception:sbt:scalajs-bug-repr> fastLinkJS
[info] compiling 1 Scala source to C:\Temp\scalajs-bug-repr\target\scala-2.13\classes ...
[info] Fast optimizing C:\Temp\scalajs-bug-repr\target\scala-2.13\scalajs-bug-repr-fastopt
Exception in thread "Thread-12" java.lang.IllegalStateException: Promise already completed.
  | => sat scala.concurrent.Promise.complete(Promise.scala:53)
        at scala.concurrent.Promise.complete$(Promise.scala:52)
        at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:187)
        at scala.concurrent.Promise.success(Promise.scala:86)
        at scala.concurrent.Promise.success$(Promise.scala:86)
        at scala.concurrent.impl.Promise$DefaultPromise.success(Promise.scala:187)
        at org.scalajs.linker.PathOutputDirectory$Handler$5$.completed(PathOutputDirectory.scala:224)
        at org.scalajs.linker.PathOutputDirectory$Handler$5$.completed(PathOutputDirectory.scala:218)
        at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:129)
        at java.base/sun.nio.ch.Invoker.invokeDirect(Invoker.java:160)
        at java.base/sun.nio.ch.Invoker.invoke(Invoker.java:188)
        at java.base/sun.nio.ch.Invoker.invoke(Invoker.java:300)
        at java.base/sun.nio.ch.WindowsAsynchronousFileChannelImpl$ReadTask.failed(WindowsAsynchronousFileChannelImpl.java:501)
        at java.base/sun.nio.ch.PendingIoCache$1.run(PendingIoCache.java:155)
        at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:113)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1623)

If second call of fastLinkJS was completed without errors, repeat again (change source code and execute cmd). After 2-3 times it should throw error.

Environment
The bug is reproduced under the following conditions:

Windows 10 22H2 build 19045.2604
Eclipse Temurin 17.0.6+10 or 20+36

With JDK 11 or Linux all works fine.

Hello, and thanks for the report.
I tried reproducing this on Windows 11 22H2 build 22621.1413, using Temurin-17+35 (build 17+35), but could not trigger the issue. I repeated the change+fastLinkJS step 20 times in a row without any issue. Is there anything else that could help me reproduce?

I have a tentative "shot in the dark" fix over at #4842. @nartamonov Perhaps you could try it locally to see if it fixes your issue? You can find instructions on how to locally publish Scala.js at
https://github.com/scala-js/scala-js/blob/main/DEVELOPING.md#publish-locally
Note that due to other recent changes, the steps you provide will not even run into the situation you describe. You will have to reload+fastLinkJS at every step, in addition to changing the file. (in a single sbt session, the code path reported in the stack trace won't be used more than once).

@sjrd Yes, it's a strange bug, it reproduces on my work PC, but don't reproduces under the same circumstances on my home PC. Home PC is much more performant that the former, may be that's the reason.
Anyway, I've upgraded to ScalaJS 1.13.1 and the bug doesnt reproduces anymore on work PC! Thank you very much for quick fix! 👍 👍


All integers >= 1000000 are treated as decimals and output with ".0"

Describe the Bug with repro steps
Summary
This defect occurs when attempting to map an int equal to or larger than 1000000. If the source value meets that criteria, it appears to be cast as a decimal when output to the destination schema. This issue occurs even if both source and destination types are integers.
Repro Steps

Create a new data map in VS Code with a both a source and target property that is an integer
Map from the source integer to the destination integer
Ensure you have saved and clicked "Generate XSLT"
Prepare test data that has an integer value >= 1000000
Open "> Test"
Paste your test data
Click "Test"
Observe that the integer value now has ".0" and appears to be treated as a "number" value

Example Schemas
source
{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "http://example.com/example.json",
  "type": "object",
  "default": {},
  "title": "Root Schema",
  "required": ["data"],
  "properties": {
    "data": {
      "type": "object",
      "default": {},
      "title": "The data Schema",
      "properties": {
        "Name": {
          "type": "string",
          "default": "",
          "title": "The Name Schema",
          "examples": ["Testing1"]
        },
        "LoginCount": {
          "type": "integer"
        }
      }
    }
  }
}

target
{
    "$schema": "https://json-schema.org/draft/2019-09/schema",
    "$id": "http://example.com/example.json",
    "type": "object",
    "default": {},
    "title": "Root Schema",
    "required": ["data"],
    "properties": {
      "data": {
        "type": "object",
        "default": {},
        "title": "The data Schema",
        "properties": {
          "FullName": {
            "type": "string"
          },
          "NumberOfLogins": {
            "type": "integer"
          }
        }
      }
    }
  }

Screenshots or Videos

Version
v4.57.6
Additional context
No response

@DanielleCogs Let's add a tracking item for v2 after you are able to debug this a bit more

Would this be a front end issue or a backend issue that we should bring up to Sanjiv?

Would this be a front end issue or a backend issue that we should bring up to Sanjiv?

Seems like a backend, yes, but will be good to confirm first

It seems to be a backend issue. I took a look, and the frontend does send over an integer to test.
{     "DirectTranslation": {         "EmployeeID": 1000000,         "EmployeeName": "John Doe"     } }
and we receive back a decimal value
{   "DirectTranslation": {     "EmployeeID": 1000000.   } }


Restic secret creation for VSB and VSR needs to move to common.go

VSB and VSR have separate functions for creating a Restic secret. We can move these to a common file and check for DMB or DMR with a flag.
https://github.com/konveyor/volume-snapshot-mover/blob/master/controllers/restic.go

#102


Can't download redial data, file is not found.

Download with the given url https://pkueducn-my.sharepoint.com/:u:/g/personal/franciszhou_pku_edu_cn/EdVnNcteOkpAkLdNL-ejvAABPieUd8jIty3r1jcdJvGLzw?download=1, but can't get data, 404 FILE NOT FOUND

We have updated all the download links. You can try again.


No module named 'tslearn.soft_dtw_fast' when importing tslearn.metrics

Describe the bug
I can't import tslearn.metrics
To Reproduce
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-95-d750b1add255> in <module>
      8 import moviepy.editor as mp
      9 from IPython.display import Video
---> 10 import tslearn.metrics as tsm

~/491/python/project_env/lib/python3.7/site-packages/tslearn-0.3.1-py3.7-macosx-10.15-x86_64.egg/tslearn/metrics.py in <module>
     11 from sklearn.metrics.pairwise import euclidean_distances
     12 from sklearn.utils import check_random_state
---> 13 from tslearn.soft_dtw_fast import _soft_dtw, _soft_dtw_grad, \
     14     _jacobian_product_sq_euc
     15

ModuleNotFoundError: No module named 'tslearn.soft_dtw_fast'

Expected behavior
import the library
Environment (please complete the following information):
macos version: 10.15.4
python version: 3.7.6
tslearn 0.3.1
Additional context
I specifically want to use DTW metrics

Make sure to run the setup.py script. You did not create a script in the tslearn directory itself right (because that could cause issues as well)?

This looks like your tslearn is not properly installed.
I have just uploaded a bunch of wheels for all platforms on PyPI, which should make it easier to install on diverse platforms.
Could you try to uninstall your tslearn and then reinstall it from PyPI and tell us if that works, and, if not, give us the output you get?

https://github.com/rtavenar/tslearn/issues/196
I installed tslearn from github manually since I had this issue. I uninstalled it now and run the pip command:
sudo python3 -m pip install tslearn
now it works! thanks

This is great news !

This is great news !


fixes for rubygems1.8 [CHEF-2790]

See ticket; this fixes deprecation warnings for rubygems 1.8 users.
Please also apply to chef 0.9. Let me know if you have any concerns or questions.

We would love to see the deprecation warnings under Rubygems 1.8 addressed. We're heavily reliant on Chef, we now have other third-party dependencies that are pulling us to current Rubygems, and the volume of resulting warnings out of Chef has become a significant nuisance.


Document testing phase and lessons learned (🌱 Kale)

What we learned from this mega sprint
What is the scope of the testing
What we plan to do next (update roadmap)
Other

Could we move this task out of work cycle Kale and into its own one-week interlude?


export to csv with default column names fails

http://javi.cartodb.com/api/v1/sql?q=select st_astext(st_makepoint(0,0))&format=csv

@javisantana by "fails" I understand that you're getting an empty document, right? I'm not getting any errors, just a blank CSV.

I think this is the same problem than #204

It's not the same problem, check:
https://documentation.cartodb.com/api/v1/sql?q=select%20st_astext(st_makepoint(0,0))&format=csv
vs
https://documentation.cartodb.com/api/v1/sql?q=select%20st_astext(st_makepoint(0,0))%20p&format=csv

ok!

vamos!

https://github.com/OSGeo/gdal/blob/trunk/gdal/ogr/ogrsf_frmts/pg/ogrpglayer.cpp#L1387-L1403

Here's where it actually gets called
https://github.com/OSGeo/gdal/blob/trunk/gdal/ogr/ogrsf_frmts/pg/ogrpglayer.cpp#L2075
OGR is trying to catch queries that generate geometry (in both binary and text representations) and avoid putting those representations into non-geometry columns. It does so in your test because you use a common output function signature (st_astext) and then output to a format that OGR doesn't write geometry into by default (so the column "disappears")

Options:

rewrite the query slightly before handing it to OGR to avoid using the "magic function names" as column names
add a config parameter to OGR to make the "magic name passing" behavior optional


inconsistent GraphQLRequest when using cacheExchange

Hey there,
I'm trying to run the SSR & NextJS example and I noticed an inconsistency with the GraphQLRequest object when cacheExchange is present.
Per my understanding, __typeName shouldn't be part of the GraphQLRequest since it's just forwarding via fetchExchange (with the default of cache-first). Any idea why the GraphQLRequest sent to the server is different when cacheExchange is defined?
Thanks!
ssrCache & fetchExchange Only
Json: GraphQLRequest { query: "query allProjects($id: Int!) {\n  allProjects(organizationId: $id) {\n    id\n    uuid\n    name\n    description\n  }\n}\n", operation_name: None, variables: Some(Object([(Spanning { item: "id", start: SourcePosition { index: 0, line: 0, col: 0 }, end: SourcePosition { index: 0, line: 0, col: 0 } }, Spanning { item: Scalar(Int(1)), start: SourcePosition { index: 0, line: 0, col: 0 }, end: SourcePosition { index: 0, line: 0, col: 0 } })])) }
Without cacheExchange, same as above
Json: GraphQLRequest { query: "query allProjects($id: Int!) {\n  allProjects(organizationId: $id) {\n    id\n    uuid\n    name\n    description\n  }\n}\n", operation_name: None, variables: Some(Object([(Spanning { item: "id", start: SourcePosition { index: 0, line: 0, col: 0 }, end: SourcePosition { index: 0, line: 0, col: 0 } }, Spanning { item: Scalar(Int(1)), start: SourcePosition { index: 0, line: 0, col: 0 }, end: SourcePosition { index: 0, line: 0, col: 0 } })])) }
Default Example Implementation (3-ssr/nextjs example)
Json: GraphQLRequest { query: "query allProjects($id: Int!) {\n  allProjects(organizationId: $id) {\n    id\n    uuid\n    name\n    description\n    __typename\n  }\n  __typename\n}\n", operation_name: None, variables: Some(Object([(Spanning { item: "id", start: SourcePosition { index: 0, line: 0, col: 0 }, end: SourcePosition { index: 0, line: 0, col: 0 } }, Spanning { item: Scalar(Int(1)), start: SourcePosition { index: 0, line: 0, col: 0 }, end: SourcePosition { index: 0, line: 0, col: 0 } })])) }
My GraphQL server is returning this error concrete_type_name must be implemented by unions and interfaces

Most if not all cache implementations require the typename to be requested so that data can be associated with the types that have come back.
Our docs have some detailed information on why the cache exchange requires this information. Because it requires it, it transforms your GraphQL queries to also request typename fields.
Now, I’m not sure what your GraphQL server has been implemented with but the error seems rather serious. It seems that some field’s type does not have a typename, because it’s part of a union or interface?

Closing due to inactivity. Please let us know if you need more help on this :)


Fix hidden unit computation

If LSTM:buildHidden() is an implementation of equation 3 in the readme, it was missing a Tanh nonlinearity at the end. This also corresponds to the "Vanilla LSTM" in LSTM: A Search Space Odyssey.

OMG thanks you very much for this.


[QUERY] ArmClient subscription scopes

Library name and version
Azure.ResourceManager.ResourceGraph 1.0.0
Query/Question
Is it somehow possible to set multiple subscriptions as scope in a ResourceGraph query like it was with the old SDK and the rest api?
I can't seem to figure it out, and i think the possibility is missing..
Subscriptions is read only:

Environment
.Net 7.0
Visual Studio 17.4.4

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

(not on the team, but...)
The Subscriptions collection/list cannot be replaced with your own (which makes sense especially with your nullable input list and the non-null base list), but the list itself is mutable and you can add entries to it.
if (SubscriptionIds != null)
{
    foreach (string SubscriptionId in SubscriptionIds)
    {
        Querytest.Subscriptions.Add(SubscriptionId);
    }
}

I truly have no words right now.. Can't believe i missed this :) Thank you @AtOMiCNebula!