iOS crashes when multiple images are compressed in a loop

When calling FlutterImageCompress.compressWithFile() within a 'for loop' multiple times, iOS crashes with:

Lost connection to device. Exited (sigterm)

A quick hack to solve this is by pausing (example in the code below) a short time just before compressing, but I'd like a more robust solution. I can confirm that this issue does not affect android. Calling the following code below on iOS with a List of 1 single element does not cause a crash. The ideal solution would be a native function compressWithFiles that accepts a List<String> paths and compresses multiples images in one call.
    List<List<int>> result = [];
    for (int i = 0; i < entityAssets.length; i++) {
      File file = await entityAssets[i].file;

      /// uncomment this to avoid crash on iOS
      /// await Future<Null>.delayed(Duration(milliseconds: 300));
      List<int> compressed = await FlutterImageCompress.compressWithFile(
          file.path,
          rotate: 90,
          minHeight: 1000,
          minWidth: 1000,
          quality: 95);

      result.add(compressed);
    }

@ROTGP
Can you give me a number corresponding to the number of pictures, such as 100 or 1000?
And give the width and height of some original pictures?

Look at your code, you seem to be using the photo_manager library, which itself has a simple API to get thumbnails of any size .
Before I solve this issue problem, you can use the code: Uint8List thumbDataWithSize = await entity.thumbDataWithSize(width,height); to get thumbData.

ok, If I can locate the error, I will fix it as soon as possible.

@ROTGP
I tried to locate the problem and use the following way to get the image compression, no crash happened.
Can you try to create a new project and use the code replace your main.dart and pubspec.yaml.
run flutter packages get and run project in xcode. If crash happens , paste your crash logs.
Or give me a minimal runnable example, because where I use the same code as you and find no errors.

@CaiJingLong I tried to replicate the crash using your code and strangely, I wasn't able to either. I abstracted the code in my other project as much as possible until it was pretty much identical to your example, but it still crashes, very strange. I'm closing this for now, if I can manage to find the issue I'll either reopen it, or comment here for future reference. Thanks for your time and good work

I believe i had the same same issue just now. In debugging, there was no specific method call or even image that caused the crash, so i have to think it is hitting some form of memory threshold. All you can do about that is make sure you don't keep the original bytes of an image in memory after having compressed it.

@MCOfficer
Can you provider some code to reproduce it?

i'll see what i can do, but essentially it comes down to

read the bytes of some (3-5) large (~10MB) images from files
throw them into compressWithList
for extra risk, don't discard the original image bytes from memory

@ROTGP Here's a minimal example. Select a big, wallpaper-size image, preferably several of them, hit multiply a couple times and then start the compression.
import 'dart:io';

import 'package:flutter/material.dart';
import 'package:flutter_image_compress/flutter_image_compress.dart';
import 'package:image_picker/image_picker.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key, this.title}) : super(key: key);

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  List<List<int>> _images = [];
  List<List<int>> _compressed = [];

  void _pickImage() async {
    File file = await ImagePicker.pickImage(source: ImageSource.gallery);
    setState(() => _images.add(file.readAsBytesSync()));
  }

  void _multiply() {
    setState(() => _images.addAll(List.of(_images)));
  }

  void _compress() async {
    for (List<int> image in _images) {
      List<int> compressed = await FlutterImageCompress.compressWithList(image,
          minHeight: 1000, minWidth: 1000);
      setState(() => _compressed.add(compressed));
    }
  }

  void _clear() {
    setState(() {
      _images = [];
      _compressed = [];
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'Selected: ${_images.length} Images',
            ),
            RaisedButton(
              onPressed: _pickImage,
              child: Text('Pick Image'),
            ),
            RaisedButton(
              onPressed: _multiply,
              child: Text('Multiply Selected'),
            ),
            Text(
              'Compressed: ${_compressed.length} Images',
            ),
            RaisedButton(
              onPressed: _compress,
              child: Text('Compress'),
            ),
            RaisedButton(
              onPressed: _clear,
              child: Text('Clear'),
            )
          ],
        ),
      ),
    );
  }
}

However, I'm not sure if this is the crash, or if this code just crudely forces the app to run out of memory.

@MCOfficer you'd best open a new issue... I haven't been working with Flutter recently.


python311Packages.sqlmodel: 0.0.14 -> 0.0.16

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python311Packages.sqlmodel is: Module to work with SQL databases
meta.homepage for python311Packages.sqlmodel is: https://github.com/tiangolo/sqlmodel
meta.changelog for python311Packages.sqlmodel is: https://github.com/tiangolo/sqlmodel/releases/tag/0.0.16
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.0.16 with grep in /nix/store/qvckypd9m1jrfwjd54gg0wi2v6b0xk1r-python3.11-sqlmodel-0.0.16
found 0.0.16 in filename of file in /nix/store/qvckypd9m1jrfwjd54gg0wi2v6b0xk1r-python3.11-sqlmodel-0.0.16

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath
khoj
python311Packages.pgvector
python311Packages.sqlmodel
python312Packages.sqlmodel

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/qvckypd9m1jrfwjd54gg0wi2v6b0xk1r-python3.11-sqlmodel-0.0.16 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python311Packages.sqlmodel https://github.com/r-ryantm/nixpkgs/archive/7e98a86a2e1ba455d8c85cc9fd6b22753ca0983f.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/7e98a86a2e1ba455d8c85cc9fd6b22753ca0983f#python311Packages.sqlmodel

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/qvckypd9m1jrfwjd54gg0wi2v6b0xk1r-python3.11-sqlmodel-0.0.16
ls -la /nix/store/qvckypd9m1jrfwjd54gg0wi2v6b0xk1r-python3.11-sqlmodel-0.0.16/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package marked as broken and skipped:

python312Packages.sqlmodel

1 package failed to build:

khoj

2 packages built:

python311Packages.pgvector
python311Packages.sqlmodel

Maintainer pings
cc @fabaff for testing.

@ofborg eval


Implement consistent input checks

Currently we have a number of input validation functions, none of them are consistently enforced and some of them don't do anything.
Related to #15

so what constraints on data/input checks do we want? I guess we need to check the data type of series (real valued or missing)?

On the low level interface, I think we agreed on the at least the following checks:

isinstance(X, pd.DataFrame)
isinstance(y, (pd.Series, np.ndarray))

They should be checked in all our methods I think.
A few other checks I can think of are:

cells of X should contain either primitives or series in form of either pd.Series or np.ndarray
series in cells of X should be of at least length >=2
missing values
consistency of series within the same column (extending pandas column typing), including equal length, shared time index, consistent type of series within column

Not sure whether to check these in our methods, or delegate them to the data container and only query them if required.

so for full flexibility, we should allow meta data loading, and characteristics from the data. So checks

Yes, most of our methods assume univariate time series data (at least until we implement bespoke methods to handle these cases), that should also be consistently checked (related to multivariate composites, see #36), so X.shape[1] == 1.

We should use sklearn's check_is_fitted in predict/transform.

Another question is whether we want the check_input kwarg for estimators/transformers allowing users to disable input checks. I thought that would be a good idea initially, but I'm less sure now.

I dont know re check_input, but my opinion is that it is up to an estimator to check its capabilities, allowing user to disable checks sounds like it breaks encapsulation and may result in undetermined behaviour.

common check if data (column typing) is series or primitives

Currently, error messages are not very helpful when passing tabular pandas dataframe (see e.g. #150). We need to add checks for nested format, either as part of the input validation functions or the pandas container.

Closed by #246


Outdated documentation on start_span

How do you use Sentry?
Sentry Saas (sentry.io)
Version
2.15.0
Steps to Reproduce

description is deprecated since https://github.com/getsentry/sentry-python/pull/3524
Documentation at https://docs.sentry.io/platforms/python/tracing/instrumentation/custom-instrumentation/ still uses it

Expected Result
The documentation should match current code.
Actual Result
Documentation recommends deprecated API.

Thanks @nijel, fixing this now.


Move verify stuff into it's own page

Fixes #3

This PR adds pages after both start pages so that users need to consent to verify data being used.

The button will throw an error if the checkbox isn't checked!

Just noticed that this wasn't fully done - I missed doing the start page link up for concessionary travel.
@kaydale Try and make sure when you review things that are for both that you check it's actually done in both places.


CMake build (Linux and Windows MSVC tested only) integration support …

CMake build (Linux and Windows MSVC tested only) integration support added;

You can reopen issue: https://github.com/Cyan4973/zstd/issues/66

Thanks @mailagentrus, it's an excellent PR.
The content of patch looks very good to me, so we just have to discuss a small detail :

I worry about having a CMakeLists.txt files within the same directory as Makefile.
Anyone like me who would blindly tape cmake . ends up with Makefile overwritten in the process.
For this reason, I tend to prefer having a separate directory for CMakeLists.txt.

Since you have 2 such files, plus a few dependency files, it could prove non-trivial to do.
Does it make sense to regroup them into a dedicated directory ?
Does it require to merge both CMakeLists.txt first ?

Hmm, I think, that it's possible to reorganize CMakeLists.txt file structure to build lib and programs in specified output directory. Now we have next directory hierarchy:
root
|
..lib (with Makefile)
..programs (with Makefile)
..Makefile
What hierarchy do you prefer to split Makefiles and CMakeLists? :) I'll fix that at this weekends..

I would like to suggest :
root
|
lib (with Makefile)
programs (with Makefile)
contrib / cmake (with CMakeLists.txt and dependencies)
The "contrib" emphasizes that the maintainer of CMakeLists.txt isn't me, since I have no skill in CMake files. In which case, it would also be appropriate for your name to be provided as the author of the files.

Added contrib folder with CMakeFiles. Added /analyze options for MSVC projects, set O3 by default optimization flag in Release mode...

minor typo : you named the directory contib instead of contrib.

You are right, sorry. Typo is fixed.

New line before EOF added (for each file in PR)...

Thank you @mailagentrus !  These changes will help future commits.

Thank you @t-mat ! Yes, for example, user can build project with MSVC 2015 with these changes (automatic conversion MSVC solution file from 2013 to 2015 version was broken for me)...

Looks like a great PR to merge,
just ... well, if you look at directory name, although the change made it closer, it has not yet reached contrib yet ....

Contrib directory name successfully changed...

Thanks @mailagentrus, it's a great PR.
I tested CMake generation of Makefile on Linux (Mint 17 x64) using gcc 4.8.4,
compilation fails on paramgrill target, which is a test tool.
It seems the linking stage fails, looking for symbol log.
So I suspect link instruction to math library -lm is missing.

Will check this issue and fix tomorrow morning... I think so, look like missed -lm linker flag for paramgrill tool.


Add Spectronaut support

=> Input files: https://cloud.cmb.ugent.be/index.php/s/zdGB3zZ7Fwed9gq?path=%2FModule_4_DIA_Quantification%2FSpectronaut
In the parameter file no clear definition of MS1 and MS2 tolerance:
│  ├─ MS1 Mass Tolerance Strategy:	System Default
│  └─ MS2 Mass Tolerance Strategy:	System Default

Can someone also help out with the documentation on Spectronaut? @wolski @AnStaes

@brvpuyve, none of the input files have species information. E.g. there's no columns with <gene_name>_HUMAN or <gene_name>_YEAST output, which is necessary.

@rodvrees Witold reran the analysis with some changed parameter settings which should have resolved the loss of species information. The files have been uploaded to the Cloud server.

Spectronaut output parsing is now ready (#399). Instead of the user having to fiddle with settings to get assure the species information to be present, we've implemented a mapping step so that gene names get mapped to the correct protein descriptor.
e.g. Q8WWZ7 (in spectronaut output) will be mapped to sp|Q8WWZ7|ABCA5_HUMAN.
This will also apply for FraggerDIA output, which has the same issue

Great, thanks Robbe!

Spectronaut support completed with PR #437


add script to remove NjP prefix from 773/774

Addresses #13.

@tventimi Can you please review?

I am sure there are plenty such quirks. That should be a separate ticket for identifying candidates for cleanup. Such data discrepancies would result in Alma not being able to create the linkage, so it's a user issue. Thank you for surfacing that and reviewing the script!


label for resource appends always extra "s" - should be only valid in english

Describe the bug
When adding the public variable $label to a resource, it appends an "s" to the Entry in the navigation bar. But this seems to be only valid in english language and maybe some others.
To reproduce
Steps to reproduce the behavior:

add a resource
add $label variable

Expected behavior
I would probably just turn on that feature when no custom label is set.
Context

Package version: [1.9.6]
Livewire version: [2.4.3]
Laravel version: [8.38.0]

Hey, this is a good default for most languages. You should override the static getPluralLabel() method on the resource to set a custom plural form, or the static $navigationLabel property to set a custom navigation label.

Hey, this is a good default for most languages. You should override the static getPluralLabel() method on the resource to set a custom plural form, or the static $navigationLabel property to set a custom navigation label.

thanks a lot - works like a charm - I prefer the $navigationLabel over the function. would be nice to have an explanation in the docs. e.g. where do I use $label and where $navigationLabel

Hey, this is a good default for most languages. You should override the static getPluralLabel() method on the resource to set a custom plural form, or the static $navigationLabel property to set a custom navigation label.

thanks a lot - works like a charm - I prefer the $navigationLabel over the function. would be nice to have an explanation in the docs. e.g. where do I use $label and where $navigationLabel

Feel free to submit a PR!

The URL uses the $slug property on the resource.

The URL uses the $slug property on the resource.

perfect ! Thanks !

add    protected static ?string $title = "Rekening"; in file the extends to ListRecord, example :
label("Tambah Rekening"),
        ];
    }

}

simple method is to add this function in Resource for example:
UserResource.php
in my case I want to Translate "Users" to arabic :
public static function getPluralLabel(): ?string
    {
        $locale = app()->getLocale();
        if($locale == 'ar'){
            return "المستخدمين" ;
        }
        else
            return "Users" ;

    }

Hey, this is a good default for most languages. You should override the static getPluralLabel() method on the resource to set a custom plural form, or the static $navigationLabel property to set a custom navigation label.

how to?

protected static ?string $pluralLabel = '...';


Search function?

Would it be possible to make a search function instead of using the email and password?

What do you mean? What search function? I am confused

Like uh what r6.tracker.network has instead of getting just your account information a search function to search players would be way better

But.. you can do that

Oh ignore me im stupid I didnt read I saw email and password and assumed it only allowed you to get your own stats


added functions to generate and insert user data

The user schema looks like following
class User(DocumentTemplate):
    _schema = schema
    _key = LexicalKey(keys="id")
    _base = "User"
    id : str
    watched_contents: Set["Content"]

@GavinMendelGleason  I have added the user schema and data into the script as discussed earlier. Please let me know if you need any other changes 😅


python 2 support

I needed these changes to persuade things to run on python2.
I don't know if this is everything that is needed because I'm hitting #11

I pushed my changes for python2. I didn't merge your PR because it was incomplete... Thanks for the PR!


TCPConnection read/write buffer sizes are using the same variable

The variable used to control how large the TCPConnection read buffer can grow to is also used to control when to yield when writing data. The two should be different.
The usage of _max_size (soon to be _read_buffer_size) in TCPConnection write methods should be replaced a new variable _bytes_to_yield_after_writing. Additionally, constructors need to be updated to allow for setting that variable and it should be given a "reasonable" default value.

@dipinhora I noticed this as part of #3185. just an FYI.

:+1:

additionally, the number of bytes to yield after for writing is also tied to the read buffer size.
we should add a yield_after_reading and yield_after_writing respectively.


Implement Anchornav

Alaska Airlines Pull Request
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Resolves: #10
Summary:
Please summarize the scope of the changes you have submitted, what the intent of the work is and anything that describes the before/after state of the project.
Instantiate a new version of auro-hyperlink called auro-anchorlink that comes along with its own styles and functions to create the new Anchornav feature.
Type of change:
Please delete options that are not relevant.

[x] New capability
[ ] Revision of an existing capability
[ ] Infrastructure change (automation, etc.)
[ ] Other (please elaborate)

Checklist:

[x] My update follows the CONTRIBUTING guidelines of this project
[x] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

There are two new issues added to this milestone for review

https://github.com/AlaskaAirlines/auro-nav/issues?q=is%3Aopen+is%3Aissue+milestone%3A"v1.1+anchor+nav"+label%3Anot-reviewed

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Update bundle output and add wait option to copyandpause

Bundles are now written without the version in their name.

Bundles have their names added as fields in the JSON

copyandpause is now copyandmaybepause, with pausing being an optional choice. This is to enable its use in init containers.

To deploy this, I'll remove the existing debian bundle manually and use our build pipeline to generate a new one in its place.

/test pull-trust-manager-smoke

Thanks for the PR. Do you think that we might remove the --wait flag in the future?
/approve
/lgtm

Thanks for the PR. Do you think that we might remove the --wait flag in the future?

I only added it because I can see the same code being used for dynamic bundles in the future, but the image could be much simpler now we don't need to worry about pausing. I think it'll be a judgement call - if we end up not doing anything with dynamic images, we can simplify the code.
Thanks for the review! :heart:


Add host prefix to indexes in Algolia

I like what we've come up with!
One thing: We should add INDEX_PREFIX to the .env.example and the docs to explain how it keeps different local indices apart.


Specify platform & folder to generate icons

Hi,
It would be nice if there is some parameters to specify wanted generated platform and output folder.
In my case I locally build Android and I want to build iOS on phonegap build. So I have to provide Android and iOS icons in the www folder but as I only added Android, iOS icons are not generated :(
Thanks for all

Same here.

Same here. The iOS icons are being generated in the wrong folder as of the latest Cordova version. It should be in Resources/icons instead of Images.xcassets. Adding this option will help in case the folder path changes again in the future cordova versions.

Maybe specify the file name too? I've used cordova-icon to generate notification icon for Android, but had to rename all generated images.

I would only like to be able to specify the platform.
File names and folder structures should be handled by this tool and it beeing kept up-to-date with cordova

Hi everyone, I made a pullrequest which it can do this. [https://github.com/AlexDisler/cordova-icon/pull/130](pull request 130).
$ cordova-icon --iconspath=platforms/res/
$ cordova-icon --platforms=android:ios

I hope it helps, regards

Another way to address this is to only generate for the platform targeted by the cordova prepare command.  For example:
cordova prepare ios ==> should only generate icons for ios
cordova prepare android ==> should only generate icons for andriod
cordova prepare ==> generates for both ios and android.
This can be fixed with a single line update to index.js (for each platform, android example below):
isAdded : fs.existsSync('platforms/android') && process.env.CORDOVA_PLATFORMS.includes('android')
I do not have time right now to put this in a pull request, but I have tested this in my local installation.


Automation: License & copyright checks

To assist with maintainer license obligations, add a Github Action that will flag any PR that meets any of the following criteria:

Touching the LICENSE or NOTICE files.
Touching any "pristine" files from the Capirca repo. A "pristine" file is one that has not yet been modified by this fork and would require a modification notice be added under the terms of the Apache License. The Github Action or its configuration can contain this list of "pristine" files and we can remove files from the list once they are modified.

We have successfully checked this prior to the soft launch

Right, at time of launch there are no pristine files. No action needed.


Option pin / unpin tabs

My proposal next feature -> option pin / unpin tab with result of query. Pin tab always remain until user click unpin option or sql editor will be closed.

+1

Can you give me some example of how it may look in UI?

Hey, I took this screenshot from SQLDeveloper and as you can see the tab has an "pin" icon. When I click on it, the tab is pinned and any other results will be shown in a new tab. I can even rename the tab.

DBeaver has a different look-and-feel.
I can put pin into results status bar or in filters toolbar. Both these ways doesn't look very good..

Can't you put it on the tab context menu?
I would love to help implement this, but I never wrote an Eclipse plugin before.

Context menu is easy. Another option is to draw a pin over tab itself.
I'll do it by myself, np, I'm just guessing about good looking and convenient UI.

A pin over the tab looks great, and the context menu could have an option to rename it.

Pin/unpin actions will be accessible thru the context menu for now.


SDP-839 reduce job interval

What

Reduced interval for TSS Monitor and Payment jobs to 10s
Removes some unnecessary logs
Switched some logs from Info to Debug
Added logs where needed.

Why

Sync payment statuses faster
Make logs less verbose when not needed

Known limitations
[TODO or N/A]
Checklist
PR Structure

[ ] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
[ ] This PR does not mix refactoring changes with feature changes (split into two PRs otherwise).
[ ] This PR's title starts with the name of the package, area, or subject affected by the change.

Thoroughness

[ ] This PR adds tests for the new functionality or fixes.
[ ] This PR contains the link to the Jira ticket it addresses.

Configs and Secrets

[ ] No new CONFIG variables are required -OR- the new required ones were added to the helmchart's values.yaml file.
[ ] No new CONFIG variables are required -OR- the new required ones were added to the deployments (pr-preview, dev, demo, prd).
[ ] No new SECRETS variables are required -OR- the new required ones were mentioned in the helmchart's values.yaml file.
[ ] No new SECRETS variables are required -OR- the new required ones were added to the deployments (pr-preview secrets, dev secrets, demo secrets, prd secrets).

Release

[ ] This is not a breaking change.
[ ] This is ready for production.. If your PR is not ready for production, please consider opening additional complementary PRs using this one as the base. Only merge this into develop or main after it's ready for production!

Deployment

[ ] Does the deployment work after merging?

stellar-disbursement-platform-backend-preview is available here:SDP:      https://sdp-backend-pr12.previews.kube001.services.stellar-ops.com/healthAP:       https://sdp-ap-pr12.previews.kube001.services.stellar-ops.com/healthFrontend: https://sdp-backend-dashboard-pr12.previews.kube001.services.stellar-ops.com

stellar-disbursement-platform-backend-preview is available here:SDP:      https://sdp-backend-pr12.previews.kube001.services.stellar-ops.com/healthAP:       https://sdp-ap-pr12.previews.kube001.services.stellar-ops.com/healthFrontend: https://sdp-backend-dashboard-pr12.previews.kube001.services.stellar-ops.com

Question: what do you think about these logs? Something we could remove or decrease as well?


useHistoryState clear function error

I imported hook into Provider file and when I invoke clear function i get Error initialState is not defined

Fixed. We'll get a release out soon. Thank you.


git-ignore log files left behind from tests

How to categorize this PR?

/kind cleanup
/area os
/os garden-linux
What this PR does / why we need it:
Integration tests might leave log files behind which should not make their way into the repository.
Release note:

chore: git-ignore logs files from tests

How can I reproduce this? Because a make dev_all does not leave anything behind.


Group Shout

So the bot use to post a shout in a channel when a shout goes out but now it wont. Note(s): the bot is super slow and always breaks

Any errors in the bot logs?

@yogurtsyum nope

If there's no errors, it never detected a shout. So you probably have the wrong group id or the shout text was the same as before.

@yogurtsyum i will go over that and try again

@yogurtsyum so everything is right with the id and the shout. any other ideas on why it wont work?

Can you send a picture of the bot logs?

what do you mean by bot logs @yogurtsyum

The logs for the bot. And can you please stop pinging me in every single reply?

ok so it works randomly now

Then maybe you didn't wait the 30 seconds it takes to log.

well i did then i just made a new channel and it worked


Allow github usernames to map to specified system usernames

cf https://github.com/jupyter/oauthenticator/issues/28
This allows one to specify
c.GitHubOAuthenticator.github_to_nix_usernames_map = {<github username>: <system username>}

in jupyterhub_config.py

Thank you for your patch! I've left some comments about possible style changes, and I can test it later as well :) Do you think you can also squash the commits into one with a more descriptive commit message?

Made the style changes and squashed down the commits.
I use it like:
c.JupyterHub.authenticator_class = 'oauthenticator.LocalGitHubOAuthenticator'
c.GitHubOAuthenticator.oauth_callback_url = os.environ['OAUTH_CALLBACK_URL']
c.GitHubOAuthenticator.username_map = {<github username>: <system username>}

Thanks, this is nice. I think we may fold this into an upstream proposal to add a generic mechanism for username normalization, but that doesn't mean we can't proceed with this PR as-is. What do you folks think about whether the whitelist should apply to the incoming or outgoing names? My initial reaction is that it should apply to the incoming names, but if we implement username_map as part of normalization, I would expect it to take effect afterward.


chore(CHANGELOG): update to 0.9.2

Changes

FIX: ignore null properties (#39)

Team work. 🎖

🏆


code improved for parameter copy

changed the parameter pass from copy to r-value ref

Maybe we could use rvalue in asyncMultiPut as well?

Please hold this PR. I need more thought. has been tested that the current changes are not effective.

To strengthen again. ready to review.


runtime: close of a channel with pending sends incorrectly flagged by race detector

What version of Go are you using (go version)?

$ go version
go version go1.11.4 windows/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\kjk\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\kjk\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\kjk\AppData\Local\Temp\go-build206244702=/tmp/go-build -gno-record-gcc-switches

What did you do?

Save this to e.g. t.go:
package main

import (
	"fmt"
	"time"
)

func main() {
	ch := make(chan string)
	for i := 0; i < 256; i++ {
		go func(n int) {
			ch <- fmt.Sprintf("string %d", n)
		}(i)
	}
	time.Sleep(time.Second)
	close(ch)
}

and run go -race t.go.
What did you see instead?
I see a data race reported:
> go run -race t.go
==================
WARNING: DATA RACE
Write at 0x00c00001a130 by main goroutine:
  runtime.closechan()
      C:/Go/src/runtime/chan.go:327 +0x0
  main.main()
      C:/Users/kjk/src/t.go:16 +0xb5

Previous read at 0x00c00001a130 by goroutine 231:
  runtime.chansend()
      C:/Go/src/runtime/chan.go:140 +0x0
  main.main.func1()
      C:/Users/kjk/src/t.go:12 +0xcc

Goroutine 231 (running) created at:
  main.main()
      C:/Users/kjk/src/t.go:11 +0x83
==================
panic: send on closed channel

goroutine 201 [running]:
main.main.func1(0xc00001a120, 0xc3)
        C:/Users/kjk/src/t.go:12 +0xcd
created by main.main
        C:/Users/kjk/src/t.go:11 +0x84
panic: send on closed channel

goroutine 235 [running]:
main.main.func1(0xc00001a120, 0xe5)
        C:/Users/kjk/src/t.go:12 +0xcd
created by main.main
        C:/Users/kjk/src/t.go:11 +0x84
exit status 2

What did you expect to see?
I didn't expect data race between close(ch) and pending ch <- v send.
The spec (https://golang.org/ref/spec#Channel_types) is curiously mum on the subject.
It says:

A single channel may be used in send statements, receive operations, and calls to the built-in functions cap and len by any number of goroutines without further synchronization.

By not mentioning close it implies that one has to synchronize close and sends.
But I wouldn't know how to synchronize those. My sends have already been dispatched and closing a channel seems like a valid (and only) way to ensure those sending goroutines don't stick forever (assumingI recover the resulting panic).
The data race seems harmless, as I'm done with this channel, but I run my tests with -race detection enabled and this makes the test fail.

Similar to https://github.com/golang/go/issues/27769

It's a valid transaction race. There is no guarantee the sends complete before the close, which would result in a panic. The race detector actually helped you here.

@as The code with data race is in Go's runtime, not in my code.
I don't see which part of the spec makes it a "valid transaction race".
As stated in my bug report, I understand that this panics as that is what the spec says and I'm fine with that.
I'm not fine with this also being reported as a data race as the spec doesn't say that closing a channel with pending sends is undefined behavior (which would be enough of an escape to consider the implementation conforming).

Thank you for this report @kjk!
In deed as @go101 noted, it is a duplicate of #27769 which is still open and trying to decide how to document this. As @as has noted it is a valid transaction race but as you too have found there is no documentation in the spec about this. Let's move this to that issue as it is also actionable but more discussion has gone on there. I'll close this one in favor of #27769 but please feel free to reopen this one.

I don't seem to have permission to re-open the bug, could you please re-open it.
This is not a dup #27769.
The discussion in that bug operates under the assumption that this:
	ch := make(chan string)
	for i := 0; i < 256; i++ {
		go func(n int) {
			ch <- fmt.Sprintf("string %d", n)
		}(i)
	}
	close(ch)

is invalid Go program, that race detector is rightfully detecting a data race here and it's just a matter of documenting that.
I've read the discussion and I didn't find a justification for that position.
The fact that close of a channel with pending sends is considered invalid Go code would certainly violate my assumptions, given that raison d'etre for channels to allow thread-safe exchange of data between goroutines.
I claim that the above is a valid Go code and there's a bug in Go's implementation of close/send or implementation of race detector.
Maybe I'm wrong but I would like someone involved in the spec/compiler/race detector to opine on that. Without that closing this bug seems premature.

Furthermore, #27070 describes a similar race between close and len/cap which was deemed a bug of enough severity that the fix was backported to 1.11.

I can't quite match up the line numbers in chan.go but chansend() is reading c.closed outside of c.lock while closechan() is writing to it inside c.lock.
That suggests the intent was to access to c.closed to be protected and chansend() failing to do so is a data race.
Furthermore chanrecv() uses atomic.Load() to read c.closed further muddying the water as to how the variable was supposed to be used.
Presumably always using atomic operations for c.closed would be the right thing to do.

There should be two checks. One fast path outside the critical section and another one inside. I'm not sure where there's suddenly an atomic being used in chanrecv. This code in particular always seemed to avoid atomics for checking closed.

Reopening this since @odeke-em said "feel free to reopen this one" and the issue's author has requested it.

@as Not sure how to parse this response.
A write to c.closed happens within a lock. A read of c.closed happens outside of a lock.
As far as I can tell, if that happens on different threads, it's a data race.
Here's chan.go code stripped down:
func chansend(c *hchan, ep unsafe.Pointer, block bool, callerpc uintptr) bool {
	if c.closed == 0 { // concurrent read on another thread, data race
		return false
	}
	lock(&c.lock)

	if c.closed != 0 {
		unlock(&c.lock)
		panic(plainError("send on closed channel"))
	}
}

func closechan(c *hchan) {
	lock(&c.lock)
	if c.closed != 0 {
		unlock(&c.lock)
		panic(plainError("close of closed channel"))
	}
	c.closed = 1 // write on one thread
}

Where does the spec declare this code as generating a data race?

See https://golang.org/ref/mem#tmp_7.
The happens-before relationship for channels is only defined between corresponding sends and receives, and between a close and the receives that observe the corresponding zero-value. There is no happens-before relationship between a send and a close, so such an event is a data race.
Data races are defined by the memory model, not what the concrete channel implementation in a given Go implementation happens to do: reads and writes to c.closed in the implementation are not particularly relevant.

Sorry for pressing this but I don't feel this was fully addressed and would appreciate clarification on 3 points.
1. What this implies is: closing a channel with pending sends is incorrect Go code?
If yes, how is one supposed to safely close such channel?
If there is no safe way, it implies that this is invalid concurrency patter in Go: I launch goroutines to send, they block because no-one is reading yet and I want to terminate the communication by closing the channel? This is the essence of my program is doing.
The implication is that if I ever want to close a channel, I'm not supposed to do a send on that channel on another goroutine.
2. A race between len and close was considered a bug (#27070) by @ianlancetaylor . Not by literal reading of the spec but presumably because of practical implications. On what basis a race between send and close is different?
3. A data race in the implementation of channel send and receive is relevant, because:
a) it's a bug in itself
b) it might be the root cause of the issue
Am I wrong that racing on c.closed is a bug that should be fixed?

And to quote @ncw from #27070:

I think that any channel operations causing the race detector to moan is surprising.

What this implies is: closing a channel with pending sends is incorrect Go code?

Yes.

If yes, how is one supposed to safely close such channel?

Wait for the senders to complete. (For your original example, that might look something like https://play.golang.org/p/4LtjIUA0tla.)

If there is no safe way, it implies that this is invalid concurrency [pattern] in Go: I launch goroutines to send, they block because no-one is reading yet and I want to terminate the communication by closing the channel?

Correct, that pattern is invalid. You can terminate communication by closing a channel on which the goroutines are receiving, but not one on which they are sending.
Note that in your original example, the goroutines are not terminated: if any are remaining, they either leak or panic. (See https://play.golang.org/p/2Tjo2a3Q3iN and https://play.golang.org/p/sHi6fxSgEwv respectively.)

A race between len and close was considered a bug […]. On what basis a race between send and close is different?

len does not change its behavior depending on whether the channel is closed (https://play.golang.org/p/WVYBJCwZf8q).

Am I wrong that racing on c.closed is a bug that should be fixed?

Since a send concurrent with a close is defined to be a data race, the implementation of chansend may assume that the closed variable is not being written concurrently.
In contrast, closechan must acquire the lock in order to avoid races with receive operations.

Sorry for pressing this but I don't feel this was fully addressed and would appreciate clarification on 3 points.

What this implies is: closing a channel with pending sends is incorrect Go code?

Yes.  In addition to what Brian said, consider the following two operations happening concurrently:
c <- x
close(c)

If the send happens first, both operations complete fine, and the channel reader sees x, then end-of-channel.
If the close happens first, the send panics and the channel reader sees just end-of-channel.
Those are wildly different behaviors, and the choice between those behaviors is made by the scheduler (which the user has no control over).  One can of course design a program so that it can handle either behavior, but normally this situation is just a bug waiting to happen.

If yes, how is one supposed to safely close such channel?

See Brian's answer.

If there is no safe way, it implies that this is invalid concurrency patter in Go: I launch goroutines to send, they block because no-one is reading yet and I want to terminate the communication by closing the channel? This is the essence of my program is doing.
The implication is that if I ever want to close a channel, I'm not supposed to do a send on that channel on another goroutine.

A race between len and close was considered a bug (#27070) by @ianlancetaylor . Not by literal reading of the spec but presumably because of practical implications. On what basis a race between send and close is different?

Consider the following two operations happening concurrently:
close(c)
... = len(c)

We get the same final result no matter which order those two operations happen.  So it's unlikely to be a bug in the user's program. (Digression: this is kind of a weird case, because presumably there is a receiver around somewhere, and the receive and len race with each other. So it's only odd cases where #27070 matters.)

A data race in the implementation of channel send and receive is relevant, because:

a) it's a bug in itself
b) it might be the root cause of the issue
Am I wrong that racing on c.closed is a bug that should be fixed?

The runtime has all sorts of data races in it.  The runtime doesn't depend on the Go memory model, but the memory model of the underlying hardware.  The particular case of reading c.closed outside of a lock is very tricky, as evidenced by the long comment before the c.closed==0 test in chansend.
I'm pretty sure there isn't a bug there, but I share your squeamishness.  Sometimes we go squeamish to go fast...
It's definitely not (b) though, the implementation is separate from the language semantics.

I believe the justification for closing #27070 was:

I think that any channel operations causing the race detector to moan is surprising.
which applies equally here.

As to closing the channel: in my case there is no guarantee that readers will drain the channel hence I would deadlock waiting for sends to complete.
For context: it's a database driver where a client can ask to be notified about future changes in the database, for example:
func (c *DatabaseChanges) ForAllDocuments() (chan *DocumentChange, CancelFunc, error)

Those changes are async by nature (might or might not happen at some point in the future) so It seemed to me that the idiomatic way to solve that in Go would be to return a channel with those changes.
My driver code has no control over when / if the client will read those notifications.
For that reason, I do each send on a goroutine, to not block the thread that talks to the database. When the client calls CancelFunc or closes the session that owns those notifications, I want to terminate those sends and potential readers, which close does.
I'm fine both with the panic that results from send on a close channel and with indeterminism of how many notifications will end read by the client.
My point here is that this seems like a reasonable thing to do yet seemingly impossible to do without a data race (which I wouldn't care about because it is seemingly benign, but it does cause tests to fail).
The "wait for readers to drain the channel" solution is not applicable here.
And sure, I could re-architect this to e.g. use a callback function or implement a thread-safe queue of messages but channel seemed like the most elegant solution.

And sure, I could re-architect this to e.g. use a callback function or implement a thread-safe queue of messages but channel seemed like the most elegant solution.

The usual approach would be a synchronous callback function. Channels in exported APIs are rarely the most elegant solution.
(For more detail, see the first section of Rethinking Classical Concurrency Patterns.)

The only reason to close a channel is to tell the receiver that there is nothing left to receive.  Therefore, closing a channel should always be thought of a send operation on a channel, and only the sender should close a channel.  When you have multiple senders, they must coordinate when to close the channel.
Note also that sending on a closed channel causes a run-time panic.  The most obvious way to see that your original code has a race is to remove the call to time.Sleep.  That program will panic, because it has a race condition.  You can never remove a race condition by adding a time.Sleep call.

I feel like those justifications are mixing up different kind of races.
This bug report is about data race as detected by race detector. An implementation of channel send and receive in the runtime is doing reads and writes to the same memory location and race detector flags it because the cpu considers this to produce an undefined result.
The race you're talking about is about not being able to say if "a" happens before "b" or "b" happens before "a". It's a logical race related to ordering. It might or might not be a bug in my code, depending on whether I rely on the order or not. In my particular case I do not so it's not a bug.
While I appreciate you take the time to offer guidance on how to structure the code, the snippet I provided is a minimized test case to demonstrate a data race in the runtime. As I described elsewhere in my real code I can't coordinate senders because they are blocked on readers that might never read. It might be bad code but the issue it surfaced is whether a close is safe for concurrent use.
I understand that this generates a panic because the spec says so. I expect it, I handle it, it doesn't make the program incorrect as per Go spec.
I concede that the spec allows it:

A single channel may be used in send statements, receive operations, and calls to the built-in functions cap and len by any number of goroutines without further synchronization.

By not including close in the verbiage the spec requires synchronization between close and send.
But it also requires synchronization between close and a receive, which, I hope we agree, is safe to do without further synchronization and would be a deal breaker in practice to have it work any other way.
To sum up: the spec requires synchronization between close and all other channel operations but in practice, the implementation is safe for concurrent use of close with receive/len/cap but not send.
Additionally, a race between close and len was fixed recently even though, as far as I can tell, the spec treats them the same.

This bug report is about data race as detected by race detector. An implementation of channel send and receive in the runtime is doing reads and writes to the same memory location and race detector flags it because the cpu considers this to produce an undefined result.

(I assume you mean send and close, not send and receive?)
The runtime arranges to have closechan write a location that chansend reads on purpose. This isn't an accidental race detection. It's not a bug, it's intentional. There are even tests: runtime/race/testdata/chan_test.go:TestRaceChanSyncCloseSend and TestRaceChanAsyncCloseSend.

It might or might not be a bug in my code, depending on whether I rely on the order or not. In my particular case I do not so it's not a bug.

The problem is that for almost everyone else, it is a bug. It is a known gotcha for inexperienced Go programmers. Their program works fine while testing, but then in production the scheduler starts doing something differently and their program panics. We're very on board with the fact that the race detector helps us detect this issue more reliably in testing.

As I described elsewhere in my real code I can't coordinate senders because they are blocked on readers that might never read.

If they are blocked on channel reads, then you can use a select on a cancellation channel to get them out of that read.
If they are blocked on an io.Reader, then it is trickier.  It would be nice to fix #20280 so io.Reader operations are cancellable.
You're right that the spec is somewhat muddied in where exactly a race should be reported, and where it shouldn't.  But we definitely want a race between send and close. Not because of the spec, but because reporting that race helps real users fix real bugs.

I do agree closing a channel with pending sends is incorrect Go code.
However, I also think @kjk's opinion is not totally unreasonable.
How about I do anticipate a send would panic on closing a channel?
Like the following code shows:
package main

import (
	"fmt"
	"time"
)

func main() {
	ch := make(chan string)
	for i := 0; i < 256; i++ {
		go func(n int) {
			defer func() {
				recover()
			}()
			ch <- fmt.Sprintf("string %d", n)
		}(i)
	}
	time.Sleep(time.Second)
	close(ch)
}

And I still feel this issue is a dup of https://github.com/golang/go/issues/27769


node-sass fails to install

I tried running npm install on this but it threw out a 404 error message when trying to install node-sass (I assume this is a dependency of some module being used).
A potential fix is found at the below page.
https://github.com/sass/node-sass/issues/2779#issuecomment-551959243

For anyone who is still having this issue, Node recently moved 12.x to LTS, and it seems like Azure DevOps is now using Node 12.x instead of Node 10.x for builds when the node version isn't specified.
To resolve the issue, I added a Node.js Tool Installer task build step to our build pipeline to specify which version of Node to use for Node commands.

For anyone else getting this issue, adding "node-sass": "^4.13.0" to the package.json devDependencies seems to sort the issue for now.


INovuClientConfiguration is not exposed via DI service provider

The values here are required for services that provide these details out to clients. For example, the server API provides clients with the resolution of the novu client for subscriber notifications (rather than hard code into the client)
Fix: register the interface for DI:
// note: IConfiguration configuration (because you are in registration land)
services.AddTransient<INovuClientConfiguration>(_ => configuration.GetNovuClientConfiguration())

Released 0.3.1


Issue #34 checking for different docker version formats

Please use: https://drupal.org/project/drupalci_testbot since this is a testing version.

Right. Sorry! ;-)


envoy crash when test http3 upgrade

I tried to load test the http3 upgrade between two envoy proxies, the deploy model as:
fortio --http1--> client side envoy proxy --http3--> server side envoy proxy --http1--> envoy as application
The config of client side envoy proxy is
admin:
  address:
    socket_address:
      protocol: TCP
      address: <IP_ADDRESS>
      port_value: 9902
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: <IP_ADDRESS>
        port_value: 10001
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  host_rewrite_literal: domain1.example.com
                  cluster: service_google
          http_filters:
          - name: envoy.filters.http.router
  clusters:
  - name: service_google
    connect_timeout: 30s
    type: LOGICAL_DNS
    # Comment out the following line to test on v6 networks
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service_google
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: <IP_ADDRESS>
                port_value: 10000
    typed_extension_protocol_options:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        explicit_http_config:
          http3_protocol_options: {}
        common_http_protocol_options:
          idle_timeout: 1s
    transport_socket:
      name: envoy.transport_sockets.quic
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.quic.v3.QuicUpstreamTransport
        upstream_tls_context:
          sni: proxy-postgres-backend.example.com
          common_tls_context:
            validation_context:
              match_subject_alt_names:
              - exact: proxy-postgres-backend.example.com
              trusted_ca:
                filename: certs/cacert.pem

The config of server side envoy proxy is:
admin:
  address:
    socket_address:
      protocol: TCP
      address: <IP_ADDRESS>
      port_value: 9901
static_resources:
  listeners:
  - name: listener_tcp
    address:
      socket_address:
        protocol: TCP
        address: <IP_ADDRESS>
        port_value: 10000
    filter_chains:
    - transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            tls_certificates:
            - certificate_chain:
                filename: certs/servercert.pem
              private_key:
                filename: certs/serverkey.pem
      filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: HTTP2
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              response_headers_to_add:
              - header:
                  key: alt-svc
                  value: h3=":10000"; ma=86400, h3-29=":10000"; ma=86400
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  host_rewrite_literal: www.envoyproxy.io
                  cluster: service_envoyproxy_io
          http3_protocol_options:
          http_filters:
          - name: envoy.filters.http.router

  - name: listener_udp
    address:
      socket_address:
        protocol: UDP
        address: <IP_ADDRESS>
        port_value: 10000
    udp_listener_config:
      quic_options: {}
      downstream_socket_config:
        prefer_gro: true
    filter_chains:
    - transport_socket:
        name: envoy.transport_sockets.quic
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.transport_sockets.quic.v3.QuicDownstreamTransport
          downstream_tls_context:
            common_tls_context:
              tls_certificates:
    udp_listener_config:
      quic_options: {}
      downstream_socket_config:
        prefer_gro: true
    filter_chains:
    - transport_socket:
        name: envoy.transport_sockets.quic
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.transport_sockets.quic.v3.QuicDownstreamTransport
          downstream_tls_context:
            common_tls_context:
              tls_certificates:
              - certificate_chain:
                  filename: certs/servercert.pem
                private_key:
                  filename: certs/serverkey.pem
      filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: HTTP3
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  #host_rewrite_literal: www.google.com
                  cluster: service_envoyproxy_io
          http3_protocol_options:
          http_filters:
          - name: envoy.filters.http.router
  clusters:
  - name: service_envoyproxy_io
    connect_timeout: 30s
    type: LOGICAL_DNS
    # Comment out the following line to test on v6 networks
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service_envoyproxy_io
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                #address: www.google.com
                address: <IP_ADDRESS>
                port_value: 8080
    #transport_socket:
    #  name: envoy.transport_sockets.tls
    #  typed_config:
    #    "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
    #    sni: www.google.com

The envoy version is 1.20 and with --concurrency of 1.
If I test with follow command (10 connections with no qps limit):
fortio load -qps -1 -c 10 -t 10s --timeout 120s  http://<IP_ADDRESS>:10001
Envoy won't crash but the server side proxy use almost 1 core and client side only use 0.1 core, then the result qps is poor.
But for test with 100 connections, the client side envoy would crash, the log as follow:
[2021-09-17 09:52:05.915][16036][info][main] [source/server/server.cc:803] all clusters initialized. initializing init manager
[2021-09-17 09:52:05.915][16036][info][config] [source/server/listener_manager_impl.cc:779] all dependencies initialized. starting workers
[2021-09-17 09:52:05.916][16036][info][main] [source/server/server.cc:822] starting main dispatch loop
[2021-09-17 09:52:11.291][16043][info][quic] [bazel-out/k8-opt/bin/external/com_github_google_quiche/quiche/quic/core/tls_client_handshaker.cc:463] Client: handshake finished
[2021-09-17 09:52:15.917][16036][info][main] [source/server/drain_manager_impl.cc:171] shutting down parent after drain
[2021-09-17 09:52:29.290][16043][info][quic] [bazel-out/k8-opt/bin/external/com_github_google_quiche/quiche/quic/core/tls_client_handshaker.cc:463] Client: handshake finished
[2021-09-17 09:53:59.468][16043][info][quic] [bazel-out/k8-opt/bin/external/com_github_google_quiche/quiche/quic/core/tls_client_handshaker.cc:463] Client: handshake finished
[2021-09-17 09:53:59.719][16043][critical][backtrace] [./source/server/backtrace.h:104] Caught Segmentation fault, suspect faulting address 0x0
[2021-09-17 09:53:59.719][16043][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2021-09-17 09:53:59.719][16043][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: cc1d41e7ee9fbfb7ee3c8f73724cdc41d7c6bbb0/1.20.0-dev/Clean/RELEASE/BoringSSL
[2021-09-17 09:53:59.720][16043][critical][backtrace] [./source/server/backtrace.h:96] #0: __restore_rt [0x7f7e471c6980]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #1: [0x55c355aa1247]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #2: [0x55c355a445b3]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #3: [0x55c355a47fcc]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #4: [0x55c355a48e34]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #5: [0x55c355a442cf]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #6: [0x55c355c3b9d6]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #7: [0x55c355c41864]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #8: [0x55c355b8496e]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #9: [0x55c355b764a5]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #10: [0x55c355bba357]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #11: [0x55c355bb7af7]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #12: [0x55c355bc017f]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #13: [0x55c355eef897]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #14: [0x55c355bbfbaf]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #15: [0x55c355bb5f44]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #16: [0x55c355bb56df]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #17: [0x55c355bb6185]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #18: [0x55c355b7203c]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #19: [0x55c355e0b6ff]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #20: [0x55c355e04afa]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #21: [0x55c355e02649]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #22: [0x55c355df39b1]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #23: [0x55c355df4c5c]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #24: [0x55c355efbb78]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #25: [0x55c355efa571]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #26: [0x55c35581d362]
[2021-09-17 09:53:59.762][16043][critical][backtrace] [./source/server/backtrace.h:98] #27: [0x55c35613aa73]
[2021-09-17 09:53:59.763][16043][critical][backtrace] [./source/server/backtrace.h:96] #28: start_thread [0x7f7e471bb6db]
ActiveStream 0x5e05bf517e00, stream_id_:<PHONE_NUMBER>474838081&filter_manager_:
  FilterManager 0x5e05bf517e78, state_.has_continue_headers_: 0
  filter_manager_callbacks_.requestHeaders():
    ':authority', 'domain1.example.com'
    ':path', '/'
    ':method', 'GET'
    ':scheme', 'http'
    'user-agent', 'fortio.org/fortio-dev'
    'x-forwarded-proto', 'http'
    'x-request-id', '00ce776b-f1ba-43d4-88c8-e10942fd7e69'
    'x-envoy-expected-rq-timeout-ms', '15000'
  filter_manager_callbacks_.requestTrailers():   null
  filter_manager_callbacks_.responseHeaders():   null
  filter_manager_callbacks_.responseTrailers():   null
  &stream_info_:
    StreamInfoImpl 0x5e05bf517f78, upstream_connection_id_: null, protocol_: 1, response_code_: null, response_code_details_: null, attempt_count_: 1, health_check_request_: 0, route_name_:
    OverridableRemoteConnectionInfoSetterStreamInfo 0x5e05bf517f78, remoteAddress(): <IP_ADDRESS>:37496, directRemoteAddress(): <IP_ADDRESS>:37496, localAddress(): <IP_ADDRESS>:10001
Http1::ConnectionImpl 0x5e05bf44d508, dispatching_: 1, dispatching_slice_already_drained_: 0, reset_stream_called_: 0, handling_upgrade_: 0, deferred_end_stream_headers_: 1, require_strict_1xx_and_204_headers_: 1, send_strict_1xx_and_204_headers_: 1, processing_trailers_: 0, no_chunked_encoding_header_for_304_: 1, buffered_body_.length(): 0, header_parsing_state_: Done, current_header_field_: , current_header_value_:
active_request_:
, request_url_: null, response_encoder_.local_end_stream_: 0
absl::get<RequestHeaderMapPtr>(headers_or_trailers_): null
current_dispatching_buffer_ front_slice length: 76 contents: "GET / HTTP/1.1\r\nHost: <IP_ADDRESS>:10001\r\nUser-Agent: fortio.org/fortio-dev\r\n\r\n"
ConnectionImpl 0x5e05bf465140, connecting_: 0, bind_error_: 0, state(): Open, read_buffer_limit_: 1048576
socket_:
  ListenSocketImpl 0x5e05bf7a8900, transport_protocol_: raw_buffer
  connection_info_provider_:
    ConnectionInfoSetterImpl 0x5e05bf735b18, remote_address_: <IP_ADDRESS>:37496, direct_remote_address_: <IP_ADDRESS>:37496, local_address_: <IP_ADDRESS>:10001, server_name_:
Segmentation fault (core dumped)

Any ideas? cc @alyssawilk  :)

Just a reminder while QUIC is in alpha we're working hard to get it to GA so please do check in with envoy-security before posting crash reports publicly :-)
That said thanks for the report.  Any chance you can get symbols on that stack trace?  cc @danzh2010

Actually I'm going to pull this from the MVP list as the connect config is separately tagged alpha, so I think QUIC in general can GA without connect support being trusted in prod.

oops, redacting, not an upgrade specific issue I misread

[2021-10-07 15:43:59.004][4210][critical][assert] [source/common/quic/codec_impl.cc:89] assert failure: stream != nullptr. Details: Fail to create QUIC stream.
[2021-10-07 15:43:59.004][4210][critical][backtrace] [./source/server/backtrace.h:104] Caught Aborted, suspect faulting address 0x106b
[2021-10-07 15:43:59.004][4210][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2021-10-07 15:43:59.004][4210][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: f90dd0b5525ae3ac6e5af320c353ff3135f4ff06/1.20.0-dev/Clean/DEBUG/BoringSSL
[2021-10-07 15:43:59.023][4210][critical][backtrace] [./source/server/backtrace.h:96] #0: Envoy::SignalAction::sigHandler() [0x55efb4fcae29]
[2021-10-07 15:43:59.023][4210][critical][backtrace] [./source/server/backtrace.h:96] #1: __restore_rt [0x7fa8a05a4980]
[2021-10-07 15:43:59.040][4210][critical][backtrace] [./source/server/backtrace.h:96] #2: Envoy::Http::CodecClient::newStream() [0x55efb4258e14]
[2021-10-07 15:43:59.057][4210][critical][backtrace] [./source/server/backtrace.h:96] #3: Envoy::Http::MultiplexedActiveClientBase::newStreamEncoder() [0x55efb4148add]
[2021-10-07 15:43:59.075][4210][critical][backtrace] [./source/server/backtrace.h:96] #4: Envoy::Http::HttpConnPoolImplBase::onPoolReady() [0x55efb41471ec]
[2021-10-07 15:43:59.092][4210][critical][backtrace] [./source/server/backtrace.h:96] #5: Envoy::ConnectionPool::ConnPoolImplBase::attachStreamToClient() [0x55efb415051e]
[2021-10-07 15:43:59.109][4210][critical][backtrace] [./source/server/backtrace.h:96] #6: Envoy::ConnectionPool::ConnPoolImplBase::newStreamImpl() [0x55efb4152812]
[2021-10-07 15:43:59.127][4210][critical][backtrace] [./source/server/backtrace.h:96] #7: Envoy::Http::HttpConnPoolImplBase::newStream() [0x55efb4146d6e]

From coredump and corresponding code, the reason of this issue is that envoy could not handle quic stream creation failure gracefully. A known issue, so close it.
By the way, I find that the support for http3 is still under heavy deployment, there are still many things to do. And I'd like to make some contributions in my spare time. Are there any suggestions on where to start @alyssawilk ? thanks :)


some buttons is disabled when run .files command

the local file button is ok.
the http files button
and '-' button at the right of uploaded local file
are disabled.

Yes that is true. The file dialog will be replaced with dedicated commands like .files add, .files download soon.

The file command no longer open an overlay and now support:
.files list            List all files.
.files add             Add files.
.files download $FILE  Download a file.
.files drop            Drop all files.
.files drop $FILE      Drop a single file.


Make --client to be true by default

How about --no-client instead of --client? I.e. making it opt-out instead of opt-in.
That would make HatTip slightly more zero config.

I agree with this.
Why would you use a Vite-based tool (instead of node, tsx, deno, wrangler dev etc.) if you're not interested in building client code. It's a frontend tool in the end.


Research about Appium automation test

Describe the solution you'd like
Research about Appium automation test

[ ] How to setup
[ ] Implement in react project
[ ] Customize behavior if necessary

Additional context

https://appium.io/docs/en/latest/quickstart/test-js/
https://webdriver.io/docs/appium
https://webdriver.io/docs/gettingstarted

1. Initiate a WebdriverIO Setup
1st method
# run
npm init wdio@latest .

# Accept install
1. What type of testing would you like to do?
->  E2E Testing - of Web or Mobile Applications

2. Where is your automation backend located?
-> On my local machine

3. Which environment you would like to automate?
-> Mobile - native, hybrid and mobile web apps, on Android or iOS

4. Android - native, hybrid and mobile web apps, tested on emulators and real devices
-> Android - native, hybrid and mobile web apps, tested on emulators and real devices
    > using UiAutomator2 (https://www.npmjs.com/package/appium-uiautomator2-driver)

5. Which framework do you want to use?
-> Mocha (https://mochajs.org/)

6. Do you want to use a compiler?
-> TypeScript (https://www.typescriptlang.org/)

7.

2nd method:
Installing packages using npm:
- @wdio/local-runner@latest
- @wdio/mocha-framework@latest
- @wdio/spec-reporter@latest
- wdio-video-reporter
- wdio-html-nice-reporter
- wdio-wait-for
- @testing-library/webdriverio
- @wdio/appium-service@latest
- wdio-vite-service
- wdio-vscode-service
- ts-node
- appium-uiautomator2-driver
- appium


The link is invalid and cannot be accessed

https://herachat.com/ai123

ok, thanks for your information. I have removed it.


StepperLayout doesnot refresh the UI

when i use StepperLayout  in my fragment does n=not open the ui for the fragment.

Could you provide a sample, please?


Add neo-scan-sync to CLI

Problem
Add neo-scan-sync to CLI
Solving issue https://github.com/CityOfZion/neo-local/issues/72
Solution
Adding neo-scan-sync
Checklist

[x] I have ran the tests locally.
[x] I branched off of the develop branch and not master.
[x] I did not change the VERSION file.

@nunojusto Can you resolve conflicts please?


weathergov.js:  Removed weatherEndpoint definition

Removed weatherEnpoint definition in defaults.  It is not used in the weathergov.js provider.

@rejas can you please review this PR, seems related to https://github.com/MichMich/MagicMirror/pull/2927, thanks.

Looks good @khassel


Filter category

Borrow from https://logging.apache.org/log4j/2.x/manual/filters.html. That's the perfect solution.

Unfortunately this breaks the registry class.
I will probably use two-phase replacement.

Make new class in experimental namespace, mark the old one as deprecated.
Drop old class. Remove namespace.


Improveimprove array/map/struct creating and function with decimalv3

Proposed changes
improve array/map/struct creating with decimalv3 and array/struct function with decimalv3
Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run p0

run p1


Increase code quality

Use code analyzers to increase the code quality. Try urbanjs.

Fixed by introducing UrbanJS.


move /.aquatone/

hey dude
how can i move /.aquatone/ to /var/ww/html/.aquatone/ and have it still work?

Hey there, it's currently not a documented "feature", but when AQUATONE tries to determine the folder to write files to, it will first look for an environment variable called AQUATONEPATH before falling back to the default location of ~/aquatone.
You can either define this environment variable somewhere like your .bashrc file, like:
export AQUATONEPATH="/var/www/html/.aquatone/"

Or you can prepend it before the aquatone commands, if it's more of a one-off thing:
$ AQUATONEPATH="/var/www/html/.aquatone/" aquatone-discover -d example.com
$ AQUATONEPATH="/var/www/html/.aquatone/" aquatone-scan -d example.com
$ AQUATONEPATH="/var/www/html/.aquatone/" aquatone-gather -d example.com

Hope that helps you, let me know if you have any problems.

thanks dude will deffo look doing this!


chore: price-feeder: handle wrong tx response code

Description
Handles the edgecase where the tx response code for a prevote is not zero, but doesn't error. Happens when a signature fails, e.g. the wrong chain is in the config.
Also updated the example config with the beta chain name to make it a bit more plug-and-play.

Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...

[ ] included the correct type prefix in the PR title
[ ] added appropriate labels to the PR
[ ] added ! to the type prefix if API or client breaking change
[ ] targeted the correct branch (see PR Targeting)
[ ] provided a link to the relevant issue or specification
[ ] added a changelog entry to CHANGELOG.md
[ ] included comments for documenting Go code
[ ] updated the relevant documentation or specification
[ ] reviewed "Files changed" and left comments if necessary
[ ] confirmed all CI checks have passed

Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
I have...

[ ] confirmed the correct type prefix in the PR title
[ ] confirmed ! in the type prefix if API or client breaking change
[ ] confirmed all author checklist items have been addressed
[ ] reviewed state machine logic
[ ] reviewed API design and naming
[ ] reviewed documentation is accurate
[ ] reviewed tests and test coverage
[ ] manually tested (if applicable)

Nice catch. Just one small minor nitpick.


Descriptors and annotations for source vs target

When a bindable service uses annotations or descriptors there are two things it needs to get across:

which spec, status or data it is referencing (i.e. "the source")
which bindable data it wants to map into (i.e. "the target")

Annotations can easily model this relationship for single items because the left side (of the : character) refers to the target and the right side refers to the source:
servicebinding/secret/host: {.status.address}

For x-descriptors, we can also easily model single items in a way where path refers to the source and the corresponding x-descriptor refers to the target:
        path: status.address
        x-descriptors:
        - servicebinding:secret:host

The difficulty arises when we want to refer to an item inside a ConfigMap or Secret as the source.
Currently the spec would say:
    path: dbCredentials
    x-descriptors:
      - urn:alm:descriptor:io.kubernetes:Secret
      - servicebinding:secret:user
      - servicebinding:secret:password

Which in its current syntax would mean:  grab the first item from the source Secret (dbCredentials) and map to the target Secret's user field, and similarly for password (second item).  But what if we wanted to name the source fields?
One approach would be to specify that item using the path key:
    path: dbCredentials.myUser
    x-descriptors:
      -servicebinding:secret:user
    path: dbCredentials.myPassword
    x-descriptors:
      - servicebinding:secret:password

And similarly using annotations:
servicebinding/secret/user: {.status.dbCredentials.myUser}
servicebinding/secret/password: {.status.dbCredentials.myPassword}

@AndrewJSchofield @sbose78 - as we discussed today.  please share your thoughts

What I've written is the following:
      statusDescriptors:
      - description: The secret containing the CA certificate
        displayName: Secret containing the CA certificate
        path: status.caCertificateSecret
        x-descriptors:
        - urn:alm:descriptor:io.kubernetes:Secret
        - binding:env:object:secret:ca.p12
        - binding:env:object:secret:ca.password

where I mean, there's a secret named in the status.caCertificateSecret property. From it, extract the ca.p12 and ca.password entries and copy them into the SB secret.
It feels to as if the path key in the CSV should only refer to paths which are in the CR, and a path which strays into a Secret or ConfigMap is a little delinquent.

I've also taken another liberty in the same area. If my CR looks like this:
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  name: my-cluster
spec:
 ...
status:
  listeners:
  - type: plain
    addresses:
    - host: myhost1.example.com
      port: 9092
    - host: myhost2.example.com
      port: 9092
  - type: tls
    addresses:
    - host: myhost3.example.com
      port: 9093
  bootstrap:
  - name: plain
    value: myhost1.example.com:9092,myhost2.example.com:9092
  - name: tls
    value: myhost3.example.com

In the CSV, I did this:
      statusDescriptors:
      - description: The addresses of the bootstrap servers for Kafka clients
        displayName: Bootstrap servers
        path: status.bootstrap
        x-descriptors:
        - binding:env:object:secret:endpoints

meaning "take the object under status.bootstrap and turn it into entries in a secret whose names start endpoints". In my example, I would expect endpoints.plain and endpoints.tls.
We're both struggling with the same thing really. How to map elements between two namespaces.

Thanks for the samples @AndrewJSchofield.   Ya - it's tricky because in some cases we want the x-descriptor to refer to the source elements (e.g. ca.p12 and ca.password) and in other cases we want the x-descriptor to refer to the target elements (e.g. endpoints.xyz).
Perhaps a way forward is to go with the latter approach:  the x-descriptor always refers to the target object/items, and path refers to the source.   This mean a limitation of not being able to "cherry pick" items from a complex source, since we can't drill into a ConfigMap, Secret or an array using path.
So the second example (endpoints) is good as-is, with the caveat that we need to document what happens when the source element is an array.
In the first example I guess would simply be:
        path: status.caCertificateSecret
        x-descriptors:
        - urn:alm:descriptor:io.kubernetes:Secret
        - servicebinding:secret

Following the semantics that SBO will simply copy all items from the source Secret into the target Secret.

@arthurdm A question I have is whether the bindable service or the user's ServiceBinding should be in control of how the binding information is made accessible. I would have thought that the later is better, meaning that the annotations are providing the source of the binding information and the x-descriptors are providing the names of the target items, but the ServiceBinding CR could then say whether it wants to use a Secret, a ConfigMap or envvars to receive the binding information. I wonder what you think.

that's a great question @AndrewJSchofield.  My view is that both should be possible.
Allowing bindable providers to map their binding data into a format that is (a) more accessible, such as the endpoints, and (b) closer to the spec binding data, solves that problem for all consumers (1..N), making it easier (because less YAML is needed) to author ServiceBinding CRs for the regular use case.
On the other hand, we need to have the flexibility of customization in the ServiceBinding CRs where cases where (a) the bindable provider did not do a good job in mapping the fields to consumable or spec-compliant fields, or (b) when they want to customize it in a way that fits their app better.

It feels to as if the path key in the CSV should only refer to paths which are in the CR, and a path which strays into a Secret or ConfigMap is a little delinquent.

Yes, this is the simplest thing to do, and that's what is supported today.

a path which strays into a Secret or ConfigMap is a little delinquent.

Today, this is built as an escape hatch as customEnvVar ( aka dataMapping ).
Thinking out aloud..

#  I want the secret `dbCredentials`'s `myHost` field to be used as  as `host`
servicebinding/secret/host: {.status.dbCredentials-myHost}

 # I want the secret `dbCredentials`'s `myPort` field to be used as is
servicebinding/secret: {.status.dbCredentials-myPort}

# I want the entire secret to be used as is
servicebinding/secret: {.status.dbCredentials}

# I want the entire configmap to be used as is
servicebinding/configmap: {.status.dbConfig}

@arthurdm @sbose78 Here are some revised examples of CR and ClusterServiceVersion to make sure we're all on the same page. I'm combining together a bunch of the ideas here.
Here's a CR status containing an array (I know this is an extension and I'll make a PR once we're happy):
status:
  bootstrap:
  - name: plain
    value: myhost1.example.com:9092,myhost2.example.com:9092
  - name: tls
    value: myhost3.example.com

Then I can use this notation in the CSV:
      statusDescriptors:
      - description: The addresses of the bootstrap servers for Kafka clients
        displayName: Bootstrap servers
        path: status.bootstrap
        x-descriptors:
        - servicebinding:secret:endpoints

And the SBO will make a secret containing:
endpoints.plain: myhost1.example.com:9092,myhost2.example.com:9092
endpoints.tls: myhost3.example.com

If I want to extract a field called password from a secret named in the status for a CR, I can do:
      - description: The secret containing the credentials
        displayName: Secret
        path: status.secret-password
        x-descriptors:
        - urn:alm:descriptor:io.kubernetes:Secret
        - servicebinding:secret:password

I could omit password from the last line because I don't want to rename the field as it's copied across. Also, I think path is being overloaded here - isn't one of the consumers of the CSV going to be an OLM UI and it probably wants to render the status.secret rather than just the element within it that binding wants? If I want to just pluck out a field from a secret and keep its name, I could do this:
      - description: The secret containing the credentials
        displayName: Secret
        path: status.secret
        x-descriptors:
        - urn:alm:descriptor:io.kubernetes:Secret
        - servicebinding:secret:password

This would allow me to take multiple fields just by adding more x-descriptors. If I use the path to name the fields, I need to specify an entry per field:
      - description: The secret containing the credentials
        displayName: Secret
        path: status.secret-user.p12
        x-descriptors:
        - urn:alm:descriptor:io.kubernetes:Secret
        - servicebinding:secret:client.p12
      - description: The secret containing the credentials
        displayName: Secret
        path: status.secret-user.password
        x-descriptors:
        - urn:alm:descriptor:io.kubernetes:Secret
        - servicebinding:secret:client.password

That I think would result in a UI rendering the secret multiple times because there are two entries for the secret.
The easiest option is not to permit renaming, but another would be to invent a syntax in the x-descriptor which has a source name and a target name, maybe servicebinding:secret[:source[:target]].
      - description: The secret containing the credentials
        displayName: Secret
        path: status.secret
        x-descriptors:
        - urn:alm:descriptor:io.kubernetes:Secret
        - servicebinding:secret:client.p12:user.p12
        - servicebinding:secret:client.password:user.password

From the secret named in status.secret, copy user.p12 renaming to client.p12 and user.password renaming to client.password.

Also, I can see how to specify that I want to use envvars, ConfigMap or secret for the binding information as the bindable service provider. I wonder if a similar syntax could be used not to specify the type of target and then let the ServiceBinding specify it.
servicebinding:secret    => put the binding information in a Secret
servicebinding:env       => put the binding information in environment variables
servicebinding:configmap => put the binding information in a ConfigMap
servicebinding:data      => add the binding information but let the ServiceBinding choose how it's rendered

In the Strimzi case, I'd prefer not to say in the CSV. The user might make the choice depending on the programming language of the binding consuming application or such, just for ease of use.
Just a thought.

Also, I think path is being overloaded here - isn't one of the consumers of the CSV going to be an OLM UI and it probably wants to render the status.secret rather than just the element within it that binding wants?

Yes, right, we can't be changing the meaning of the value of the path

+1 to not messing around with path.

So, this looks good to me

I agree with that sample too.  Basically, to rename items inside a Secret or a Config you must add the x-descriptors in the same order.  Otherwise, SBO will just pull in the Secret items as-is.

The user might make the choice depending on the programming language of the binding consuming application or such, just for ease of use

Right - that's some of the discussion happening in https://github.com/application-stacks/service-binding-specification/issues/15
The current winning thought is that Operators can express an opinion on how particular items are best consumed, but that can completely be overriden by the consumer via ServiceBinding CR.

Quoting..
status:
  bootstrap:
  - name: plain
    value: myhost1.example.com:9092,myhost2.example.com:9092
  - name: tls
    value: myhost3.example.com

Then I can use this notation in the CSV:
      statusDescriptors:
      - description: The addresses of the bootstrap servers for Kafka clients
        displayName: Bootstrap servers
        path: status.bootstrap
        x-descriptors:
        - servicebinding:secret:endpoints

And the SBO will make a secret containing:
endpoints.plain: myhost1.example.com:9092,myhost2.example.com:9092
endpoints.tls: myhost3.example.com

Curious, how do you plan to create a convention for the following?
"If bootstrap is a slice, then look for a "name"-"value" pairs" ?
How about using an existing model?
status:
  bootstrap:
     plain: myhost1.example.com:9092,myhost2.example.com:9092
     tls: myhost3.example.com

could then be annotated as
      statusDescriptors:
      - description: Theplain addresses of the bootstrap servers for Kafka clients
        displayName: Bootstrap plain servers
        path: status.bootstrap.plain
        x-descriptors:
          - servicebinding:secret:plain

      - description: Theplain addresses of the bootstrap servers for Kafka clients
        displayName: Bootstrap plain servers
        path: status.bootstrap.tls
        x-descriptors:
          - servicebinding:secret:tls:TLS <--- call it TLS in the binding secret ?

There are few things we could improve further:
Today the use of env:object:configmap is redundant because we already know from urn:alm:descriptor:io.kubernetes:ConfigMap that dbConfigMap is a configMap . We do not need env:object:configmap is supposed to tell that to SBO
- description: Name of the ConfigMap to hold the DB config
       displayName: DB Config Map
       path: dbConfigMap
       x-descriptors:
          - urn:alm:descriptor:io.kubernetes:ConfigMap <--- Already tells me its a configmap
          - binding:env:object:configmap:db.host. <--- tells me again that dbConfigMap is a configmap
          - binding:env:object:configmap:db.port

In the specification, I see that the use of servicebinding:secret:tls. By that, I understand that we imply that tls needs to land in a binding secret.
Example:
 statusDescriptors:
      - description: The plain addresses of the bootstrap servers for Kafka clients
        displayName: Bootstrap plain servers
        path: status.bootstrap.plain
        x-descriptors:
          - servicebinding:secret:plain

My question is,

Should we ask the operator author to define whether something needs to land in a binding configmap or a binding secret ? Or,
Should the only thing that the operator author be advising is

plain should be env var _ OR _
tls should be mounted as a volume.

How that gets used as an env var ( through a configmap or a secret ? ) or a volume mount is the consumer's problem.
So, in essence,
Operator author:  plain should be used as an environment variable
 statusDescriptors:
      - description: The plain addresses of the bootstrap servers for Kafka clients
        displayName: Bootstrap plain servers
        path: status.bootstrap.plain
        x-descriptors:
          - servicebinding:envvar:plain

When the operator author wants to convey ..
tls should be used as an environment variable_
- description: The tls addresses of the bootstrap servers for Kafka clients
        displayName: Bootstrap plain servers
        path: status.bootstrap.tls
        x-descriptors:
          - servicebinding:volume:tls:TLS

When the operator author wants to convey ..

username should be an env var
password should be mounted as a volume
url should be an env var and used as connectionURL in the secret/configmap

- description: The secret containing the credentials
        displayName: MyCredentials
        path: status.credentials
        x-descriptors:
        - urn:alm:descriptor:io.kubernetes:Secret    # let's the world know credentials is a Secret
        - servicebinding:envvar:username
        - servicebinding:volume:password
        - servicebinding:envvar:url:connectionURL

I like the proposal.
One thing we would have to document is that the ServiceBinding CR doesn't give you the same level of granularity in terms of picking env vs volume per item.  In the CR it's a global switch - which I think makes sense, so for example if your app was expecting everything in a volume you could do that globally.
One part I am trying to understand is the source:target notation.  It works really nice for objects (Secrets, ConfigMap, probably arrays too), but for single properties we were saying that something like this:
        path: status.myHost
        x-descriptors:
        - servicebinding:envvar:host

would mean that host was the target.   Maybe that's all OK, but in the proposal servicebinding:envvar:username was referencing the source, where-as in the sample above servicebinding:envvar:host is referencing the target.
Perhaps the spec would then have to distinguish what a single item in the source:target tuple means in terms of complex objects vs single properties.

@sbose78 The reason that I was keen to use the name-value notation for the bootstrap slice is that the set of keys is not known in advance. The user could create a new listener with a name of their choosing and I'd like the CSV to be annotated in a way such that this just works, so it cannot name the keys.
So, I was just aiming for a straightforward convention that the SBO could use to handle slices. Hence:
status:
  bootstrap:
  - name: plain
    value: myhost1.example.com:9092,myhost2.example.com:9092
  - name: tls
    value: myhost3.example.com

Maybe a hint of servicebinding:slice:name:value would help.
This is non-sensitive information and I could use servicebinding:envvar:endpoints or servicebinding:volume:endpoints in an x-descriptor with path status.bootstrap.
Then we get binding information:
endpoints.plain ---> myhost1.example.com:9092,myhost2.example.com:9092
endpoints.tls   ---> myhost3.example.com

which could be rendered in either a volume-mounted secret or envvars as appropriate.

Makes sense @AndrewJSchofield

I'll summarize this discussion shortly.

Thanks to @sbose78 for getting this done via PR #25 - the only things that remains is a new PR to address https://github.com/application-stacks/service-binding-specification/pull/25#pullrequestreview-390363182

Let's close this and track a separate issue for https://github.com/application-stacks/service-binding-specification/pull/25 so that it gets better feasibility and focus ?

Sure, sounds good to me.

Opened https://github.com/application-stacks/service-binding-specification/issues/26


[Impeller] mask blurs don't work for textures

reproduction
TEST_P(AiksTest, GaussianBlurStyleInnerTexture) {
  Canvas canvas;
  canvas.Scale(GetContentScale());
  Paint paint;
  paint.color = Color::Green();
  paint.mask_blur_descriptor = Paint::MaskBlurDescriptor{
      .style = FilterContents::BlurStyle::kInner,
      .sigma = Sigma(30),
  };
  std::shared_ptr<Texture> boston = CreateTextureForFixture("boston.jpg");
  canvas.DrawImage(std::make_shared<Image>(boston), {200, 200}, paint);
  Paint red;
  red.color = Color::Red();
  canvas.DrawRect(Rect::MakeXYWH(0, 0, 200, 200), red);
  ASSERT_TRUE(OpenPlaygroundHere(canvas.EndRecordingAsPicture()));
}

expected result
I expect the texture to be blurred.
observed result
The GaussianBlurFilterContents object isn't allocated.

We shouldn't use a Gaussian blur to handle mask blurring for anything drawn with rectangle/rrect/elliptical geometry regardless of the color source.

We shouldn't use a Gaussian blur to handle mask blurring for anything drawn with rectangle/rrect/elliptical geometry regardless of the color source.

I'll update the description to avoid talking about GaussianBlurFilterContents.

Here is an example of how skia handles this: https://fiddle.skia.org/c/d0be824c0999691bb7e62c446f7db0ce

notes from the chat:
bdero: BorderMaskBlurFilterContents was specifically made to be used for the DrawImageRect mask filter case long ago.
But today we should reuse the Aiks path I added later on that supports all of the color sources


example/run_generation.py fails with unexpected argument for TextGenerationStreamingResponse

System Info
Python 3.10.12
Requirements:
huggingface_hub==0.20.3
requests==2.31.0
datasets==2.18.0
transformers>=4.37.0
Information

[X] Docker
[X] The CLI directly

Tasks

[X] An officially supported command
[x] My own modifications

Reproduction

setup a virtual environment and activate it
navigate to tgi-gaudi/examples
run python3 run_generation.py against a functional server.
The program will exit with:
$ python3 run_generation.py --model_id meta-llama/Llama-2-70b-chat-hf --max_concurrent_requests 8 --total_sample_count 1000  --max_output_length 1024 --max_input_length 1024 None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. 0%|                                                                                | 0/1000 [00:00<?, ?it/s]Thread failed with error: TextGenerationStreamResponse.__init__() got an unexpected keyword argument 'index' $

Expected behavior
Expected behavior is to run the test and produce results, as it does with huggingface-hub==0.23.5.

@regisss @libinta I added a PR to address this.


Bump version to 0.2.4

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.
I do have some suggestions for making it better though...
For recipe:

Whenever possible python packages should use pip. See https://conda-forge.org/docs/maintainer/adding_pkgs.html#use-pip
license_file entry is missing, but is expected.

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.
I do have some suggestions for making it better though...
For recipe:

license_file entry is missing, but is expected.

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-webservice.
I tried to re-render for you, but it looks like there was nothing to do.

@conda-forge-admin, please rerender

It looks good now. Please just reset the build number to 0 since it is first built for this package version.

Great. :) @astaric @kernc can one of you merge this PR?

It says:

Only those with write access to this repository can merge pull requests.

:sweat_smile:

@conda-forge/core, can someone merge it?

@conda-forge-admin, please update team

@kernc, did you leave conda-forge?


remove jupyterlab-github from environment

jupyterlab-github requires additional setup after pip installation and isn't as generally useful as the jupyterlab_git package. The jupyter lesson doesn't rely on it, it's just mentioned as an optional extension

Mentioned also in the other repo because here issues tend to get lost. The other is more visible. But this is the right repo to track it in a sense.