BouncingScrollPhysics issue

This is a pagination case. If a user in overscroll (maxExtended - position < 0) while new data coming into the list and didn't unpin it's finger, CustomScrollView with BouncingScrollPhysics will kick you into the end of the list.
Flutter channel: stable.
Flutter version: v1.20.0
Example:
import 'package:flutter/material.dart';

void main() {
  runApp(MyHomePage());
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key}) : super(key: key);

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  final list = [0, 1, 2];
  var loading = false;
  final colors = [
    Colors.cyan,
    Colors.indigo,
    Colors.pink,
    Colors.red,
    Colors.blue,
    Colors.yellowAccent
  ];

  final sc = ScrollController();

  @override
  void initState() {
    super.initState();
    sc.addListener(_onScroll);
  }

  Future<void> loadMore() async {
    if(list.length < 6){
      await Future.delayed(const Duration(seconds: 3));
      list.addAll([3, 4, 5]);
    }
  }

  void _onScroll() {
    final maxExtended = sc.position.maxScrollExtent;
    final position = sc.position.pixels;
    final dif = maxExtended - position;
    if (dif == 0.0 && !loading) {
      loading = true;
      loadMore().then((value) {
        setState(() {});
        loading = false;
      });
    }
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: CustomScrollView(
        controller: sc,
        physics: BouncingScrollPhysics(),
        slivers: [
          SliverList(
              delegate: SliverChildBuilderDelegate(
                  (_, index) => SizedBox(
                      width: double.infinity,
                      child: Container(
                        height: 400,
                        color: colors[index],
                        child: Text(
                          index.toString(),
                          style: TextStyle(
                            fontSize: 30,
                          ),
                        ),
                      )),
                  childCount: list.length))
        ],
      ),
    );
  }

  @override
  void dispose() {
    sc.dispose();
    super.dispose();
  }
}

Video:

Hi @Renesanse,
I tried to replicate the code on latest Flutter dev and observed RangeError exception as below:

Exception:
Another exception was thrown: RangeError (index): Invalid value: Not in inclusive range 0..5: 6

════════ Exception caught by widgets library ═══════════════════════════════════════════════════════
The following RangeError was thrown building:
RangeError (index): Invalid value: Not in inclusive range 0..5: 6

When the exception was thrown, this was the stack:
#0      List.[] (dart:core-patch/growable_array.dart:177:60)
#1      _MyHomePageState.build.<anonymous closure> (package:hello_world/main.dart:2830:38)
#2      SliverChildBuilderDelegate.build (package:flutter/src/widgets/sliver.dart:448:22)
#3      SliverMultiBoxAdaptorElement._build (package:flutter/src/widgets/sliver.dart:1129:28)
#4      SliverMultiBoxAdaptorElement.createChild.<anonymous closure> (package:flutter/src/widgets/sliver.dart:1142:55)

I tested on Android 8.1.0 device.
Did you get this exception when you tried ?

Hi, @darshankawar. You don't need to scroll more than once to understand the issue, btw i can fix my example

@darshankawar in release 1.17 everything is fine. I tested in 1.20

@Renesanse - could you write a test case that fails for what you're talking about? Your application looks like it's probably pretty close, but it's a little hard for me to tell right now what assertion would/should fail there.

The animated screenshot above is working as intended, FWIW (it's definitely what you want if the cause is just the contents changing height; it's less clear if it's what you want in a case such as this where there's just more content). You should be able to change it by replacing the scroll physics to not include the RangeMaintainingScrollPhysics, though.

@Renesanse - could you write a test case that fails for what you're talking about? Your application looks like it's probably pretty close, but it's a little hard for me to tell right now what assertion would/should fail there.

Hello @dnfield, I guess good explanation will be is a during configuration of ScrollPhysics right now, new data, for example, in ListView will move user's scroll position directly to the bottom of whole ListView widget ignoring that user did not scroll anything or other activities to appear on the bottom. I sure this issue connected with this pr https://github.com/flutter/flutter/pull/65135

Hello @darshankawar,
after Flutter update from 1.20.4 to 1.22.0 I'm receiving the Range error as below:
flutter: ══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
flutter: The following RangeError was thrown building:
flutter: RangeError (index): Invalid value: Not in inclusive range 0..8: 9
flutter:
flutter: When the exception was thrown, this was the stack:
flutter: #0      List.[] (dart:core-patch/growable_array.dart:177:60)
flutter: #1      _SubArticlePageState.build.<anonymous closure> (package:wamii_app/pages/sub_article_page.dart:72:46)
flutter: #2      SliverChildBuilderDelegate.build (package:flutter/src/widgets/sliver.dart:449:22)
flutter: #3      SliverMultiBoxAdaptorElement._build (package:flutter/src/widgets/sliver.dart:1130:28)
flutter: #4      SliverMultiBoxAdaptorElement.createChild.<anonymous closure> (package:flutter/src/widgets/sliver.dart:1143:55)
flutter: #5      BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2683:19)
flutter: #6      SliverMultiBoxAdaptorElement.createChild (package:flutter/src/widgets/sliver.dart:1136:11)
flutter: #7      RenderSliverMultiBoxAdaptor._createOrObtainChild.<anonymous closure> (package:flutter/src/rendering/sliver_multi_box_adaptor.dart:350:23)
flutter: #8      RenderObject.invokeLayoutCallback.<anonymous closure> (package:flutter/src/rendering/object.dart:1883:59)
flutter: #9      PipelineOwner._enableMutationsToDirtySubtrees (package:flutter/src/rendering/object.dart:915:15)
flutter: #10     RenderObject.invokeLayoutCallback (package:flutter/src/rendering/object.dart:1883:14)
flutter: #11     RenderSliverMultiBoxAdaptor._createOrObtainChild (package:flutter/src/rendering/sliver_multi_box_adaptor.dart:339:5)
flutter: #12     RenderSliverMultiBoxAdaptor.insertAndLayoutChild (package:flutter/src/rendering/sliver_multi_box_adaptor.dart:485:5)
flutter: #13     RenderSliverList.performLayout.advance (package:flutter/src/rendering/sliver_list.dart:231:19)
flutter: #14     RenderSliverList.performLayout (package:flutter/src/rendering/sliver_list.dart:274:19)
flutter: #15     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
flutter: #16     RenderSliverEdgeInsetsPadding.performLayout (package:flutter/src/rendering/sliver_padding.dart:132:12)
flutter: #17     RenderSliverPadding.performLayout (package:flutter/src/rendering/sliver_padding.dart:371:11)
flutter: #18     RenderObject.layout (package:flutter/src/rendering/object.dart:1777:7)
flutter: #19     RenderViewportBase.layoutChildSequence (package:flutter/src/rendering/viewport.dart:507:13)
flutter: #20     RenderViewport._attemptLayout (package:flutter/src/rendering/viewport.dart:1561:12)
flutter: #21     RenderViewport.performLayout (package:flutter/src/rendering/viewport.dart:1470:20)
flutter: #22     RenderObject._layoutWithoutResize (package:flutter/src/rendering/object.dart:1634:7)
flutter: #23     PipelineOwner.flushLayout (package:flutter/src/rendering/object.dart:884:18)
flutter: #24     RendererBinding.drawFrame (package:flutter/src/rendering/binding.dart:436:19)
flutter: #25     WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:914:13)
flutter: #26     RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:302:5)
flutter: #27     SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1117:15)
flutter: #28     SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1055:9)
flutter: #29     SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:971:5)
flutter: #33     _invoke (dart:ui/hooks.dart:251:10)
flutter: #34     _drawFrame (dart:ui/hooks.dart:209:3)
flutter: (elided 3 frames from dart:async)
flutter: ════════════════════════════════════════════════════════════════════════════════════════════════════

The identical code deployed in production with 1.20.4 works perfectly but it generate exception with 1.22.0 and 1.22.1

Is that bug fixed on master/dev branch? We are waiting to release our app update due to this bug

Hello, you can try this fix for a while: https://github.com/flutter/flutter/pull/67178

Any solution or work around?


Multiple pipelines in single yaml

Hello,
either there is no such option and I would like to have it or I haven't seen it anywhere in the docs: is there an option to configure multiple pipelines in a single azure-pipelines.yml file? Use case: define several pipelines for branches, PRs, specific paths etc. and have VSTS automatically pick it up instead of manually clicking and importing them. I assume that a pipeline name and location option would be necessary for that.

Thanks for your question. It looks like you're working through an issue with your scenario or implementation, rather than an issue with the documentation. Here are a couple of options where you might consider asking your question:

Azure DevOps on Stack Overflow
Azure DevOps Support Bot


How you guys inject character identity?

Hello, thanks for your great work!
I would like to ask how you guys inject character identity? In the demo, the identity looks pretty good.
Any response will be greatly appreciated!

NVM I ignore the paper. Dreambooth.


Update v1.3 feature about new command owe

The new command owe is the command that checks the amount of money each person should submit.
The format of the new command is: owe INDEX m/MONEY e.g. owe 3 m/20.
The Money part can maximum support 2 decimal digits.
**Original graph: **

**After command: **

**Possible invalid commands handle: **

Check fail due to some UG format

Can you open an issue and link it to that?

Can you open an issue and link it to that?

Okay done


feat: add Column.to_table

Closes #695
Summary of Changes
Add method Column.to_table to get a Table from a Column.

:tada: This PR is included in version 0.23.0 :tada:
The release is available on:

v0.23.0
GitHub release

Your semantic-release bot :package::rocket:


[Bug]: pip dependency issue (evaluate_models_in_vertex_ai_studio_and_model_garden)

File Name
gemini/evaluation/evaluate_models_in_vertex_ai_studio_and_model_garden.ipynb
What happened?
When running the notebook for the first time the following error appears:

Running the cell for the second and consecutive times doesn't cause the error anymore.
Relevant log output
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Looks like a dependency conflict for the evaluation SDK requirement. Creating a fix for this.

Fix is submitted and the release will be in one week.


Fixes silly error in importing pathos

This should fix a silly error in importing pathos caused by a missing comma.
Sorry about the inconvenience, and please merge once we confirm that tests work.

merged!


Global Filter is ignored when using Query Builder

Describe the bug
It seems the "applyFilters" logic that happens when using EntityManager.find does not appear to exist when using the query builder.
Stack trace
No stack trace needed

To Reproduce
Steps to reproduce the behavior:

Make global Filter orm.em.addFilter("tenant",(args: any) => ({}))
Query the model with query builder (await orm.em.createQueryBuilder(ModelToFilterWithGlobalFilter).getResultList()

Expected behavior
Same behavior as EntityManager.find
Additional context
const temp = await orm.em.createQueryBuilder(TModel).getResultList(); // does not fire global filter
const temp2 = await orm.em.find(TModel, {}); //does fire global filter
Versions

Dependency
Version

node
irrelevant

typescript
irrelevant

mikro-orm
5.1

your-driver
postgres

Yes, because filters are only handled by EM, with QB things need to be explicit.

@B4nan Hey! Is there any workaround for auto filters when using filters? (Because not all queries are available with EM.find)

Not tested, but something like this might work:
const qb = ...;
const filters = em.applyFilters('EntityName', {}, {}, 'read');
qb.andWhere(filters);

@B4nan, Is there any way to get global filters from EM and apply them to QB? Or maybe an option on createQueryBuilder() to apply global filters.
IMO, the fact that they are global makes them different from other filters, and it's better to apply them or at least give the option to apply them on QB, too.

Its a sync method, to apply filters you need async context as they can be async. You can use em.applyFilters() to do this.

Yeah, I ended up doing that.
Also, I think I found a bug on filters; relation filters do not apply on the count query of the findAndCount() method. (In the  JOIN ON condition of the query)
I will test it further and create a new issue for it.

I see, looks like the regular filters work, but not the filters on relations.

I will wait for your repro, but after spending a few hours looking into the count and filters problem, I feel like it actually works correctly. Filters on relations are not considered for the where condition (unless you use populateWhere: infer), so it has no effect on the root entity count. I just got it implemented and realized that it only complicates the query without any outcome, as you get left joins that cannot affect the root entity count.


:sparkles: Consider 'src/test' test directories

What kind of change does this PR introduce?
Enhancement: improve the quality of the default scoring

[x] PR title follows the guidelines defined in our pull request documentation

What is the current behavior?
Files in testdata directories are ignored by default, but files in src/test directories are not.
What is the new behavior (if this is a feature change)?**
src/test is also recognized as a test directory

[x] Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes
NONE
Special notes for your reviewer
This directory is found in projects that follow the Maven Standard Directory Layout, which is fairly widely used, not only in Maven but also in adjacent ecosystems like Gradle and clojure.
Longer-term this would become part of the default policy, see discussion at https://github.com/ossf/scorecard/pull/1408#issuecomment-999806097 etc.
Does this PR introduce a user-facing change?
For more information on release notes see: https://git.k8s.io/release/cmd/release-notes/README.md
`src/test` directories are now considered test directories by default.

(the e2e/sast_test.go failure seems to be a quota thing, but the e2e/fuzzing_test.go failure might be caused by this change - will dig in later)

(the e2e/sast_test.go failure seems to be a quota thing, but the e2e/fuzzing_test.go failure might be caused by this change - will dig in later)

can you rebase on main, or merge in the updates? As far as I know, the failing e2e test is due to a fuzzing e2e test which has been temporarily skipped since your branch was created

can you rebase on main, or merge in the updates? As far as I know, the failing e2e test is due to a fuzzing e2e test which has been temporarily skipped since your branch was created

Ah, thanks, that indeed seems to have done the trick. The unit tests fail because of an intermittent error:
time="2023-03-02T22:53:41Z" level=error msg="GitHub token env var is not set. Please read https://github.com/ossf/scorecard#authentication" error="an error occurred while getting GitHub credentials"
time="2023-03-02T22:53:42Z" level=info msg="Rate limit exceeded. Waiting 28m1.981501491s to retry..."

Trigger a new build for that by force-pushing without further changes.


use callout without custom jekyll plugin

use callout without custom jekyll plugin

:link: https://github.com/twbs/bootstrap/pull/25319

@XhmikosR Can I support or help you with something to push this thought the finish line?

@m5o: I haven't forgot about it, I just haven't got the time to review your PRs yet.
Bear with me, and I'll get to it soon-ish :)

@XhmikosR I'am very close to present a solution to get rid of the last custom plugin for building the toc. This PR is currently blocking me. I have to do minor markup change in the callout includes and don't want to do it twice. Please don't forget me :o)

I won't. But personally, I'm against dropping the Jekyll plugin for TOC.
We will never build on GH automatically so it makes no sense. What it makes sense is to try to speed things up if possible.


[ms-quic] Build error

Host Environment

OS: [e.g. Windows/Linux etc...]
Compiler: revision
windows 10 + VS 2022
To Reproduce
Steps to reproduce the behavior:
vcpkg install ms-quic

Failure logs
vcpkg install ms-quic
Computing installation plan...
A suitable version of cmake was not found (required v3.22.2). Downloading portable cmake v3.22.2...
Downloading cmake...
  https://github.com/Kitware/CMake/releases/download/v3.22.2/cmake-3.22.2-windows-i386.zip -> G:\vcpkg\downloads\cmake-3.22.2-windows-i386.zip
Extracting cmake...
The following packages will be built and installed:
    ms-quic[core]:x86-windows -> 1.2.0
Detecting compiler hash for triplet x86-windows...
A suitable version of git was not found (required v2.35.1). Downloading portable git v2.35.1...
Downloading git...
  https://github.com/git-for-windows/git/releases/download/v2.35.1.windows.2/PortableGit-<IP_ADDRESS>-32-bit.7z.exe -> G:\vcpkg\downloads\PortableGit-<IP_ADDRESS>-32-bit.7z.exe
Extracting git...
Restored 0 packages from C:\Users\xx\AppData\Local\vcpkg\archives in 267.4 us. Use --debug to see more details.
Starting package 1/1: ms-quic:x86-windows
Building package ms-quic[core]:x86-windows...
-- Using cached microsoft-msquic-v1.2.0.tar.gz.
-- Cleaning sources at G:/vcpkg/buildtrees/ms-quic/src/v1.2.0-19ce393c24.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source G:/vcpkg/downloads/microsoft-msquic-v1.2.0.tar.gz
-- Applying patch fix-warnings.patch
-- Applying patch fix-platform.patch
-- Applying patch fix-install.patch
-- Using source at G:/vcpkg/buildtrees/ms-quic/src/v1.2.0-19ce393c24.clean
-- Using cached quictls-openssl-a6e9d76db343605dae9b59d71d2811b195ae7434.tar.gz.
-- Extracting source G:/vcpkg/downloads/quictls-openssl-a6e9d76db343605dae9b59d71d2811b195ae7434.tar.gz
-- Using source at G:/vcpkg/buildtrees/ms-quic/src/b195ae7434-aa88ae7c62.clean
-- Found external ninja('1.10.2').
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Building x86-windows-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:158 (message):
    Command failed: G:/vcpkg/downloads/tools/cmake-3.22.2-windows/cmake-3.22.2-windows-i386/bin/cmake.exe --build . --config Debug --target install -- -v -j17
    Working Directory: G:/vcpkg/buildtrees/ms-quic/x86-windows-dbg
    See logs for more information:
      G:\vcpkg\buildtrees\ms-quic\install-x86-windows-dbg-out.log

Call Stack (most recent call first):
  installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake:111 (vcpkg_execute_build_process)
  installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_install.cmake:41 (vcpkg_cmake_build)
  ports/ms-quic/portfile.cmake:64 (vcpkg_cmake_install)
  scripts/ports.cmake:145 (include)

Error: Building package ms-quic:x86-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `git pull` and `.\vcpkg update`.
Then check for known issues at:
  https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+ms-quic
You can submit a new issue at:
  https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=[ms-quic]+Build+error
including:
  package: ms-quic[core]:x86-windows -> 1.2.0
    vcpkg-tool version: 2022-02-11-dac008bf9b55ef48e10bcc81e273b97a82bbf72d
    vcpkg-scripts version: bd602277b 2022-03-04 (2 days ago)

Additionally, attach any relevant sections from the log files above.

Please provide G:\vcpkg\buildtrees\ms-quic\install-x86-windows-dbg-out.log first.
Thanks!

install-x86-windows-dbg-out.zip

got it , thank you


Follow IncludeFile paths

Have to tool check IncludeFile paths as well as files provided as arguments.

In v0.0.7, the tool is only checking IncludeFile paths relative to the current directory:
$ ls test.txt test-includes
test.txt

test-includes:
test-include.txt

$ cat test.txt
Title Google
URL https://google.com

Title Yahoo
URL https://yahoo.com

IncludeFile test-includes/test-include.txt
$ cat test-includes/test-include.txt
Title Bing
URL https://bing.com

$ ./ezproxy-config-lint_Linux_x86_64-0.0.7/ezproxy-config-lint -annotate test.txt
test.txt:1: Title Google
test.txt:2: URL https://google.com
test.txt:3:
test.txt:4: Title Yahoo
test.txt:5: URL https://yahoo.com
test.txt:6:
test.txt:7: IncludeFile test-includes/test-include.txt
test-includes/test-include.txt:1: Title Bing
test-includes/test-include.txt:2: URL https://bing.com

But not the one in which the config.txt file is located:
$ cd ezproxy-config-lint_Linux_x86_64-0.0.7
$ ls
ezproxy-config-lint  LICENSE.md  README.md
$ ./ezproxy-config-lint -annotate ../test.txt
../test.txt:1: Title Google
../test.txt:2: URL https://google.com
../test.txt:3:
../test.txt:4: Title Yahoo
../test.txt:5: URL https://yahoo.com
../test.txt:6:
../test.txt:7: IncludeFile test-includes/test-include.txt
Error encountered when processing line "IncludeFile test-includes/test-include.txt".
test-includes/test-include.txt resolves to the absolute path /tmp/ezproxy-linter/ezproxy-config-lint_Linux_x86_64-0.0.7/test-includes/test-include.txt.
Error processing ../test.txt: open test-includes/test-include.txt: no such file or directory

Try the  -include-file-directory parameter. If the IncludeFile path isn't absolute, the linter will first join the provided path to the paths in the config file.
In your example, I think the command would be ./ezproxy-config-lint -include-file-directory .. -annotate ../test.txt.
Appreciate the feedback and the detailed issues.

@seththeriault You were right, it didn't make a lot of sense to look in the current working directory for IncludeFile paths. In 0.0.9, the linter joins the config file's directory with the IncludeFile path, unless the path is absolute in the config file or the -include-file-directory parameter is used.


Missing device configuration: 0x200 and neo cool cam

Checklist

[X] It is not in the configuration DB

[X] It was not merged recently or has a pending PR

Which device is missing?
Cool cam NAS PD01
Manufacturer ID
0x0258
Product Type
0x0200
Product ID
0x1036
Firmware Version
3.3.0
Is the device listed on the Z-Wave Alliance website?
No response
Do you have a manual?
No response

This needs some more information to write the config file. At the very least a manual.

This does exist: https://devices.zwave-js.io/?jumpTo=0x0258:0x0200:0x1036:0.0


Add option to allow selecting of duplicate multi value options

Todo

[x] Add option on FE
[x] Add option on BE
[x] Apply it to User
[x] Apply it to Select
[x] Apply it to Record
[ ] Test FE with BE

vue-select doesn't allow duplicate multi values

Update: BE has already been added by Denis. Vivek is currently researching why the option (isUniqueMultiValue) is disabled by default. We want it to be enabled => values should not be unique by default

Update: FE will sync with the default value of isUniqueMultiValue on BE

Tested and confirming that everything works as expected.

Added CL:
Added:

{
      "what": [
        "option to allow duplicate values for single and multi value user, record and select fields."
      ],
      "why": [
        "to allow user to have more control over their selections and whether they will be allowing duplicate values or not."
      ],
      "refs": [
        "https://github.com/cortezaproject/corteza/pull/1136/commits/9a2bc8c31aa6623a4f525711722ea09e24bfb3f2"
      ]
    }


Reduce info[] use

Ticker.info contains a lot of data, but some of it is accessible other faster ways:

recent price data - Ticker.history() or download()
exchange, timezone, currency - Ticker.get_history_metadata()
share count (#1133 will prioritise merge)
market cap (= price * share count)

I often see users accessing these items, including some large projects that should know better, but info is ~a relatively~ the most expensive request and may be contributing to Yahoo encrypting data. So I propose deleting these items from info, with accompanying documentation / warnings etc for a smooth transition.
Open to contrary views.

Caveat: I have limited familiarity with the architecture of yfinance; I recently switched over to it from my own yfinance-ish library when the previous cryptography issues hit a month or so ago. I'm assuming that a lot of the architecture is similar to how I built it, as it was dictated by how Yahoo APIs offered its data.
If someone is grabbing current price, previous close, and a handful of other non-pricing data points from info , wouldn't this change require to now call info AND history() and result in more load on Yahoo?
It seems this change would only yield improvement if the only reason that the person was calling info was to only get pricing data.

Basically history and download are fast, everything else is slower - info is the worst at 6.5x slower. Also you can call history 100-1000 times in a loop before being temporarily blocked, but info[] much less (maybe 10-100 times?).

the person was calling info was to only get pricing data.

I've seen this in that large project. For individual users I often see them wanting just marketCap.

Got it. But isn't the proposed change to actually remove it from info? So for those who are being deliberate and not wasteful, this will increase the load / number of calls? For example, I will now have to call history() every time I currently call info as my goal when I call info is to get the summary-level data on a single security which info provides, not to get deep history on pricing.
It seems like the real solution is that the people who are being wasteful just need to not be wasteful :) Unless I'm missing something, this change will tangibly increase server load for those who ARE being resource-conscious, so we'd be going in the wrong direction.

The number of calls is less important than the amount of work those calls triggers. For whatever reason, info is ~6.5x more costly to Yahoo than history.
Another example: the proposed new way for getting market cap, using history and share count, is 3x faster despite 2 calls instead of 1.

There's little room for optimization since info is such a big chunk of data (but Yahoo only offers it in its entirety), and the initial yf.info call is made to do the retrieval, yfinance has no idea which of the 300 metrics are sought by the caller. It could be all 300 metrics. It could be current px.
What may be ideal for this goal is if it was possible to define exactly which metrics are being sought when making the initial info call (or any yf.* call for that matter). Then if were just info.(pricing data), it can source it from the most efficient API. Just to illustrate:
my_metrics = [ "regularMarketPreviousClose", "marketCap" ] yf.info(my_metrics)
could be processed differently (more efficiently) under the hood vs.
my_metrics = [ "regularMarketPreviousClose", "marketCap", "payoutRatio", "numberOfAnalystOpinions", "operatingMargins" ] yf.info(my_metrics)
Probably a far more substantial change though.

The number of calls is less important than the amount of work those calls triggers at Yahoo. For whatever reason, info is ~6.5x more costly to Yahoo than history.
Another example: the proposed new way for getting market cap, using history and share count, is 3x faster despite 2 calls instead of 1.

I think I must misunderstand this then. Taking my second code block above, I see the amount of work done by Yahoo as one info call (6.5 units). I see the  amount of work done after this proposed change as being the cost of history() call (let's say 1 unit) + one info call (6.5 units).
Current 6.5 units vs. Future 7.5 units.
What am I missing? Is this proposed change only to ALSO make market cap available from history() but not to remove it from info?

90% of the time spent fetching & constructing info in yfinance, is spent only on the web request itself. There is only one web request. All of the decrypting and parsing is in the remaining 10%.

You're missing that most users won't need info anymore, because they only want market cap, or last price, or currency, which can be provided without info faster.

You're missing that most users won't need info anymore, because they only want market cap, or last price, or currency, which can be provided without info faster.

Are you saying that the answer to my last question...

Is this proposed change only to ALSO make market cap available from history() but not to remove it from info?
is "yes"?

Then this would all make sense. I thought the intention was to actively remove these data points from info even though they're already coming in summary data provided from Yahoo on that API request.

Yes
Delete those specific keys from info, deliberately breaking userscode so they switch. But also provide clear documentation, including a message when importingyfinance`, so transition is as painless as possible.

Ok that sounds like it's actually no then haha :) See my comparison above where I say that the cost is going to 6.5 to 7.5.
This change may create efficiency for others, but it will increase my load on Yahoo. I don't have a clue what the usage patterns are; I certainly defer to you there. If my load needs to increase so that 99% of other yfinance users decrease, I can get behind that. I also share your concern that these recent client-side encryptions are very clearly trying to get API folks off of their servers.
I would like to propose an alternative approach though. If there are users taking tiny bits of data from info, they undoubtedly observe that it's brutally slow. Perhaps adding another call such as yf.basic_market_data and having it include these basic pricing and descriptive metrics and be retrieved under the hood from history() or whatever other API is most efficient.
Or - as I had done in my old custom version of yfinance and refactored to surround yfinance when swtiching over - embed throttling + back-off logic underneath info / summary requests to Yahoo to ensure there is no pounding their servers or getting blocked.

yf.basic_market_data

Good suggestion, and probably the best option. Only challenge is ensuring most/all affected users know & switch.

Yep. I literally was just editing my comment to point out that the downside to this is that there's no disincentive for users who currently misuse info for this purpose, but are not put-off enough by its slowness haha
As I mentioned, I'm new to the yfinance community so I have no idea whether light communication is enough to motivate the majority.
If not, perhaps just deliberately break pricing in info by renaming the fields in question to make them touch their code and also call out that it's available on this new faster API? Such as renaming regularMarketPrice to basicMarketPrice to indicate that this is a field that's best retrieved from this new API? IIRC, regularMarketPrice is the exact term used by Yahoo in the summary data, but it's not really a good name anyways.

Do you mind trying branch feature/prune-info? Hasn't implemented basic_market_data, just tells users to use other methods.

Basically history and download are fast, everything else is slower - info is the worst at 6.5x slower. Also you can quickly call history 100s-1000s times in a loop before being temporarily blocked, but info much less (maybe 10-100 times?).

the person was calling info was to only get pricing data.

I've seen this in that large project. For individual users I often see them wanting just market cap.

I would like a way to just get the market cap for an individual or group of stocks. I only run this quarterly, but at the moment my code that snags info['marketCap'] doesn't work and comes back with no data.

I just checked out that branch and ran a task that would take a lot of fields from info. It fails with the following error on AAPL:
AAPL failed on:  Object of type LazyLoadInfo is not JSON serializable
So this is a breaking change and it's not entirely apparent to the user why that is. Because python is quite fickle on data types, it's probably not a great idea to swap in this new object which seems like it may have different accessors.
I'm not sure what benefit would come from doing any kinda lazy-loading since the price of retrieving the data comes regardless of whether you take 1 field from info or all fields.

@cmjordan42 I've now removed the lazy-loading stuff. I only thought it would help in cases like i = dat.info ; i["marketCap"] - they might leave the first line. But your error obviously means not worth effort.

Gotcha. If you have a clear idea on how you'd implement that basic_market_data call, I think that it's still worthwhile to get out there sooner even without yet having a fool-proof strategy to motivate people to switch over to it. The responsible/attentive people who seek just that data from yfinance would probably hugely appreciate its performance improvements.

And those who adopt it instead of info will still impact the bottom-line on reducing load on Yahoo.

I attempted to switch my info[“marketcap”] call today to Ticker.shares but ran into another issue. I see the fix is in 0.2.4 but pip install yfinance couldn’t find it yet. I’ll watch for it as I imagine it takes some time for the packages to flow to their repositories.LLSent from my iPadOn Jan 16, 2023, at 8:45 PM, Chris Jordan @.***> wrote:
And those who adopt it instead of info will still impact the bottom-line on reducing load on Yahoo.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

implement that basic_market_data call

Done. See Ticker.basic_info, acts like a dict so should be easy to understand.

Awesom, thank youSent from my iPhoneOn Jan 17, 2023, at 8:13 AM, ValueRaider @.***> wrote:
implement that basic_market_data call
Done. See Ticker.basic_info, acts like a dict so should be easy to understand.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

implement that basic_market_data call

Done

I just looked to try to check it out to try to help and give it a test drive, but I can't find the new call available in master. Do you mean you have it done but still on a local branch? If so, lmk if/when you'd like someone to test drive it.

Branch feature/prune-info

I am doing an analysis of about 800 stocks on a daily basis. Previously it took about 2.5 hours to download all the information. To speed up, I started to use the cashing as described on the homepage of the yfinance project. Now the full analysis is done in less then 15 min.

Just tested basic_info - thank you! It works well. Be great if  you merge this into main branch.

@gene-git I fast-tracked it today, it's in latest PIP release.

Awesome thank you - I build Arch package - I'll go rebuild and install.

@gene-git wait, there's a bug pending #1340

ok no worries thanks

You're missing that most users won't need info anymore, because they only want market cap, or last price, or currency, which can be provided without info faster.

nope mate, there were ratios in info data that we used. How can it be get now? (debttoequity, returnonequity, margins, quickratio, pricetobook, pegratio etc.)

"Mate", I only deleted those info keys that can be reproduced faster with other data sources. Rest of info remains.

I'm confused - I thought we all agreed that nothing should be deleted from info? Will I be unable to upgrade past 0.2.4?

Agreement was on introducing basic_info data structure. Users still need to be forced away from unnecessary use of info, to easy Yahoo demand. Because: the performance incentive to move away always existed (except for 'market_cap'), e.g. history() for the last price, so users were too lazy to switch.
Why can't you upgrade? basic_info has been thoroughly tested to match info. Other users like it.

Basic_info is really basic.  There is neither balance sheet nor income statement in new version.  unfortunately not usable for me anymore.

@0zd3m1r Read #1329 . That data hasn't been removed.

Agreement was on introducing basic_info data structure.

Okay, I misunderstood the conclusion of our prior conversation.
But it sounds like ozd3m1r may just be confused. I'll just have to look much more closely before I do this upgrade.

Are data points that were previously available on info in 0.2.3 no longer there in 0.2.5?

You are confused. Your data was never removed. The problem now is decryption broken by a Yahoo change. Data still available in encrypted form.

@cmjordan42 https://github.com/ranaroussi/yfinance/issues/1300#issuecomment-1404890089

You're missing that most users won't need info anymore, because they only want market cap, or last price, or currency, which can be provided without info faster.

I guess I didn't understand it clearly.

@cmjordan42 #1300 (comment)

I just looked at the changes for the 0.2.5 release and it seems reasonable to me. Will see if I find any surprises when I get a chance to go update, probably later today.

i think it would be useful to add to basic_info the sector and the industry like info in previous version. i understand that these informations may not be used widely but they seem basic enough to be in basic_info

@leoBitto The condition for entry is having an alternative data source available, not usage. Where are you finding those fields?

when i called info on the ticker i was looking for. for example:
ticker = yf.Ticker("A")
info = ticker.info

print(info['sector'])
print(info['industry'])

i may have misundestood what are the changes that you are doing

@leoBitto Read the top post.

@leoBitto Does renaming to fast_info make sense?

@ValueRaider yes now it makes more sense. i completely misunderstood what were your intentions. thank you for your work

please leave basic_info name - fast is less informative of what it does (fast is not what it does but how well it does it)
-Not a huge deal to change code but also simpler to leave it - its better to simply  document basic_info being fast.

I've already renamed and shipped out. basic_info will still work too, just prints a slightly annoying message.
I was seeing too many users confidently misunderstanding from the name. And "fast" is its purpose.

ok no worries - minor change to code - no biggie.
thanks for staying on top of this.

I just had a chance to start looking at the impact of the changes... and it's a total disaster. For the task that I mentioned above which collects all of the summary level statistics on a given security, it is entirely broken and this is what the debugging output is now...
1 of 1 AAPL
Price data removed from info. Use Ticker.fast_info or history() instead
Exchange data removed from info. Use Ticker.fast_info or Ticker.get_history_metadata() instead
Exchange data removed from info. Use Ticker.fast_info or Ticker.get_history_metadata() instead
Exchange data removed from info. Use Ticker.fast_info or Ticker.get_history_metadata() instead
Symbol removed from info. You know this already
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Exchange data removed from info. Use Ticker.fast_info or Ticker.get_history_metadata() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Market cap removed from info. Use Ticker.fast_info instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead
Price data removed from info. Use Ticker.fast_info or history() instead

These error messages are totally useless in finding out which field is being accessed and causing this message to come back. It would have been so nice to echo back the key, and just as easy to also indicate which exact new field that you've created under fast_info is the replacement (as their names were not maintained).
I'm now going cry for 15 minutes then start hunting down fields one by one to switch their calls and for no actual benefit :(

@cmjordan42 Or wait for a fix? #1360

#1360 seems to be a very specific issue, not related to what I'm encountering?
So far, I've found that my issues are largely relating to this new fake Dict type not behaving as Dict does
i.e.
y.info.get('regularMarketPrice', 'error')

whereas get() is fine for a Dict, it's unsupported on the fake Dict so needs to be the following (and I'll need
y.fast_info['last_price']

i.e. it cannot be marshalled directly to json so enabling debugging logging of the data returned by yfinance errors.
fp.write(json.dumps(yfinfo, indent=4))
TypeError: Object of type InfoDictWrapper is not JSON serializable

The get() is simple.
For serializing, is a 'toJSON' method good enough? Easier than extending JSONencoder
Both implemented now btw.

I've already addressed workarounds to these; need to get this task running again as all of the data I'm running on is a couple days stale at this point.
toJSON() would be fine as far as I'm concerned.

#1360 seems to be a very specific issue, not related to what I'm encountering?
So far, I've found that my issues are largely relating to this new fake Dict type not behaving as Dict does
i.e.
y.info.get('regularMarketPrice', 'error')

whereas get() is fine for a Dict, it's unsupported on the fake Dict so needs to be the following (and I'll need some additional logic to detect key-value errors)
y.fast_info['last_price']

i.e. it cannot be marshalled directly to json in dumping output for debugging.
fp.write(json.dumps(yfinfo, indent=4))
TypeError: Object of type InfoDictWrapper is not JSON serializable

You can try it
fp.write(json.dumps(yfinfo, default=vars))

New here, and noticed the break on monthly .info process today.  Upgraded to 0.2.9.  Per this topic, I do see the move of some fields that I retrieved with .info away to .fast_info.  Appreciate all the work here, but some feedback from this user on this change:
a) Must there be mutual exclusivity between .info and .fast_info?
I was surprised to see (what I assume are) more static fields moved to .fast_info (e..g 'currency', 'exchange', 'timezone').  I see the relevance of these associated with the other .fast_info fields, but since these are more of a static nature, would it be viable to have fields like these replicated in .info without violating your goals?
b) surprised to see apparent name changes for fields moved from .info (e.g. 'marketCap' in .info is now 'market_cap' in .fast_info  Small and perhaps naiive question: is there a reason camelCase is not used in .fast_info?

The mutual exclusivity is to encourage users to switch, to reduce demand on Yahoo (info is very expensive).
Case change - no good reason. Release was fast-tracked and case overlooked. Will enable camel-case very soon.

@sanyearng Camel case now available in branch fix/fast-info-camel-case. Can you try it? Instructions here #1080

@sanyearng Camel case now available in branch fix/fast-info-camel-case. Can you test it? Instructions here #1080

Tested that and .fast_info does produce camelCase now, thanks.  One potential problem, I did notice that after the update in this version with camelCase, .fast_info no longer retrieves keys and values (as dict or pseudo-dict) with code, such as the following
tickerdata = yf.Ticker("IBM") for k, v in tickerdata.fast_info.items():     print(k, v)
Error now (but not previously, is "AttributeError: 'FastInfo' object has no attribute 'public_keys'"
Related to the subject of dictionaries or pseudo-dictionaries, the following is the availability status of the dictionary-like attributes:
For .info: .items() [now produces error], .keys() [type list], .values() [type collections.abc.ValuesView] are all available
For .fast_info: .items() no longer available, .keys() available, .values() is unavailable
For .get_history_metadata(): .items() [type dict_items], .keys() [type dict_keys], .values() [type dict_values] are all available
From only my simple perspective, it would be great if all these areas behaved uniformly as dictionaries or pseudo-dictionaries, if that is possible.

For .info: .items() [now produces error], .keys() [type list], .values() [type collections.abc.ValuesView] are all available

Correction: .items() [type collections.abc.ValuesView], .keys() [type dict_keys], .values() [type collections.abc.ValuesView] are all available

items() should have been broken, but I have just pushed a fix.

items() and values() now fixed/added

items() and values() now fixed/added

I see items() and values() for .fast_info, along with keys()... all are type=lists. Thanks.
Last comment on the following:

The mutual exclusivity is to encourage users to switch, to reduce demand on Yahoo (info is very expensive).
Just to clarify my suggestion if it wasn't obvious: I had understood the goal above, and thought that the concern was with "expensive" frequent calls to .info for more "live" data (i.e. highly changeable as a function of time).  I suggested redundantly/additionally adding the more static fields of .fast_info ('currency', 'exchange', 'timezone') additionally to .info in order to
a) cut the need for static data (non-pricing) calls to .info from having to additionally tap .fast_info for those common static fields, while
b) presumably not increasing the likelihood that data calls for "pricing/live" data would ever want to call .info (as none of those fields have moved).

The mutual exclusivity is to encourage users to switch, to reduce demand on Yahoo (info is very expensive).

Apologies... formatting error in reference.  First sentence is reference, rest is my comment.  Copied below for certainty:
Just to clarify my suggestion if it wasn't obvious: I had understood the goal above, and thought that the concern was with "expensive" frequent calls to .info for more "live" data (i.e. highly changeable as a function of time). I suggested redundantly/additionally adding the more static fields of .fast_info ('currency', 'exchange', 'timezone') additionally to .info in order to
a) cut the need for static data (non-pricing) calls to .info from having to additionally tap .fast_info for those common static fields, while
b) presumably not increasing the likelihood that data calls for "pricing/live" data would ever want to call .info (as none of those fields have moved).

info is just a dict, not lazy-loading like fast_info. For info to exist at all requires the expensive web request - adding/replacing data would trigger the fetch. Once info is created, no further web requests are made.
Maybe info should be lazy-loading too, but that feels like an even bigger bug-prone job than fast_info.

Is there a fast way to get longName or shortName rather than use info[]?LukeSent from my iPadOn Jan 27, 2023, at 7:17 PM, ValueRaider @.***> wrote:
info is just a dict, not lazy-loading like fast_info. For info to exist at all requires the expensive web request - adding/replacing data would trigger the fetch. Once info is created, no further web requests are made.
Maybe info should be lazy-loading too, but that feels like an even bigger bug-prone job than fast_info.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

@ValueRaider :  I'm facing an issue with obtaining enough historical data on share count, which means that the resulting market cap (market cap = price * share count) data won't have enough historical data.
Could you please provide some guidance on this matter.

@ValueRaider is the stock.info  a single endpoint that returns all those 50+ datapoints and there is no way to make a single request per variable, or set conditions that only returns what you need?
Ex: I want the sector, industry, marketcap, totalCash of a stock
stock.info["sector"]
stock.info["industry"]
stock.info["totalCash"]
stock.info["marketcap"]

@pineapplentrees yfinance scrapes webpages (except price data) so it's everything or nothing.

info dict is fixed and fast_info deprecated in release 0.2.17


i18n(ja): Update manual-setup.mdx

Description

Update manual-setup.mdx.

diff

https://github.com/withastro/starlight/pull/2087
https://github.com/withastro/starlight/pull/1255

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

ja
manual-setup.mdx
Localization changed, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.


Simpler CURIE prefix expansion is broken

Looks like IRI's with a fully expanded URL (eg, http://purl.obolibrary.org/obo/GO_0005829) have their prefix removed & re-added without the ":".  So, as written, an IRI with a value of "http://purl.obolibrary.org/obo/GO_0005829" will get transformed into "http//purl.obolibrary.org/obo/GO_0005829" when decoded from JSON.  If the IRI value is collapsed (eg, NCBIGene:558), then it gets transformed properly into "http://www.ncbi.nlm.nih.gov/gene/558".  I think the following line needs to have a ":" added to it.
https://github.com/NCATS-Tangerine/cam-kp-api/blob/ec9256813e0a516924273ddbaafc4c43c200950d/src/main/scala/org/renci/cam/Implicits.scala#L19

D'oh!


Color Picker Converter made responsive

Fixed- #854
Made site responsive
Screenshots
Earlier:

NOW:

@Vishal-raj-1 @urvashi-code1255 @harshgupta20 @991rajat @ridsuteri Please view my PR

Please add labels to my PR
@Vishal-raj-1 @urvashi-code1255 @harshgupta20 @991rajat @ridsuteri


Revamping building and release with dagger multibuild pipeline

Fixing #22
Previous State

Previously, building Makefile based
releasing with dagger pipeline missing

Current Behavior

[x] Added building file as go code with dagger build.go
[x] Automated the release with dagger CI by calling for release and pull_request event separately [added deb rpm package based binary release with architecture]
[x] Added Software Bill of Materials (SBOM) generation during release for each archive by analyzing the various generated artifacts using Syft
[x] Added brew install support

We need to create a separate repo under goharbor named homebrew-tap for uploading the tap formula which will eventually be used during brew installation

Screenshot

PS:-  should be removed go.work & go.work.sum file after using the release pipeline once!
//cc @Vad1mo

Awesome!
I'll looking forward to try this out!

Hello @Vad1mo @OrlinVasilev
PTAL in this PR!

@Vad1mo could you pls take a look on it. Now with dagger CI it's also adding release binary to multiple pkg manager and its workflow files running successfully!

build go is building not the harbor cli but rather the build.go file
Something lke this:
build = build.WithExec([]string{"go", "build", "-o", path, "/src/cmd/harbor/main.go"})

can you also provide instrustion on how to test ideally locally the release and pull-request flow?

can you also provide instrustion on how to test ideally locally the release and pull-request flow?

To test the release and pull-request flow locally we need to follow the below steps:-
1. Release

Set the environment variable for GITHUB_TOKEN like this export GITHUB_TOKEN="Your GH token here"

after that make sure to push a new tag beforehand proceeding with release flow git tag -a v0.2.0 -m "message here..." && git push origin v0.2.0

NOTE:- The below command will automatically be triggered whenever a new tag will be pushed
to the repo for release

Now we are ready to run the release command go run ci/dagger.go release $GITHUB_TOKEN

2. Pull-request

Here just run the command with GITHUB_TOKEN set go run ci/dagger.go pull-request $GITHUB_TOKEN

NOTE:- This pull-request action flow will be triggered whenever a new pull-request come to this repo. At first it will check whether all test files are passed or not then will create a --snapshot for the current release and will signal for ready-to-merge
I hope this will help!!

We need to make sure that this PR replaces the current Makefile.
To do so builder.go should have feature and function parity with the Makefile.
Also remove in the PR, the Makefile.

also update the Readme.md, explainin how to build and test locally

The name of the generated SPDX file should have this pattern harbor_cli_0.2.x.sbom
There should be only one sbom. It makes things more confusing having two.

What is the status here ? :thinking:

What is the status here ? 🤔

Currently migrating from using dagger package to dagger module like dagger call and was facing some weird error of exporting the build directory from container to host but finally I got it solved today and will soon update a commit to merge!!

@bishal7679 please upload screenshots of the features added.

@bishal7679 please upload screenshots of the features added.

Now we are able to build harbor cli through dagger call by this command
dagger call build-harbor --directory-arg . export --path=./

But the obstacle for this PR is with the release part. Getting again a weird go version error in go.mod even though its carrying the version with 1.22.5
See the logs here:
https://github.com/bishal7679/harbor-cli/actions/runs/10018821023/job/27694660442#step:4:357

@bishal7679 please squash and sign the commits.

@bishal7679 please fix DCO and the failing test.  https://github.com/goharbor/harbor-cli/actions/runs/10074141257/job/27969136509?pr=25

do you require any help @bishal7679

Closing as completed by #133


Bug? User error? Tags - Removing changes requested and adding RFR when syncing branch with upstream

I think this is either my fault for an unconventional workflow, or perhaps a bug in the tag logic.
First the broad situation:
I have a PR, I get some reviews, before I can address those reviews, some changes are pushed to upstream branch.
As I'm going through my reviews, I sync with upstream. Because i'm using the button on the github.com interface, this also pushes a commit to the repository. [time 0]
I then make some changes that incorporate the comments, and push that to my repository the PR is based on. [time 1]
The bot seems to remove the Changes Requested tag at time 0, but I think it should only be happening at time 1?
Or, if I was using an IDE, syncing with main wouldn't cause a commit and this wouldn't be a problem, so this is really just because I'm using the web interface only?

I haven't hit this as you say because I do my syncs and commits then push after.
I believe this should have been the same with the Trafico implementation looking at the code though. Did you have the same issue before?
The ideal solution would be to actually check if the requested changes had been made, or acknowledged, discussed and dismissed. This feels computationally expensive though.

It really depends on the context of the review. If the review comment says "please update this from master to bring in xyz", :shrug:
Though one thought, is that we are listing it as ready for review even though github still has the some reviews listed as changes requested. Can fork communinity users re-request review or remove reviewers? If so then maybe we leave it as changes requested whenever github reviews say there are still changes requested.

I haven't hit this because, as you say, I do my rebasing and commits locally then push after.
I believe this should have been the same with the Trafico implementation looking at the code though. Did you have the same issue before?
The ideal solution would be to actually check if the requested changes had been made, or acknowledged, discussed and dismissed. This feels computationally expensive though.

Unfortunately, I don't think I had a PR open with the Trafico bot. I'm not sure though. New to contributing (if that wasn't clearly evidenced by me using the github.com interface).
When I have been getting reviews, my general workflow has been to;

respond, basically acknowledging the review contents and kind of stating my intent to do [x] to 'fix' this comment. so that if i'm wildly off-base and my solution sounds like i'm missing the point, why i went where I did with the code can at least make more sense in context.
incorporate the changes into the code, push the fix
mark the review comment as 'resolved'
as the bot as already marked the pr as ready for review, i do nothing else.

It really depends on the context of the review. If the review comment says "please update this from master to bring in xyz", 🤷
Though one thought, is that we are listing it as ready for review even though github still has the some reviews listed as changes requested. Can fork communinity users re-request review or remove reviewers? If so then maybe we leave it as changes requested whenever github reviews say there are still changes requested.

I can't seem to remove reviewers, even for non-maintainer reviewers. I can 're-request review' though. I didn't click it (until just now, for testing) because it felt kind of rude if it was going to send a reviewer a push notification or something? if clicking it is ok or desirable as a way to say "I'm ready, i have finished all my fixes, and addressed the build/lint issues" then I think as long as some part of the documentation made it clear that is the right way to get the "Ready for Review" tag back, it could be a good way to handle the automation I think? Or if that is the only way to get the right tag, people might be more likely to click it. I felt no "need" to click request re-review, because the PR had the RFR tag. If I had done everything to be ready for review but still had the "Changes requested" tag, I would have likely tried clicking the other interactable elements, like "resolve" for the comment threads, and ultimately the "request" button.
"requests review" seems to be a tracked action that the bot could presumably interact with?
e.g.;
@wdpk wdpk requested a review from twk3 2 minutes ago


Change the loading animation

This... sucks

It was made almost 2 years ago on a site called loading.io, which, well, it's... not the best... quality.
As noted by @svobodavl in a Discord message today, the dot above the i doesn't even flippin' spin!

The dot actually spins. It's just that it doesn't spin with the I, which makes me incredibly sad


test(versions): add "insights" test

Summary
Ensure that search-insights is always on the latest version on CI
requires #5958 of course to pass CI

Result
We are warned in CI if there's a newer search-insights version available
example failing run: https://app.circleci.com/pipelines/github/algolia/instantsearch/9868/workflows/c1d28ba1-bcfd-4c1e-920c-b91e33ccf86a/jobs/47989
example passing run: (not done yet)

We could go one step further and set up a weekly/monthly GitHub Action that updates the Insights version in all relevant source files + creates a PR (for example with https://github.com/marketplace/actions/create-pull-request).

Yes, that could be combined


Monte Carlo objectives for VI (IW-ELBO and friends)

Now that the poutine rebake is done, we should re-implement the importance weighted estimator for ELBo (i.e. the one for the IWAE). Should be straightforward.

Covered by algorithm-specific issues


Feature:  periodic check of multiple proxy instances, and if any are out of sync, update it

When running multiple proxies, the configs can get out of sync (when new services are added very quickly). Can there be an automatic check and resync of each instance?  Is it possible to have some way to manually check individual instances when running on swarm network mesh and manually trigger the resync?

Sorry for not responding earlier. Too much work...
I can add automatic check and resync. The major issue is the frequency of it. If it's too frequent (e.g. every second), it might introduce too much overhead. If it's not very frequent, it might not be very usable. For example, if a check is executed every minute, that would mean that a service could be potentially unreachable for that time and that's not a very good result.
What do you think? Do you see a third option?

Hi Viktor,
No I don’t see any alternative off hade. Some tunable default value will have to be used until experience makes clear what a good value is. I agree every second is too much, I would start with every 20-30s to keep the cross chatter as low as possible.
Larry Henry
Staff Infrastructure Architect - Digital Strategy Group   GE Digital
T<PHONE_NUMBER>
F<PHONE_NUMBER>
M<PHONE_NUMBER>
Larry.Henry<EMAIL_ADDRESS>Grace Lake Corporate Center
One Village Center Drive
Van Buren Twp, Michigan 48111
General Electric Company
GE imagination at work
From: Viktor Farcic<EMAIL_ADDRESS>Sent: Saturday, April 29, 2017 5:19 PM
To: vfarcic/docker-flow-proxy<EMAIL_ADDRESS>Cc: Henry, Larry (GE Digital<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: EXT: Re: [vfarcic/docker-flow-proxy] Feature: periodic check of multiple proxy instances, and if any are out of sync, update it (#199)
Sorry for not responding earlier. Too much work...
I can add automatic check and resync. The major issue is the frequency of it. If it's too frequent (e.g. every second), it might introduce too much overhead. If it's not very frequent, it might not be very usable. For example, if a check is executed every minute, that would mean that a service could be potentially unreachable for that time and that's not a very good result.
What do you think? Do you see a third option?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/vfarcic/docker-flow-proxy/issues/199#issuecomment-298195273, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AWkCGMI1uNgetV1Mvf-7l6DpB1mXBhqJks5r06lZgaJpZM4M00Vn.

Sorry for not getting back to this issue earlier. I could not find time to work on this until now.
I added two new environment variables. RELOAD_INTERVAL defines the frequency (in milliseconds) between automatic config reloads from Swarm Listener. It has a default value of 5000 (5 seconds). The second variable is REPEAT_RELOAD. If it's set to true, it will periodically reload the config, using RELOAD_INTERVAL as pause between iterations.
I did not yet make the release official so it is not yet documented. I would appreciate having a second opinion. Can you try it out?

I have tried this option, using a 30s time interval. Not sure it helps as the proxy still gets corrupted somehow.
From within the proxy we noticed this:
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 <IP_ADDRESS>:40710        <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:80              <IP_ADDRESS>:*               LISTEN      6234/haproxy
tcp        0      0 <IP_ADDRESS>:80              <IP_ADDRESS>:*               LISTEN      652/haproxy
tcp        0      0 <IP_ADDRESS>:80              <IP_ADDRESS>:*               LISTEN      231/haproxy
tcp        0      0 <IP_ADDRESS>:443             <IP_ADDRESS>:*               LISTEN      6234/haproxy
tcp        0      0 <IP_ADDRESS>:443             <IP_ADDRESS>:*               LISTEN      652/haproxy
tcp        0      0 <IP_ADDRESS>:443             <IP_ADDRESS>:*               LISTEN      231/haproxy
tcp        0      0 :::8080                 :::*                    LISTEN      1/docker-flow-proxy
Multiple instances of HAProxy get started? I don’t know what triggers this to happen, but we may have a high number of services stopping and starting, which I think has something to do with it.
Hopefully this gives a clue as to what might be happening here.
Thanks
Larry Henry
Staff Infrastructure Architect - Digital Strategy Group   GE Digital
T<PHONE_NUMBER>
F<PHONE_NUMBER>
M<PHONE_NUMBER>
Larry.Henry<EMAIL_ADDRESS>Grace Lake Corporate Center
One Village Center Drive
Van Buren Twp, Michigan 48111
General Electric Company
GE imagination at work
From: Viktor Farcic<EMAIL_ADDRESS>Sent: Sunday, June 11, 2017 4:09 AM
To: vfarcic/docker-flow-proxy<EMAIL_ADDRESS>Cc: Henry, Larry (GE Digital<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: EXT: Re: [vfarcic/docker-flow-proxy] Feature: periodic check of multiple proxy instances, and if any are out of sync, update it (#199)
Closed #199https://github.com/vfarcic/docker-flow-proxy/issues/199 via 0eedacehttps://github.com/vfarcic/docker-flow-proxy/commit/0eedace30e7e303bd5955b9f770b4ab2000d1093.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/vfarcic/docker-flow-proxy/issues/199#event-1118456053, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AWkCGBtE_feZ-krfcFrhxiQeEiSJ1WH_ks5sC6C1gaJpZM4M00Vn.

Every time HAProxy is reloaded, it replaces one process with another. If you check the processes, you should see only one running and the rest of them dead or in zombie state.
Could you send the commands that would help me reproduce the issue in my cluster?

The problem is I don’t know what is causing it, or how to reproduce.
Larry Henry
Staff Infrastructure Architect - Digital Strategy Group   GE Digital
T<PHONE_NUMBER>
F<PHONE_NUMBER>
M<PHONE_NUMBER>
Larry.Henry<EMAIL_ADDRESS>Grace Lake Corporate Center
One Village Center Drive
Van Buren Twp, Michigan 48111
General Electric Company
GE imagination at work
From: Viktor Farcic<EMAIL_ADDRESS>Sent: Friday, June 16, 2017 12:36 PM
To: vfarcic/docker-flow-proxy<EMAIL_ADDRESS>Cc: Henry, Larry (GE Digital<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: EXT: Re: [vfarcic/docker-flow-proxy] Feature: periodic check of multiple proxy instances, and if any are out of sync, update it (#199)
Every time HAProxy is reloaded, it replaces one process with another. If you check the processes, you should see only one running and the rest of them dead or in zombie state.
Could you send the commands that would help me reproduce the issue in my cluster?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/vfarcic/docker-flow-proxy/issues/199#issuecomment-309073727, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AWkCGL6MB3e0ZFX03Ikvq9Mw8JYF2Ls8ks5sEq73gaJpZM4M00Vn.

It'll be hard for me to fix the problem unless I manage to reproduce it myself. The same issue is not present in the cluster I'm running.
Can you send me the logs from DFP and DFSL as well as DFP config when that happens? I might be able to spot something through them.

Closing due to inactivity. Feel free to reopen the issue if you think we can obtain more information or reproduce it.


Great difference results of generalized linear model between xgboost and glmnet

The xgboost for glm code below:
require(xgboost)
data(agaricus.train, package='xgboost')
data(agaricus.test, package='xgboost')
dtrain <- xgb.DMatrix(agaricus.train$data, label = agaricus.train$label)
dtest <- xgb.DMatrix(agaricus.test$data, label = agaricus.test$label)
param <- list(objective = "binary:logistic", booster = "gblinear",
nthread = 2, alpha = 0.0001, lambda = 1)
num_round <- 1
bst <- xgb.train(param, dtrain, num_round)
importance_matrix <- xgb.importance(colnames(agaricus.train$data), model = bst)
(gg <- xgb.ggplot.importance(importance_matrix, measure = "Weight", rel_to_first = TRUE))
gg + ggplot2::ylab("Weight")
ypred <- predict(bst, dtest)
labels <- getinfo(dtest, 'label')
cat('error of preds=', mean(as.numeric(ypred>0.5)!=labels),'\n')
error of preds= 0.01675978
head(importance_matrix)

The glmnet for glm code below:
require(glmnet)
require(varhandle)
trainx <- as.matrix(agaricus.train$data)
trainy <- factor(agaricus.train$label, labels = c("A","B"))
testx <- as.matrix(agaricus.test$data)
testy <- factor(agaricus.test$label, labels = c("A","B"))
testy2 <- agaricus.test$label
glm_result <- glmnet(trainx, trainy, family = "binomial", alpha = 0.0001, lambda = 1)
glm_result
coef_importance <- coef(glm_result, s=1)
pre_result <- predict(glm_result, newx = testx, s = 1,  type = "class")
pre_result2 <- as.numeric(unfactor(factor(pre_result, labels = c('0','1'))))
cat('error of preds=', mean(pre_result2!=testy2),'\n')
error of preds= 0.02979516
head(coef_importance)

There is so much difference between each others. The glmnet's prediction error higher than xgboost  even the same paramters!
how to explain the difference between coef (glmnet) and weight (xgboost) ?

sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=Chinese (Simplified)_China.936
[2] LC_CTYPE=Chinese (Simplified)_China.936
[3] LC_MONETARY=Chinese (Simplified)_China.936
[4] LC_NUMERIC=C
[5] LC_TIME=Chinese (Simplified)_China.936
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods
[7] base
other attached packages:
[1] varhandle_2.0.1 glmnet_2.0-5    foreach_1.4.3   Matrix_1.2-6
[5] xgboost_0.6-4
loaded via a namespace (and not attached):
[1] Rcpp_0.12.7           lattice_0.20-33       codetools_0.2-14
[4] digest_0.6.10         plyr_1.8.4            chron_2.3-47
[7] grid_3.3.1            gtable_0.2.0          magrittr_1.5
[10] scales_0.4.0          ggplot2_2.1.0         stringi_1.1.1
[13] data.table_1.9.6      labeling_0.3          iterators_1.0.8
[16] tools_3.3.1           Ckmeans.1d.dp_3.4.6-1 munsell_0.4.3
[19] rsconnect_0.4.3       colorspace_1.2-6

They are two rather different models in terms of how they were built. And their parameters are not even comparable. In other words, you threw some rather random sets of parameters (that also were far from optimal) at two fairly different algorithms - no wonder the results were not the same.

@khotilov Thank you for you quicklly response! Can you explain the weight calculation method in the xgboost for gblinear booster? It seems rarely comment or papers about it.

In this site: https://www.kaggle.com/general/25924#post148061, there are following words about the gblinear:
Put it simply it is really a "regularized linear model" using delta with elastic net regularization (L1 + L2 + L2 bias) and parallel coordinate descent optimization.
So, I think the model difference between xgboost's gblinear and glmnet is not so much, if we can set the same parameters.

I should have phrased my earlier response more clearly. It should have been: "The two models produced in your example are the results of two different processes. In large part, that difference comes from your parameter choices. There's also a fair difference in the algorithms... But the main reasons seem to be that you don't quite understand what each of those parameters means (the same names != the same meaning) and that you are comparing largely suboptimal models."
Anyway, glmnet uses cyclical gradient descent. When running in a single thread mode, gblinear also does a similar "cycle" of gradient updates at each iteration. However gradient boosting iterations work their way in a fairly different manner than the iterations in glmnet. Moreover, when running multithreaded, there's some hogwild (non-thread-safe) parallelization happening.
As for the parameters, please carefully read the documentation of both packages. Also keep in mind that the loss in xgboost is not normalized to #samples, which means that the penalty parameters would differ by a factor of N. Plus, you would have to run gblinear through quite a lot of iterations to get the coefficients comparable between the two models.


REPL does not understand built-in functions

For instance:
ergotop$ return flatten([[]]);
Runtime error at 1:7-1:20 'Function 'org.accordproject.ergo.stdlib.flatten' not found during eval'

Might be best handled as part of #42, with refactoring of the built-in library (which currently is written in ErgoNNRC rather than ErgoC).

Fixed in PR #319:
Welcome to ERGOTOP version 0.1.2
ergotop$ return flatten([[]]);
Response. []
ergotop$ return flatten([["this"],["should","be"],["flat"]]);
Response. ["this", "should", "be", "flat"]

Note, this does not address #255 (built-in functions for dateTime still won't work in the REPL).
ergotop$ return now();
Response. null


Remove UseOCM Flag

What this PR does / why we need it:
Remove UseOCM Flag
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Release note:

NONE

ToDo search for occurrences of:

UseOCM / UseOCMLib
USE_OCM / USE_OCM_LIB


Using ESP8622 - LED 1 bulb lit, no seeming input from RX

I've been racking my brain and trying everything trying to get it to work, I'm not sure what's going wrong.
Things I've done: flashed the ESP8622 (Adafruit Huzzah Feather) module using the Arduino IDE successfully. I then disconnected it from the computer and powered it separately then pinged its local IP to make sure it's connected and received response with no packet loss.
I made sure the number of LEDs is the same on config.py and ws2812_controller.ino before flashing it onto module. Same applies for local IP.
I'm using a 5m WS2812b strip with a total of 300 LEDs, but I've set the # of LEDS in the scripts at 256.
I've also tested this using a smaller 1m strip with 56 LEDs and had the same problem.

Maintaining that RX is always connected to the data input of the LED strip:

I've tried powering ESP8622 using USB, and powering LED separately with 5V 2.4A power supply
I've tried powering ESP8622 using USB, then taking USB output (pin) straight to the +V of the LED strip, and grounding LED with ground pin on module
I've tried powering LED Strip using 5V supply, and sharing that power supply using the USB pin (functions as Voltage out as well as Voltage In according to Adafruit's documentation) and ground pin.
I've tried powering BOTH ends of the LED strip.
I've powered ESP8622 using computer USB and monitored Serial Monitor while running visualization.py, and I see that it is receiving input, but seemingly no output.

All resulted with a single LED lit up (the first bulb-- on either end when powering from both ends, which just tells me that's not even tied to the module).
I've also tried to test the strip using led.py, but nothing happens.

After some more research, I came up with that a single LED on a WS2812/2812b strip draws ~ 0.06A. For a total of 256 LEDs, that comes out to 15.36A, but since the strip has 300 LEDs (18A), a 5V 15A power supply might work dimly, a 5V 20A power supply is guaranteed to power it properly.
I've only been using a 2.4A power supply, so this might where my issue is. I found these little things which should provide the correct current, I'll find one and post an update with my results.

I'm going to wire it up as follows and see the results:

It's very likely that your LED's drew too much power causing the ESP to brownout, have you tried it with say, 10 leds?
Also give one of the adafruit demo sketches a try and see if that works


[9.x] use null coalescing operator and null safe operator

minor refactor in three methods to use null safe and null coalescing operators instead of if and isset.

If the code is working I do not like to change it much.


Access to the Login object without having a LoginStorage assigned to the Authenticator

I'm using the Authenticator without a LoginStorage assigned.
We need to manage and store Login objects (and tokens) in the keychain but associated to a specific service accounts so extra processing is needed on our side
I was wondering if we could make the retrieveLogin() function public so we can access the local version.
Any drawbacks?

(Sorry very limited internet)
Very open to changes, but I’m not yet quite following what problem you have. LoginStorage is just a struct with functions and has no internal functionality. Can you just do what you need outside of this struct? That’s how I currently use it.

Sorry my issue was indeed not clear enough.
In a multi-account systems, where the user can authenticate more than one account, there is a need to know which account is associated to the Login information.
So the RetrieveLogin and StoreLogin could be provided with an AccountID. That AccountID would identify which account the Login is for.
Perhaps I'm overthinking this....
Check sample change below:

After further investigation, this will not fulfill my needs.
Closing without resolution for now...


fake_useragent not working on Multiprocessing Python?

Hi Everyone,
I'm not sure if this has been solved but yes, as the title says, can't make the fake_useragent work with Multiprocessing Python, using the Process() module. I haven't tested on Pool() but it does work when the function is not on a multiprocess.
it always gives me this Error:
fake_useragent\fake.py", line 140, in __getattr__     raise FakeUserAgentError('Error occurred during getting browser')  # noqa fake_useragent.errors.FakeUserAgentError: Error occurred during getting browser
Any idea what's going on?
Thanks.
By the way, I've tried to use this:
ua = UserAgent(fallback=r'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36')
but it's giving me Error:
Error occurred during getting browser, but was suppressed with fallback.
TypeError: 'str' object is not callable

I hope it's solved by now using the latest release: pip install --upgrade fake-useragent


Using Sol with Boost::property_tree: issues with is_container

I am attempting to use Sol2 with some existing data structures that make heavy use of boost::property_tree to store general data within them. I'm hoping to find an easy way to inter-operate between Lua tables and boost::poroperty_tree which are very similar. However, I can't seem to get a working example. It appears as thou the is_container specialization is not working.
Here is a basic example:
#include<sol.hpp>
#include<boost/property_tree/ptree.hpp>

namespace sol{
template<>
struct is_container<boost::property_tree::ptree> : std::false_type {};
}

class hasP{
public:
  boost::property_tree::ptree data;
};

int main()
{
  sol::state lua;
  lua.new_usertype<hasP>("hasP", "data", &hasP::data);
  return 0;
}

This will not compile, throwing:
sol.hpp:15603:36: error: no match for ‘operator+’ (operand types are ‘sol::container_detail::container_traits_default<sol::as_container_t<boost::property_tree::basic_ptree<std::__cxx11::basic_string, std::__cxx11::basic_string > >, void>::K {aka std::pair<const std::__cxx11::basic_string, boost::property_tree::basic_ptree<std::__cxx11::basic_string, std::__cxx11::basic_string > >}’ and ‘int’)
p = stack::push_reference(L, k + 1);
I'm new to using sol, so I might just be going about this the wrong way, but it seems like the is_container specialization is failing...

I'm a bit confused. The reason you do
namespace sol{
template<>
struct is_container<boost::property_tree::ptree> : std::false_type {};
}

Is because you don't want boost::property_tree to have the container specializations applied to it (that is, you explicitly specialize it to std::false_type). ... But, from your issue request, you kind of do want it to behave like a container/table in Lua?
Either way, this code shouldn't fail to compile, but I'm still confused as to your intent here. I'll look into the compiler error, but in the meantime please give me a little example lua code so I can understand what's you're going for...

Sorry if that was confusing. I am attempting to somehow interface boost::property_tree with Lua. Originally, I had hoped to directly use a ptree as a table in Lua, but I have since realized that there are differences in their definitions that will make that impossible. So now, I want to define my own usertype for ptree and expose its api in a simplified way to Lua. However, I cannot get any usertype for ptree to compile as the automatic container code seems to kick in no matter what. Here is a more direct example:
#include <LuaPlanning/sol.hpp>
#include <boost/property_tree/ptree.hpp>

namespace sol {
template<>
struct is_container<boost::property_tree::ptree> : std::false_type {};
}

int main()
{
    sol::state lua;

    lua.new_usertype<boost::property_tree::ptree>("Ptree");

}

That will fail to compile with the same error as above. Any ideas?

You could get a very basic system only dealing with getting keys, and setting keys, with simple tables, but no ability to manipulate arrays. I suppose anything is possible in programming, but to deal with setting/getting, children/parents, emulating a FOR_EACH loop, and other complexities, that will take a bit of work and I have no idea how to deal with it.
But carrying on, since the property tree's >basic< function is almost like a std::map, you could use this example, and simply modify the get and set, with ptree's "get()", and "put()"(or add()?). And these functions can use periods to access tables.

Pull from the latest and your problem should be solved.
The issue was that boost::property_tree::basic_ptree<...> doesn't exactly obey the standard container methods. Even if you mark it as "not a container", we still access the container meta-table information so we can automatically generate begin()/end() pairs on it that map to Lua's pairs(my_container) calls and similar. It was erroring because the key type we identified for the type was not exactly what we were looking for.
The latest should fix your problems, but I can't really write tests against boost right now, not until I finish working on my CMake harness to have a third class of tests against 3rd-party libraries. I did confirm it builds on my machine, but that doesn't guarantee its type will work out.
If you want to create a custom container around boost::property_tree::ptree, then I urge you to look at the container documentation. It includes a detailed description of how you would override the operations for a single container type, and thus make it behave as you would like it to. This is probably the best way to get the "full experience", but you can also do what the dynamic_object examples and similar linked earlier do as well.

Awesome! Thank you, I'll take a look. I appreciate the support. I'll let you know what I get working.


500 Internal Server Error

Hello,
I get a 500 internal server error when I try to access the API

Hey man, this API is down at the moment, and we are currently working on a new version. Apologies for the inconvenience.


Plugin listed under dependencies

Why is avro-maven-plugin configured as a dependency?

because you need it ^^


Auto boot actors on system startup

An often requested feature is the ability to boot specific actors during system start.
e.g. say you want all your PlayerActors to boot up before getting any direct messages.
I'm thinking this ought to be super easy to set up.
e.g. plugin that just reads a list of actor names from a DB. then send a Touch message to each of them.
(just loop over a stream from the DB, we don't even need to read them all into memory.)

https://github.com/asynkron/protoactor-dotnet/blob/dd8e7983951037a95edbee8ca443e83cc885b1b9/src/Proto.Cluster/Singleton/SingletonManager.cs

Very much related to https://github.com/asynkron/protoactor-dotnet/issues/1114


Castle Windsor resolving more dependencies than it releases with SignalR Activator

Hi,
I don't know if my analyse is correct or not. But i'm facing some memory leak with castle and SignalR. It's a well known "issue" for Transient registered components see for example (https://stackoverflow.com/questions/34395834/signalr-server-memory-consumption-windsor-ihubactivator-to-blame / https://github.com/SignalR/SignalR/issues/3208 and a lot of others).
I'm using this SignalR activator :

public class SignalRHubActivator : IHubActivator
    {
        private readonly IWindsorContainer _container;

        public SignalRHubActivator(IWindsorContainer container)
        {
            _container = container ?? throw new ArgumentNullException("container");
        }

        private int GetTrackedComponentsCount()
        {
            var host = (IDiagnosticsHost)ContainerManager.Container.Kernel.GetSubSystem(SubSystemConstants.DiagnosticsKey);
            var diagnostics = host.GetDiagnostic<ITrackedComponentsDiagnostic>();
            var inspect = diagnostics.Inspect();
            var all = inspect
                .Select(x => new { Name = x.Key.ComponentModel.ToString(), Count = x.Count() }).ToList();
            return all.Sum(x => x.Count);
        }
        public IHub Create(HubDescriptor descriptor)
        {
            var logger = _container.Resolve<ILogger>();

            var beforeResolvingCount = GetTrackedComponentsCount();
            logger.Info($"BEFORE Resolving - In SignalR Activator. Components waiting to release : {beforeResolvingCount}");

            var instance = _container.Resolve(descriptor.HubType) as IHub;
            var beforeReleasingCount = GetTrackedComponentsCount();
            logger.Info($"Before Releasing  - In SignalR Activator. Components waiting to release : {beforeReleasingCount} |  Difference {beforeResolvingCount - beforeReleasingCount}");

            _container.Release(instance);

            var afterReleasingCount = GetTrackedComponentsCount();
            logger.Info($"After Releasing  - In SignalR Activator. Components waiting to release : {beforeReleasingCount} | Difference {beforeReleasingCount - afterReleasingCount}");
            return instance;
        }
    }

You can see that I have added some logs to see the number of tracked components to be released. AS you may know castle with its default Release Policy keep tracks of Transient components that needs to be released (for some reason). And it prevents them to be collected by the Garbage collector. My hubs are registered to be Transient, so Castle container should keep track them when I resolve a Hub. And that's what happen, you can see that i add 3 logs. Each time i log the number of Castle Windsor Tracked Components.

For the first time it logs me 0 because nothing has been resolve at that time.
For the second log, we can see that that number changes, in my case 456 (that's a lot I know)
For the third time, it should be 0 because, i released the hub... But for some reason that i don't know it's printed 436... So only 20 components are removed from tracked components... Even i release the entry object ...

Does anyone have any idea why i have this strange behaviour ? Why Castle Windsor still keep track of the dependencies of my Hub?
I checked in the Castle Windsor code and we can see that when an component is released, the dependencies of that component are also released :
https://github.com/castleproject/Windsor/blob/master/src/Castle.Windsor/MicroKernel/Burden.cs#L132
And of course we can see in the logs that it releases more than one component... So it actually releasing some dependencies but not all. Why ?
Thank you for reading me ! I hope i can have more info about the issue or the misunderstanding.

The problem wasn't Castle Windsor. The problem was that i was registering the dependencies of my Hub using HybridPerWebRequestTransient. Which is not totally the same as registering them as Transient.


refactor: manage reusable ApiClient in RemoteApiHelper

⚠ Pre Checklist

Please complete ALL items in this checklist, and remove before submitting

[ ] I have added relevant tests.
[ ] I have added relevant documentation.
[ ] I will add labels to the PR, such as pr-type/bug-fix, pr-type/feature-development, etc.

Summary
What does this PR do?
Does this close any open issues?
Closes #5993
Screenshots
Include any relevant screenshots here.
Other Information
Any other information that is important to this PR.

How much can we gain in performance and latency by doing this?

Test result in my local dev environment:
Direct access:
time curl -X GET -H "Authorization:Bearer glpat-*****" 'https://gitlab.com/api/v4/projects/8967944'
1
real    0m0.707s
user    0m0.064s
sys     0m0.010s
2
real    0m0.652s
user    0m0.055s
sys     0m0.000s
3
real    0m0.674s
user    0m0.056s
sys     0m0.000s

Via DevLake proxy without client reuse:
time curl -X GET http://<IP_ADDRESS>:30017/plugins/gitlab/connections/1/proxy/rest/projects/8967944
1
real    0m2.222s
user    0m0.004s
sys     0m0.000s
2
real    0m2.097s
user    0m0.003s
sys     0m0.000s
3
real    0m2.321s
user    0m0.004s
sys     0m0.000s

As the request jump through a remote server, this test result may be more affected by network latency
Via DevLake proxy with client reuse:
time curl http://localhost:8080/plugins/gitlab/connections/1/proxy/rest/projects/8967944
1
real    0m1.556s
user    0m0.006s
sys     0m0.000s
2
real    0m0.499s
user    0m0.006s
sys     0m0.000s
3
real    0m0.527s
user    0m0.005s
sys     0m0.000s

Have merged to release-0.18, will be cherry picked back to main in https://github.com/apache/incubator-devlake/pull/6068


Separate Trait Imports

Is it possible to separate the imports for traits? I couldn't find an option for this in php-cs-fixer but it may exist. Ideally, it would handle it similar to regular imports, one per Use statement.

We are not sure. You can search here: https://mlocati.github.io/php-cs-fixer-configurator/.

In case anyone is looking for the rule, you need to add this to your pint.json configuration to achieve this:
{
    "single_trait_insert_per_statement": true
}

Thanks! Shouldn't this be default for PINT as it's a PSR-12 standard?
https://www.php-fig.org/psr/psr-12/#42-using-traits

@billriess the default for Laravel is the Laravel preset, not the PSR-12 one.

Yes but I thought the Laravel preset was mostly based on PSR.
On Fri, Jul 29, 2022, 3:09 AM Dries Vints @.***> wrote:

@billriess https://github.com/billriess the default for Laravel is the
Laravel preset, not the PSR-12 one.
—
Reply to this email directly, view it on GitHub
https://github.com/laravel/pint/issues/74#issuecomment-1198952065, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AA75DXUVT6XOZPPLVTKGDH3VWN7TJANCNFSM522RBO6Q
.
You are receiving this because you were mentioned.Message ID:
@.***>


EAS build with custom metro.config.js wrong warning

It seems the metro config check is wrong. I have the following metro config;
const { getDefaultConfig } = require("expo/metro-config")
const path = require("path")

const localNodeModules = {
  shared: path.resolve(__dirname + "/shared"),
  features: path.resolve(__dirname + "/features"),
  pages: path.resolve(__dirname + "/pages")
}

const extraNodeModules = {
  ...localNodeModules,
  stream: require.resolve("stream-browserify")
}

const config = getDefaultConfig(__dirname)

config.transformer.getTransformOptions = async () => ({
  transform: {
    experimentalImportSupport: true,
    inlineRequires: false
  }
})

config.watchFolders = Object.values(localNodeModules)
config.projectRoot = path.resolve(__dirname)
config.resolver.extraNodeModules = new Proxy(extraNodeModules, {
  get: (target, name) =>
    //redirects dependencies referenced from common/ to local node_modules
    name in target
      ? target[name]
      : path.join(process.cwd(), `node_modules/${name}`)
})

module.exports = config

It works without EAS. However, since I needed to add Intl for Android (for luxon), I decided to try EAS build. However, it shows the warning below:
It looks like that you are using a custom metro.config.js that does not extend @expo/metro-con
fig.
This can result in unexpected and hard to debug issues, like missing assets in the production
bundle.
We recommend you to abort, fix the metro.config.js, and try again.

I logged my metro config, and it does have the asset path, which was the condition used to show the warning:
https://github.com/expo/eas-cli/blob/4cb726f3740c95b2db3e6115528ac895060ff93d/packages/eas-cli/src/project/metroConfig.ts#L16-L18
transformer: {
    assetPlugins: [
      'P:\\Projects\\native\\node_modules\\expo-asset\\tools\\hashAssetFiles.js'
    ],
}

NOTE: I'm on Windows 10

It seems on windows, the backtick separator used  \ is causing this check to fail. I will make a quick PR to address this issue


Multi-image render is failing

When I use more than 1 image source, render fails.
Image URLs work and there is no issue, but when I add more than 1 image to AE template, render fails.
"assets": [
{
"type" : "data",
"layerName" : "BASLIK",
"property" : "Source Text",
"value": BASLIK
},
{
"src": IMAGEURL2,
"type": "image",
"layerName": "IMAGE1"
},
{
"src": IMAGEURL3,
"type": "image",
"layerName": "IMAGE2"
}
],

I found the problem, AE 2023 latest version fails with NexRender. I am using 2022 version and it works now.

I found the problem, AE 2023 latest version fails with NexRender. I am using 2022 version and it works now.

Wait, are you saying that only AE 2022 works with Nexrender (the latest release)? Can you specify which version are you using?
@inlife, can you shed any light on this?

According to the latest release, this bug should be fixed: https://github.com/inlife/nexrender/pull/831

According to the latest release, this bug should be fixed: #831

OK, I will give it a go right now.

Hm, I think I should open a new issue. ->
checking After Effects command line renderer patch...
command line patch already is in place
[omitted] setting up job...
[omitted] working directory is: C:\Users\ADMINI~1\AppData\Local\Temp\2\nexrender\omitted
[omitted] applying predownload actions...
[omitted] downloading assets...
[omitted] skipping the clean up...
(node:8132) UnhandledPromiseRejectionWarning: #<Object>
(node:8132) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:8132) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I am sending a payload to the server. I noticed something while downloading assets (locally). One of the images returns HTTP 403 from some CloudFront URL. Could it be that this is the exception the worker is throwing?
But, in any case, this should not kill the worker, it should be an error that is handled.

I can verify that the latest release works with AE 2023. Just rendered one video.


Fix releases-tab when more-dropdown is disabled

Closes #2361

Test

This repo

Test this on a repo with admin access, I believe the releases tab should be before "Settings".
Right now "Settings" is always the last nav item...

I forgot the settings button. Solved that.

Well, now this won't work for repositories where you don't have admin access 😅
Also need to consider that this won't work as expected if the repository name ends in settings, like:
https://github.com/probot/settings
https://github.com/ledermann/rails-settings
(The selector would select the Code nav link)

Well, now this won't work for repositories where you don't have admin access 😅

Does it? Works fine on my browser...

Well, now this won't work for repositories where you don't have admin access 😅

Does it? Works fine on my browser...

I haven't actually tried, just looked at the code - How could this work if there is no link that ends in settings when you don't have admin access?

It's handled by the util:
https://github.com/sindresorhus/refined-github/blob/0b2400511c28c9e22edbfc18770f113eafcecfc4/source/libs/dom-utils.ts#L25-L37

Also need to consider that this won't work as expected if the repository name ends in settings

Note that the code will only select direct children, so it's fine.
P.S. I spent some time figure out why it doesn't work https://github.com/probot/settings you give, and eventually realized it has no release 🤣

Just to mention, it's probably not exactly related to this PR but today I noticed that more-dropdown is kind of bugged and was always showing the dropdown, so I disabled it.

@tunnckoCore That was reported in #2357
@kidonng made #2359 to solve it.

Ah, cool. Sorry for messing here, but didn't find it earlier :)

We’re not gonna support all combinations of supported features.


Content search by a tag name returns no API

Description:
Searching by tag name does not return any API. (prefix search tag:<tag_name> returns the result correctly)
Steps to reproduce:

Create an API, and attach a tag.
Execute a content search by the tag name
No API is listed

Affected Product Version:

Environment details (with versions):

OS:
Client:
Env (Docker/K8s):

Optional Fields
Related Issues:

Suggested Labels:

Suggested Assignees:

This issue is in progress

Tags are saved as type of properties in solr. Since in api /search it perform a attribute search we can not search for those properties directly with the same call. So as an alternate solution for this we can use tags:<tag_name> when searching by tags.
Since the current product consists of this solution as mentioned in docs, we can close this issue.


Raw logging

Implement a way to log the raw data received from input devices in a single, hourly rolling file with timestamps. This way the raw data can be played back in the order the data was received from the various input devices, in actual rate or speeded up/slowed down.

@tkurki wouldn't this be more useful if it were it's own "producer"/stream? Currently you'd have to use liner in order to use logging, right..

But the output must be line by line, so it makes sense. At least with nmea and n2k. This way playback can also be line by line.

I didn't really get the producer/stream part - elaborate?