pythonPackages.gbulb, pythonPackages.pygtkspellcheck: fix SkyTemple’s dependencies

Description of changes
python gtk3 in the build inputs of 2 packages. I’m worried there might be a reason for them not to be in here already, but SkyTemple, the only package that us it, works correctly.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/) (for SkyTemple)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

This probably should be in propagatedNativeBuildInputs I'm not sure if it should be propagated, but I guess that kinda makes sense, I will leave that to people who are more familiar with Nixpkgs Python ecosystem.

How is gtk3 used in those packages?

How is gtk3 used in those packages?

Logs: https://termbin.com/g3hp

Result of nixpkgs-review pr 188479 run on x86_64-linux 1

7 packages built:

python310Packages.gbulb
python310Packages.pygtkspellcheck
python310Packages.skytemple-ssb-debugger
python39Packages.gbulb
python39Packages.pygtkspellcheck
python39Packages.skytemple-ssb-debugger
skytemple

@SuperSandro2000 Any opinion on propagatedNativeBuildInputs here?
I can fix it in case it is decided to be: propagatedNativeBuildInputs

I think the normal propagation should be fine here.

I think the normal propagation should be fine here.

I just tested it and did not work as propagatedNativeBuildInputs.


[Backport workspace-pr-integr][Workspace] Add a workspace client in workspace plugin (#6094)

feat: add comment

feat: update unit test

feat: add CHANGELOG

feat: optimize comment

feat: optimize comment

feat: optimize code

feat: optimize code

Description

Issues Resolved

Screenshot

Testing the changes

Check List

[ ] All tests pass

[ ] yarn test:jest
[ ] yarn test:jest_integration

[ ] New functionality includes testing.
[ ] New functionality has been documented.
[ ] Update CHANGELOG.md
[ ] Commits are signed per the DCO using --signoff

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 55.39%. Comparing base (0f44eff) to head (62c0b58).

Additional details and impacted files
@@                   Coverage Diff                    @@
##           workspace-pr-integr     #308       +/-   ##
========================================================
- Coverage                67.05%   55.39%   -11.67%
========================================================
  Files                     3345     1193     -2152
  Lines                    65044    24121    -40923
  Branches                 10505     4059     -6446
========================================================
- Hits                     43615    13361    -30254
+ Misses                   18849    10118     -8731
+ Partials                  2580      642     -1938

Flag
Coverage Δ

_1
?

_2
55.39% <ø> (ø)

_3
?

_4
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add search result summary component

Make a functional React component
Start with a pre-built Material UI component like a Button or a Card
Reference the component in Figma here
Include example code and a screenshot of the result in your PR

Requirements

It should be reusable for both mobile and web
It should take id, title, year, artist, type ("Mural", "Collection", or "Tour"), photoURL, and description as properties

Closed by #90

Closed by #90


Swift 4 compatible

Coverage increased (+0.002%) to 99.617% when pulling a2874130e75cb5b392dd341a2212be74db15c0d0 on basvankuijck:master into 93b1ab41ebfb76a615364c0c64a49fa553524a68 on nakiostudio:master.

Hi @basvankuijck 👋
Thanks for looking after another Swift migration! @muukii was a bit faster this time with his PR #61.
I would like to wait for Xcode 9 release before updating the .swift-version file. Would you like to only change that file in this PR to have everything ready for the Xcode release?

cool

It's the moment @basvankuijck :)

here we go!

@basvankuijck
Please change SWIFT_VERSIONto Swift4.0 in the xcconfig?
I think we need it when building with Carthage.

Coverage decreased (-0.5%) to 99.111% when pulling 2586f2188d8bfc996311c6998b1d2a444b0ab355 on basvankuijck:master into 3330374c20221ac111cb666f98f4a2c0c1904db4 on nakiostudio:master.

Thank you @basvankuijck 🙌
I will create a release a fix the issue with Danger :)

@basvankuijck Thank you!


Abililty to DELETE models and or loras from UI

Feature Idea
With so many Loras coming out every day, I would love a delete feature straight from the UI.
Other
No response

I was just thinking this the other day! Great idea!

Why did you mark this as complete??


Controller's menuIsShowing is not getting updated.

It fits the exact needs I have for my project, but I have encountered a problem with it. Unless you explicitly call "controller.showMenu()" or "controller.hideMenu" or "controller.toggleMenu()", the controller.menuIsShowing property never gets updated.
It should be automatically set to true when the menu appears in my opinion, because outside of the component we cannot know if it is true or not, as we cannot wrap the child component of CustomMenuPopup with a GestureDetector, as the menu would not show anymore.
Thank you and I am looking forward to a reply on this matter.

It fits the exact needs I have for my project, but I have encountered a problem with it. Unless you explicitly call "controller.showMenu()" or "controller.hideMenu" or "controller.toggleMenu()", the controller.menuIsShowing property never gets updated.
It should be automatically set to true when the menu appears in my opinion, because outside of the component we cannot know if it is true or not, as we cannot wrap the child component of CustomMenuPopup with a GestureDetector, as the menu would not show anymore.
Thank you and I am looking forward to a reply on this matter.

You can init a controller outside, and pass it to CustomMenuPopup.
plz provide the code snippet for me to understand better.

@cs-vladpaianu
Thanks for your demo. This has fixed on version 1.2.2.
For your case, your should add these code to work:
List<CustomPopupMenuController> _controllers = [];
List<String> stringList = snapshot.data;
if (_controllers.isEmpty) {
  stringList.forEach((_) {
    _controllers.add(CustomPopupMenuController());
  });
}
return Container(
  height: stringList.length.toDouble() * 64,
  child: ListView.builder(
    itemCount: stringList.length,
    itemBuilder: (BuildContext context, int index) {
      return Row(
        mainAxisAlignment: MainAxisAlignment.start,
        crossAxisAlignment: CrossAxisAlignment.center,
        children: [
          SizedBox(width: 18),
          CustomPopupMenu(
            child: Container(
              height: 32,
              decoration: BoxDecoration(
                border: Border.all(
                  color: _controllers[index].menuIsShowing ? Colors.red : Colors.transparent,
                ),
              ),
            ),
            // ****
            menuOnChange: (flag) {
              print(flag);
              setState(() {});
            },
            menuBuilder: () {
              // ...
            },
            controller: _controllers[index],

          ),
        ],
      );
    },
  ),
);

However, i think your case is similar with DropDownMenu, maybe you can try mk_drop_down_menu.

Thank you very much for the change, it is working as expected now. I prefer this package to the other one, because this fits my design better.
After updating to 1.2.2 though, the issue I have now is that when I call _controllers[index].hideMenu(), it is now not closing anymore. When a certain element in the menu in pressed, I want to show an AlertDialog and therefore hide the menu, but it currently is not dismissing.
If I were to use it outside of a ListView, with a single controller, the hideMenu() method works. Do you have any idea what the issue might be?


Rename the jspm provider to jspm.io

It's a bit confusing have jspm refer to both the company, the CDN and the CLI tool. We should rename the jspm provider to jspm.io to make it clear that it's referring to the CDN.

Duplicate of #230, whoops.


In the function hdwallet::getPublicKeyFromExtended function, to create the PublicKey object, can we use the return value by calling Coin::publicKeyType(coin)?

In the function hdwallet::getPublicKeyFromExtended function, to create the PublicKey object, can we use the return value by calling Coin::publicKeyType(coin)?

didn't get it, they're different types, Coin::publicKeyType returns enum type


refactor: add disable_dashboard option and disable dashboard in metasrv and datanode

I hereby agree to the terms of the GreptimeDB CLA
What's changed and what's your intention?
Add disable_dashboard option and disable dashboard in metasrv and datanode.
Resolve #1337
Please explain IN DETAIL what the changes are in this PR and why they are needed:

Summarize your change (mandatory)
How does this PR work? Need a brief introduction for the changed logic (optional)
Describe clearly one logical change and avoid lazy messages (optional)
Describe any limitations of the current code (optional)

Checklist

[ ]  I have written the necessary rustdoc comments.
[x]  I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

wait, we dont need to provide option to disable it we just disable it by default because it's not usable with datanode and metasrv

@sunng87 Your suggestion is right. I use #[serde(skip)] to skip the disable_dashboard filed so it can't be parse from toml file. Only the frontend cli option can change disable_dashboard field.

wait, we dont need to provide option to disable it we just disable it by default because it's not usable with datanode and metasrv

+1. It's better for user not need to explicitly disable Dashboard when starting Datanode or Metasrv.

wait, we dont need to provide option to disable it we just disable it by default because it's not usable with datanode and metasrv

+1. It's better for user not need to explicitly disable Dashboard when starting Datanode or Metasrv.

The datanode and metasrv will disable dashboard internal.

LGTM


+c interfaces returning +o +j

Hi,
the current code generation is broken for cases where a +c interface returns +j or +o interfaces.
Given:
extern = interface +j +o {...}
intern = interface +c { foo(): extern; }

On the JNI side this tries to call Extern::toJava(), which does not exist, and for Obj-C it tries to static_pointer_cast from Extern to ExternObjcProxy, which is mostly likely not the case and causes undefined behavior.
I think the easiest fix would be to disallow returning +j +o interfaces from +c. Supporting this properly would be a bit more involved I think.

This is fixed now, right?

Well, there is an assert(dynamic_cast...) in there now, so if someone screws up it shows up in a debug build, but with asserts disabled it's just a plain static_cast. Question is does this suffice or should dynamic_cast stay there and make a fallback solution kick in if it fails.

I think what we should probably do is dynamic_cast a reference, so we'll get a bad_cast if the type is wrong (essentially, always asserting). I made that change in my proposed fix for #81 - in particular, see the new wrapping/unwrapping code here: https://github.com/dropbox/djinni/commit/f0aec36d99764ca62903ba58fbb9c4af9c886c54#diff-61477e190b9e4951838edf1fa955dd19R216
This mirrors the existing Java scheme, though with slightly less template trickery (see https://github.com/dropbox/djinni/blob/master/support-lib/jni/djinni_support.hpp#L438 and particularly _unwrapJavaProxy)
Closing this out; let's discuss in #81.


Links have blind spot while being dragged over comment node

When node is places on the left side of the patch canvas, try to move link coming from the outlet to the right and top, before connecting it.

Turned out to be caused by comment node lying below (catches text selection)?


UI doesn't automatically refresh changes

OS: Linux Mint 19.3 Cinnamon
App Version: 1.1.1 flatpak
Making changes such as editing/adding/renaming files doesn't automatically appear in the UI. The UI only updates when manually clicking View -> Refresh.
I couldn't find the issue here but might have found something similar in gitahead https://github.com/gitahead/gitahead/issues/318#issue-523909985

Hi @ruiznick  thanks for pointing to the issue!
We have to distinguish:

Editing file locally
Editing the git history by another application or the git command line

In the linked issue they are talking about the second point. Are you refering to the first point?
I have seen this behaviour on hidden files. There I have to refresh manually. For normal files the gui shall be refreshed automatically.

Hi @ruiznick thanks for pointing to the issue! We have to distinguish:
1. Editing file locally

2. Editing the git history by another application or the git command line

In the linked issue they are talking about the second point. Are you refering to the first point?
I have seen this behaviour on hidden files. There I have to refresh manually. For normal files the gui shall be refreshed automatically.

@Murmele If I make a change in JetBrains Rider and switch to Gittyup the changes do appear to load reliably. But if I make changes in Unity and alter a scene file (.unity file extension) then Gittyup doesn't auto refresh.

We are using a Repositiory watcher to check if we have local changes. Currently we are ignoring all .* files / folders. This behaviour comes from GitAhead and we did not touch it yet.
const QDir::Filters kFilters = (QDir::Dirs | QDir::NoDotAndDotDot); in RepositoryWatcher_linux.cpp on mac and windows it looks different.
Even for branch changes we don't detect them for this we would have to watch on .git/HEAD

I should clarify, I meant Filename.unity files and not dot files.

hmm ok then it is a bug ^^

@ruiznick is this still reproducable? is the can you provide me two examples of such a .unity file? So I can test the behaviour

Most (or all?) git operations performed outside of Gittyup won't be detected. Also in my case I have to remember to refresh with CTRL+R.
Examples are: to reset a branch to another commit; or pushing a new commit to remote.

Can you open for this an issue, because this is not yet implemented. Only changes in the working directory are considered


Target of URI doesn't exist.

Hello.
When I attempt to run the example app,I get a error from vscode.
Target of URI doesn't exist: 'package:audioplayers/audioplayers_web.dart'.
Are there any extra configuration steps that I need to take?

Try this
Target of URI doesn't exist: 'package:audioplayers/audioplayers_web.dart.tmp'

tmp

doesn't work for me

That file clearly exists: https://github.com/luanpotter/audioplayers/blob/master/lib/audioplayers_web.dart
If this is still a problem please open a new issue following the mandatory issue template.


The clone repo table is not End to end

Dev Home version
No response
Windows build number
No response
Other software
No response
Steps to reproduce the bug
Description
The clone repo table should be extended the width of 924 edge to edge
Expected result
No response
Actual result
No response
Included System Information
No response
Included Extensions Information
No response

Fixed.


Request for Contact Info

Hello,
I've noticed something about this project that I'd like to discuss privately with the maintainers. Could you please provide a secure method to contact you regarding project-related matters?
Thank you for your time.

Is it the API key being leaked?


packaging: updated patch

Signed-off-by: Miquel Sabaté Solà<EMAIL_ADDRESS>

Forgot about this in #1329.


Need to add an input field to a table row

What is the proper way to add an input field to a table row and get the typed value within the input field when selecting the row?

just use the render: () => () property in the column.

Closing for now.


Ruby support

We want to add language support for Ruby.

[ ] Install Ruby in Docker containers
[x]  Introduce Ruby as a new language
[ ] Register the language when the symflower test is working for Ruby
[ ] Implement write-tests task

[x] Create "plain" repository for Ruby
[x] Translate the light repository using a LLM
[ ] Support Ruby in symflower test

[ ] Implement  the ExecuteTests method in the Ruby language

[ ] Implement code_repair task

[x] Translate the  mistakes repository using a LLM
[ ] Implement the Mistakes method in the Ruby language
[ ] TODO

[ ] Implement transpile task

The transpile repository used examples from the light repository, so no need to translate them again
[x] Change the way we get the origin/destination languages for transpilation

[x] The transpile package names must be e.g. for Go   transpile-java, transpile-ruby, instead of just transpile so it is clear which language we are transpiling from

[ ] TODO

For new transpilation tasks, there needs to be some small changes to how the eval figures out which languages are targeted.
Cause right now if we have Go as transpilation source, it automatically assumes Java must be the target cause no other language existed yet.

Checking for errors on the Ruby mistakes repository
For Java and Go we check for compilation errors with:

Go: go build
Java: mvn compile

Since Ruby is an interpreted language, there is no compilation step. I tried the following approaches.

Check Ruby's syntax with ruby -c lib/<ruby-file> (see Command line options)

argumentsMissing: Syntax OK
endKeywordMissing: lib/end_keyword_missing.rb:9: syntax error, unexpected end-of-input, expecting keyword_end
importMissing: SyntaxOK
typeUnknown: SyntaxOK
variableUnknown: SyntaxOK

Conclusion: only only error caught

Running rake test at the root of the repository

argumentsMissing

Output
Run options: --seed 64839

# Running:

EEE

Finished in 0.000889s, 3373.6712 runs/s, 0.0000 assertions/s.

  1) Error:
TestArgumentMissing#test_argument_missing2:
ArgumentError: wrong number of arguments (given 1, expected 0)
    /home/rui/dev/ruby/playground/mist/argumentMissing/lib/argument_missing.rb:1:in `argument_missing'
    /home/rui/dev/ruby/playground/mist/argumentMissing/test/argument_missing_test.rb:15:in `test_argument_missing2'

  2) Error:
TestArgumentMissing#test_argument_missing3:
ArgumentError: wrong number of arguments (given 1, expected 0)
    /home/rui/dev/ruby/playground/mist/argumentMissing/lib/argument_missing.rb:1:in `argument_missing'
    /home/rui/dev/ruby/playground/mist/argumentMissing/test/argument_missing_test.rb:22:in `test_argument_missing3'

  3) Error:
TestArgumentMissing#test_argument_missing1:
ArgumentError: wrong number of arguments (given 1, expected 0)
    /home/rui/dev/ruby/playground/mist/argumentMissing/lib/argument_missing.rb:1:in `argument_missing'
    /home/rui/dev/ruby/playground/mist/argumentMissing/test/argument_missing_test.rb:8:in `test_argument_missing1'

3 runs, 0 assertions, 0 failures, 3 errors, 0 skips
rake aborted!
Command failed with status (1)

Tasks: TOP => test
(See full trace by running task with --trace)

endKeywordMissing

Output
/home/rui/dev/ruby/playground/mist/endKeywordMissing/lib/end_keyword_missing.rb:4: warning: mismatched indentations at 'end' with 'if' at 2
/home/rui/dev/ruby/playground/mist/endKeywordMissing/lib/end_keyword_missing.rb:7: warning: mismatched indentations at 'end' with 'if' at 5
Traceback (most recent call last):
        6: from /usr/lib64/ruby/gems/2.5.0/gems/rake-12.3.3/lib/rake/rake_test_loader.rb:5:in `<main>'
        5: from /usr/lib64/ruby/gems/2.5.0/gems/rake-12.3.3/lib/rake/rake_test_loader.rb:5:in `select'
        4: from /usr/lib64/ruby/gems/2.5.0/gems/rake-12.3.3/lib/rake/rake_test_loader.rb:17:in `block in <main>'
        3: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
        2: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
        1: from /home/rui/dev/ruby/playground/mist/endKeywordMissing/test/end_keyword_missing_test.rb:2:in `<top (required)>'
/home/rui/dev/ruby/playground/mist/endKeywordMissing/test/end_keyword_missing_test.rb:2:in `require_relative': /home/rui/dev/ruby/playground/mist/endKeywordMissing/lib/end_keyword_missing.rb:9: syntax error, unexpected end-of-input, expecting keyword_end (SyntaxError)
        return 0
                ^
rake aborted!
Command failed with status (1)

Tasks: TOP => test
(See full trace by running task with --trace)

importMissing

Output
/home/rui/dev/ruby/playground/mist/importMissing/test/import_missing_test.rb:9: warning: mismatched indentations at 'end' with 'def' at 5
Run options: --seed 60902

# Running:

EEEEEE

Finished in 0.001216s, 4932.5231 runs/s, 0.0000 assertions/s.

  1) Error:
TestImportMissing#test_invalid_json:
NameError: uninitialized constant TestImportMissing::JSON
    /home/rui/dev/ruby/playground/mist/importMissing/test/import_missing_test.rb:31:in `test_invalid_json'

  2) Error:
TestImportMissing#test_valid_json:
NameError: uninitialized constant JSON
    /home/rui/dev/ruby/playground/mist/importMissing/lib/import_missing.rb:2:in `parse_json'
    /home/rui/dev/ruby/playground/mist/importMissing/test/import_missing_test.rb:8:in `test_valid_json'

  3) Error:
TestImportMissing#test_json_with_nested_objects:
NameError: uninitialized constant JSON
    /home/rui/dev/ruby/playground/mist/importMissing/lib/import_missing.rb:2:in `parse_json'
    /home/rui/dev/ruby/playground/mist/importMissing/test/import_missing_test.rb:26:in `test_json_with_nested_objects'

  4) Error:
TestImportMissing#test_json_with_numbers:
NameError: uninitialized constant JSON
    /home/rui/dev/ruby/playground/mist/importMissing/lib/import_missing.rb:2:in `parse_json'
    /home/rui/dev/ruby/playground/mist/importMissing/test/import_missing_test.rb:20:in `test_json_with_numbers'

  5) Error:
TestImportMissing#test_json_with_array:
NameError: uninitialized constant JSON
    /home/rui/dev/ruby/playground/mist/importMissing/lib/import_missing.rb:2:in `parse_json'
    /home/rui/dev/ruby/playground/mist/importMissing/test/import_missing_test.rb:37:in `test_json_with_array'

  6) Error:
TestImportMissing#test_empty_json:
NameError: uninitialized constant JSON
    /home/rui/dev/ruby/playground/mist/importMissing/lib/import_missing.rb:2:in `parse_json'
    /home/rui/dev/ruby/playground/mist/importMissing/test/import_missing_test.rb:14:in `test_empty_json'

6 runs, 0 assertions, 0 failures, 6 errors, 0 skips
rake aborted!
Command failed with status (1)

Tasks: TOP => test
(See full trace by running task with --trace)

typeUnknown

Output
Run options: --seed 56356

# Running:

EEE

Finished in 0.001033s, 2904.8122 runs/s, 0.0000 assertions/s.

  1) Error:
TestTypeUnknown#test_type_is_wrong1:
NameError: uninitialized constant Intt
    /home/rui/dev/ruby/playground/mist/typeUnknown/lib/type_unknown.rb:2:in `type_unknown'
    /home/rui/dev/ruby/playground/mist/typeUnknown/test/type_unknown_test.rb:8:in `test_type_is_wrong1'

  2) Error:
TestTypeUnknown#test_type_is_wrong3:
NameError: uninitialized constant Intt
    /home/rui/dev/ruby/playground/mist/typeUnknown/lib/type_unknown.rb:2:in `type_unknown'
    /home/rui/dev/ruby/playground/mist/typeUnknown/test/type_unknown_test.rb:22:in `test_type_is_wrong3'

  3) Error:
TestTypeUnknown#test_type_is_wrong2:
NameError: uninitialized constant Intt
    /home/rui/dev/ruby/playground/mist/typeUnknown/lib/type_unknown.rb:2:in `type_unknown'
    /home/rui/dev/ruby/playground/mist/typeUnknown/test/type_unknown_test.rb:15:in `test_type_is_wrong2'

3 runs, 0 assertions, 0 failures, 3 errors, 0 skips
rake aborted!
Command failed with status (1)

Tasks: TOP => test
(See full trace by running task with --trace)

variableUnknown

Output
/home/rui/dev/ruby/playground/mist/variableUnknown/lib/variable_unknown.rb:2: warning: possibly useless use of == in void context
Run options: --seed 59116

# Running:

EEE

Finished in 0.000951s, 3153.5992 runs/s, 0.0000 assertions/s.

  1) Error:
TestVariableUnknown#test_variable_unknown2:
NameError: undefined local variable or method `y' for #<TestVariableUnknown:0x000055b56a9accf0>
    /home/rui/dev/ruby/playground/mist/variableUnknown/lib/variable_unknown.rb:2:in `variable_unknown'
    /home/rui/dev/ruby/playground/mist/variableUnknown/test/variable_unknown_test.rb:15:in `test_variable_unknown2'

  2) Error:
TestVariableUnknown#test_variable_unknown1:
NameError: undefined local variable or method `y' for #<TestVariableUnknown:0x000055b56a9ac160>
    /home/rui/dev/ruby/playground/mist/variableUnknown/lib/variable_unknown.rb:2:in `variable_unknown'
    /home/rui/dev/ruby/playground/mist/variableUnknown/test/variable_unknown_test.rb:8:in `test_variable_unknown1'

  3) Error:
TestVariableUnknown#test_variable_unknown3:
NameError: undefined local variable or method `y' for #<TestVariableUnknown:0x000055b56a6eddc0>
    /home/rui/dev/ruby/playground/mist/variableUnknown/lib/variable_unknown.rb:2:in `variable_unknown'
    /home/rui/dev/ruby/playground/mist/variableUnknown/test/variable_unknown_test.rb:22:in `test_variable_unknown3'

3 runs, 0 assertions, 0 failures, 3 errors, 0 skips
rake aborted!
Command failed with status (1)

Tasks: TOP => test
(See full trace by running task with --trace)

Conclusion: It catches all errors, but the structure of syntax errors are different from runtime errors.

Adding a call.rb that calls the function defined in lib/<ruby-source-file, then running ruby call.rb and check the output
Example of call.rb
require_relative 'lib/end_keyword_missing'

end_keyword_missing(1)

argumentsMissing

Output
Traceback (most recent call last):
        1: from call.rb:3:in `<main>'
/home/rui/dev/ruby/playground/mist/argumentMissing/lib/argument_missing.rb:2:in `argument_missing': undefined local variable or method `x' for main:Object (NameError)

endKeywordMissing

Output
Traceback (most recent call last):
        1: from call.rb:1:in `<main>'
call.rb:1:in `require_relative': /home/rui/dev/ruby/playground/mist/endKeywordMissing/lib/end_keyword_missing.rb:9: syntax error, unexpected end-of-input, expecting keyword_end (SyntaxError)
        return 0

importMissing

Output
Traceback (most recent call last):
        1: from call.rb:3:in `<main>'
/home/rui/dev/ruby/playground/mist/importMissing/lib/import_missing.rb:2:in `parse_json': uninitialized constant JSON (NameError)

typeUnknown

Output
Traceback (most recent call last):
        1: from call.rb:3:in `<main>'
/home/rui/dev/ruby/playground/mist/typeUnknown/lib/type_unknown.rb:2:in `type_unknown': uninitialized constant Intt (NameError)

variableUnknown

Output
Traceback (most recent call last):
        1: from call.rb:3:in `<main>'
/home/rui/dev/ruby/playground/mist/variableUnknown/lib/variable_unknown.rb:2:in `variable_unknown': undefined local variable or method `y' for main:Object (NameError)

Conclusion: better error messages. All errors have the same structure

Conclusion

Using ruby -c only checks for syntax errors. We can only catch one error within 5 packages
Using rake test

We get the errors for all packages, but syntax errors have a different error structure
Implies running all tests
A little bit harder to parse

Using the call.rb

All errors have the same format
Implies adding a file to each package calling the function with the errors
Easier to parse

Claude suggested rubocop, it even has JSON formatted output: https://docs.rubocop.org/rubocop/formatters.html#json-formatter

All PRs are merged so closing this one.


[SPRINT] flip: growth rate measurement code with peculiar velocity

flip
The objective of this sprint is the improvement of a code for growth rate measurement with SNIa peculiar velocities. This code is used in Damiano Rosselli's PhD thesis, which aims at forecasting LSST growth rate measurement.
Contacts: Corentin Ravoux, Damiano Rosselli, Bastien Carreres
Time: Morning
Main communication channel: #desc-sprint-feb2024-growth-rate
GitHub repo: https://github.com/corentinravoux/flip
In-person/Virtual/Hybrid: Virtual
Zoom room (if applicable):
Goals and deliverable
The first goal is to present the code and have a short tutorial for people unfamiliar with it or wanting to use it.
The second goal is to improve the fitting part of the code. The potential ways of improvement will depend on the defined needs.
There is no needed deliverable; this is a relatively informal sprint.
Resources and skills needed
I will set up some basic examples for testing and coding, and we can test the code on LSST-like simulations. It is better to have NERSC access.
Detailed description
flip (Field Level Inference Package) is a Python package that uses the Maximum likelihood method to fit growth rate based on velocity and density field. The first part of the software is the computation of a covariance matrix from a model power spectrum and the considered coordinates. This part is generalized to work for any linear power spectrum models, both for velocities, densities, and cross-terms, and it is optimized with Hankel transform for any model. In the second part, the covariance is used to create a likelihood by multiplying it by velocities or densities.
Currently, in LSST, this software is used for peculiar velocities computed from SNIa. The main objective is to develop the software and to improve its computation speed.
Possible improvements are:

Improvement of likelihood computation time with Jax, and more generally in the whole software.
Improvement of interpolation of matrix covariances with jax_cosmo or equivalent available tools.
Full implementation of MCMC fitter with emcee.
Investigation of the possibility of implementing non-linear models for velocity-velocity, density-velocity, and density-density correlations
Improvement of the power spectrum generator module.
Implementation of a likelihood which accounts for sample selection bias effects
Adding of a zero point velocity in the estimated velocity model.
Creation of tests.

Tutorial:

To access the tutorial, you have several possible choices:

On NERSC, with lsst group permission, the tutorials are located in: /global/cfs/cdirs/m1727/groups/TD/PV/desc_sprint_feb24_flip/tutorials
Without NERSC, the tutorials can be directly launched from the [Open in Collab] buttons of the main page: https://github.com/corentinravoux/flip

A general introduction to flip is given in the following presentation:
https://github.com/corentinravoux/flip/blob/main/slides/intro_to_flip_ravoux.pdf

We will set up examples with the latest simulations of LSST SNIa in /global/cfs/cdirs/m1727/groups/TD/PV/desc_sprint_feb24_flip/forecast_tests
The objective of those tests is to have a basis to make the possible improvement listed in the beginning of the sprint


Segmentation fault using tf.lite.TFLiteConverter with representative_dataset

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: No
TensorFlow installed from (source or binary): Source
TensorFlow version (use command below): 2.0beta1
Python version: 3.6.8
Bazel version (if compiling from source):  0.24.1
GCC/Compiler version (if compiling from source): 7.4.0
CUDA/cuDNN version: 10.0
GPU model and memory: GTX2080TI 11 GB
Exact command to reproduce:

Describe the problem
I am trying to quantize (post-training) my model, using a representative dataset as explained in the latest documentation. I tried to build the representative dataset by loading some relevant PNGs from a folder. But I get a segmentation fault error when I set the representative dataset (no error if I do not set it or use mnist as dataset).
Source code / logs
This works:
mnist_train, _ = tf.keras.datasets.mnist.load_data()
images = tf.cast(mnist_train[0], tf.float32)/255.0
mnist_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
def representative_data_gen():
  for input_value in mnist_ds.take(100):
    yield [input_value]

tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()

This doesn't (SEGFAULT):
train = []
files = glob.glob ('/myPath/data/images/test/*.png')

for myFile in files:
    image = cv2.imread (myFile)
    train.append (image)
train = np.array(train,dtype='float32') #as mnist
images = tf.cast(train[0], tf.float32)/255.0
my_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)

def representative_data_gen():
  for input_value in my_ds.take(100):
    yield [input_value]

tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()

Output:
....
2019-06-15 20:40:32.261287: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-15 20:40:32.261634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10246 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5)
W0615 20:40:32.319736<PHONE_NUMBER>44416 deprecation_wrapper.py:118] From quantize.py:47: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

Fatal Python error: Segmentation fault

Current thread 0x00007f6abf885740 (most recent call first):
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/optimize/calibrator.py", line 51 in __init__
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 197 in _calibrate_quantize_model
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 922 in convert
  File "quantize.py", line 50 in <module>

Segmentation fault (core dumped)

If I use ciphar instead of mnist as representative dataset I also get the same SEGFAULT.
images = tf.cast(cifar_train[0], tf.float32) / 255.0
cifar_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
def representative_data_gen():
  for input_value in cifar_ds.take(100):
    yield [input_value]`

Any ideas? Thank you.

Are you also setting converter.representative_dataset = representative_data_gen in both examples?
Could you provide the complete code for the working MNIST case and the failing cifar test case (the ones where you are using tf.keras.datasets), to help reproduce the error. Thanks!

Hello suharshs. Yes, I am setting converter.representative_dataset = representative_data_get, that's what triggers the SEGFAULT (if I do not set that the process doesn't crash at least). I'll post the code you ask later. Thank you.

Full code that works (needs some cleaning):
import os
from PIL import Image
import random
import tensorflow as tf
import sys

if sys.version_info.major >= 3:
    import pathlib
else:
    import pathlib2 as pathlib
from pathlib import Path

os.environ['PYTHONPATH'] = ':/content/models/research/:/content/models/research/slim/'
tf.enable_eager_execution()
tf.logging.set_verbosity(tf.logging.DEBUG)

frozen_graph='/somePath/tflite/saved_model.pb'
input_arrays=["normalized_input_image_tensor"]
output_arrays=['TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3']
input_shapes={"normalized_input_image_tensor":[1,300,300,3]}
converter = tf.lite.TFLiteConverter.from_frozen_graph(frozen_graph,input_arrays,output_arrays,input_shapes=input_shapes)
converter.allow_custom_ops=True

mnist_train, _ = tf.keras.datasets.mnist.load_data()
images = tf.cast(mnist_train[0], tf.float32)/255.0
mnist_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
def representative_data_gen():
  for input_value in mnist_ds.take(100):
    yield [input_value]

tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

tflite_quant_model = converter.convert()
tflite_model_quant_file = "path/tflite/model_quant.tflite"
p = Path(tflite_model_quant_file)
p.write_bytes(tflite_quant_model)

I'm sorry, but it seems I accidentally deleted that line in the mnist example that seemed to work (converter.representative_dataset = representative_data_gen).
So now I get SEGFAULT with mnist also, and here it is one example of the code that crashes (I got this idea from here: :
import glob
import cv2
import numpy as np
import os
from PIL import Image
import random
import tensorflow as tf
import sys
import faulthandler; faulthandler.enable()

if sys.version_info.major >= 3:
    import pathlib
else:
    import pathlib2 as pathlib
from pathlib import Path

os.environ['PYTHONPATH'] = ':/content/models/research/:/content/models/research/slim/'
tf.enable_eager_execution()
tf.logging.set_verbosity(tf.logging.DEBUG)

frozen_graph='/somePath/tflite/saved_model.pb'
input_arrays=["normalized_input_image_tensor"]
output_arrays=['TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3']
input_shapes={"normalized_input_image_tensor":[1,300,300,3]}
converter = tf.lite.TFLiteConverter.from_frozen_graph(frozen_graph,input_arrays,output_arrays,input_shapes=input_shapes)
converter.allow_custom_ops=True

mnist_train, _ = tf.keras.datasets.mnist.load_data()
images = tf.cast(mnist_train[0], tf.float32)/255.0
mnist_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
def representative_data_gen():
  for input_value in mnist_ds.take(100):
    yield [input_value]

converter.representative_dataset = representative_data_gen

tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset=representative_data_gen
tflite_quant_model = converter.convert()
tflite_model_quant_file = "/somePath/tflite/quantized.tflite"
p = Path(tflite_model_quant_file)
p.write_bytes(tflite_quant_model)

Output:
python quantize2.py
WARNING: Logging before flag parsing goes to stderr.
W0617 17:23:37.896252<PHONE_NUMBER>63776 deprecation_wrapper.py:118] From quantize2.py:18: The name tf.enable_eager_execution is deprecated. Please use tf.compat.v1.enable_eager_execution instead.

W0617 17:23:37.896475<PHONE_NUMBER>63776 deprecation_wrapper.py:118] From quantize2.py:19: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

W0617 17:23:37.896584<PHONE_NUMBER>63776 deprecation_wrapper.py:118] From quantize2.py:19: The name tf.logging.DEBUG is deprecated. Please use tf.compat.v1.logging.DEBUG instead.

2019-06-17 17:23:37.903709: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2019-06-17 17:23:43.597809: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.598136: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.65
pciBusID: 0000:01:00.0
2019-06-17 17:23:43.598281: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-06-17 17:23:43.598920: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-06-17 17:23:43.599496: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-06-17 17:23:43.599652: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-06-17 17:23:43.600447: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-06-17 17:23:43.601066: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-06-17 17:23:43.602911: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-06-17 17:23:43.602994: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.603347: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.603640: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2019-06-17 17:23:43.628186: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2019-06-17 17:23:43.629547: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1b49b80 executing computations on platform Host. Devices:
2019-06-17 17:23:43.629572: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2019-06-17 17:23:43.766065: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.766403: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x37b6d40 executing computations on platform CUDA. Devices:
2019-06-17 17:23:43.766415: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
2019-06-17 17:23:43.766522: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.766808: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.65
pciBusID: 0000:01:00.0
2019-06-17 17:23:43.766834: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-06-17 17:23:43.766843: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-06-17 17:23:43.766851: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-06-17 17:23:43.766859: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-06-17 17:23:43.766867: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-06-17 17:23:43.766876: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-06-17 17:23:43.766885: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-06-17 17:23:43.766912: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.767203: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.767469: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2019-06-17 17:23:43.767488: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-06-17 17:23:43.768066: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-06-17 17:23:43.768074: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0
2019-06-17 17:23:43.768080: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N
2019-06-17 17:23:43.768133: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.768426: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.768716: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10311 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5)
2019-06-17 17:23:43.982463: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.982782: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.65
pciBusID: 0000:01:00.0
2019-06-17 17:23:43.982815: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-06-17 17:23:43.982825: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-06-17 17:23:43.982835: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-06-17 17:23:43.982860: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-06-17 17:23:43.982868: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-06-17 17:23:43.982875: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-06-17 17:23:43.982897: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-06-17 17:23:43.982967: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.983295: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.983592: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2019-06-17 17:23:43.983609: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-06-17 17:23:43.983613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0
2019-06-17 17:23:43.983616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N
2019-06-17 17:23:43.983688: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.984070: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-17 17:23:43.984359: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10311 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5)
W0617 17:23:44.392145<PHONE_NUMBER>63776 deprecation_wrapper.py:118] From quantize2.py:39: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

Fatal Python error: Segmentation fault

Current thread 0x00007fa3f7279740 (most recent call first):
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/optimize/calibrator.py", line 51 in __init__
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 197 in _calibrate_quantize_model
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 922 in convert
  File "quantize2.py", line 42 in <module>
Violación de segmento (`core' generado)

Ciphar example (also crashes)
https://github.com/tensorflow/tensorflow/issues/29443

Hey hvico, the segfault of mnist seems strange. I am investigating with the code you provided. Will post here once I have any update. Thanks.

Hi hvico, the representative_data_gen part is reasonable and I suspect the error is from some mismatch between the data and the model. Is it possible for you to share the model? Thanks.

representative_data_gen should(better) be the images that used to train our model?
@jianlijianli

I ran into a similar problem and debugged my script using PDB, caught an exception related to custom operators. I then removed the custom operator from the model. Then the conversion finished without a problem.

@jianlijianli  hello,have you solved the issue ? I have the same problem as yours. But it seems to work when you downgrade your tensorflow version tensorflow-gpu==1.13.1, but then you should use converter.post_training_quantize=True instead of convert.optimizations= [tf.lite.Optimize.DEFAULT], because the last one haven't exist in the early version yet .
Then I noticed that the size of my graph is about 1/4 as the original one, thus when test in the mobile the fps don't look like improve, do you have any idea of this ?

@chenyuZha I think by default a lot of the operations in your graph is still floating point-based hence there might not be a speed advantage. But your weights are stored as uint8 instead of float32 hence you see 1/4 size reduction. For speed improvement, you need to set converter.inference_type = tf.lite.constants.QUANTIZED_UINT8. This will require you to have added fake quant nodes in the training step. See tutorial https://github.com/tensorflow/tensorflow/tree/r1.13/tensorflow/contrib/quantize.

@LiyouZhou  Thanks a lot for your response. But I have a question when I converted my model to type QUANTIZED_UINT8, which is as I took a non-quantized model for training(ex: ssd_mobilenet_v2_coco), so I must give a the min and max values for activations(default_ranges_stats in  TFLiteConverter). But for the training aware models, these values are calculated during training. I wonder how can I get the correct range when I convert my float model to quantized uint8..?Because once I gave  like min=0 and max=255, I got my quantized model, but it fails to predict bounding-boxes during inference..

I have not figured out how to convert model fully to 8 bits post-training. I suspect it cannot be done. The tool always prompts warning saying I should insert fake quant into training graph. For an example of quantized training https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_mobilenet_v1_quantized_300x300_coco14_sync.config#L195-L201

@LiyouZhou  Ok I see.. So the best option to 'quantize' a float model is using representative_dataset to define the dynamic range of activations, which returns back to the topic of this issue:segmentation fault. I wonder if it is related the version of tensorflow: as I trained my model with tensorflow-gpu==1.13.1 which is not as same as the version required(tensorflow-gpu==1.14.0) to process representative_dataset option...

#26412

So it is confirmed, it does not support custom ops.

@LiyouZhou  so I've rerun export_tflite_ssd_graph.py with flag add_postprocessing_op=false, in the end I got 3 raw outputs :'raw_outputs/box_encodings','raw_outputs/class_predictions','anchors' . By using the TFliteConverter I've succeeded to get my tflite file( I didn't set representative_dataset yet in order to see if with these kind of outputs I can run inference with mobile). Now I get another problem of output size:   Cannot copy between a TensorFlowLite tensor with shape [1, 1917, 4] and a Java object with shape [1, 10, 4].
I suppose is that as I didn't add post processing operations, the 1917 corresponds the num of anchors which is not the num of max_boxes detections.

@chenyuZha You basically have to now implement what TFLite_Detection_PostProcess https://github.com/tensorflow/tensorflow/blob/f34d571332336f325223be6030368390283818cf/tensorflow/lite/kernels/detection_postprocess.cc#L716 does yourself.
This looks messy but involves the following steps:

Get a list of the anchor boxes. When you ran the export, they are written into a tf.constant node. This means you can extract its value like this:

anchors = tf.get_default_graph().get_tensor_by_name("anchors:0")
output = sess.run(anchors)
np.save("ssd_anchors.npy", output)

In your case there should be 1917 anchor boxes.
2. The box_encodings are coded and you need to decode them using anchor box to get the true values. You have now 1917 box_encodings, one for each anchor box.
3. The exact way to decode is defined in your training config in the box_coder section, you can find the python code to decode in this folder https://github.com/tensorflow/models/tree/master/research/object_detection/box_coders
4. Do a bit of reverse engineer and try to find the exact sequence to decode your boxes. In my case the trick bit was that anchor box values are scaled by the image width and height.

@LiyouZhou  Thanks a lot for your advice.. So all what I should do is to find the way to decode the bbox then add the bloc to the train config, then I re-run the export_ssdlite_graph.py with this new config ?

@chenyuZha I think the training is not affected at all. You don't need to change anything about the training. Here are the steps after training

export the graph without custom operators
quantize to 8 bit
Run inference and get output
decode output by writing your own decoder

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: No
TensorFlow installed from (source or binary): Source
TensorFlow version (use command below): 2.0beta1
Python version: 3.6.8
Bazel version (if compiling from source):  0.24.1
GCC/Compiler version (if compiling from source): 7.4.0
CUDA/cuDNN version: 10.0
GPU model and memory: GTX2080TI 11 GB
Exact command to reproduce:

Describe the problem
I am trying to quantize (post-training) my model, using a representative dataset as explained in the latest documentation. I tried to build the representative dataset by loading some relevant PNGs from a folder. But I get a segmentation fault error when I set the representative dataset (no error if I do not set it or use mnist as dataset).
Source code / logs
This works:
mnist_train, _ = tf.keras.datasets.mnist.load_data()
images = tf.cast(mnist_train[0], tf.float32)/255.0
mnist_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
def representative_data_gen():
  for input_value in mnist_ds.take(100):
    yield [input_value]

tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()

This doesn't (SEGFAULT):
train = []
files = glob.glob ('/myPath/data/images/test/*.png')

for myFile in files:
    image = cv2.imread (myFile)
    train.append (image)
train = np.array(train,dtype='float32') #as mnist
images = tf.cast(train[0], tf.float32)/255.0
my_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)

def representative_data_gen():
  for input_value in my_ds.take(100):
    yield [input_value]

tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()

Output:
....
2019-06-15 20:40:32.261287: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-15 20:40:32.261634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10246 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5)
W0615 20:40:32.319736<PHONE_NUMBER>44416 deprecation_wrapper.py:118] From quantize.py:47: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

Fatal Python error: Segmentation fault

Current thread 0x00007f6abf885740 (most recent call first):
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/optimize/calibrator.py", line 51 in __init__
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 197 in _calibrate_quantize_model
  File "/usr/tf2/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 922 in convert
  File "quantize.py", line 50 in <module>

Segmentation fault (core dumped)

If I use ciphar instead of mnist as representative dataset I also get the same SEGFAULT.
images = tf.cast(cifar_train[0], tf.float32) / 255.0
cifar_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
def representative_data_gen():
  for input_value in cifar_ds.take(100):
    yield [input_value]`

Any ideas? Thank you.

Hi.Can you solve it?
I need that

@Davari393 Maybe it is to do with input shape not what the network is expecting?

My model is mobilenetv2 ssdlite that its input shape is 300x300x3
my code is:
coco_train = []
directory_images = "/home/davari/detection_on_mobile/val_coco_dataset/coco2014/output/kam_images/"
for img in os.listdir(directory_images):
coco_data = cv2.imread(os.path.join(directory_images,img))
coco_data = cv2.resize(coco_data , (300 , 300))
coco_train.append(coco_data)
print(tuple(coco_train))
def representative_data_gen():
for input_value in tuple(coco_train):
yield [input_value]
converter.representative_dataset = representative_data_gen
tflite_model_quant = converter.convert()
open("/home/davari/detection_on_mobile/MobilenetV2SSDLite_model_quant.tflite", "wb").write(tflite_model_quant)
but after run :
32412 segmentation fault (core dumped)

Some times you need to supply the input size of (1, 300, 300, 3). But with a segfault, there isn't really much information to go on. Try tracking down the failure point with pdb, that will tell you more information.

Sorry to piggyback on an old issue. I'm using tf==1.15 and I'm hitting a segmentation fault attempting int8 quantization using converter.representative_dataset. I set a breakpoint at the beginning of my representative_dataset_gen function and it never hits it.
I get a segmentation fault here when running the conversion:
.../tensorflow_core/lite/python/optimize/calibrator.py", line 51 in __init__

The full stack trace is here:
> /home/ubuntu/denoise-gst/venv/lib/python3.6/site-packages/tensorflow_core/lite/python/lite.py(993)convert()
-> inference_output_type)
(Pdb)
Fatal Python error: Segmentation fault

Current thread 0x00007ff40ee9f740 (most recent call first):
  File ".../python3.6/site-packages/tensorflow_core/lite/python/optimize/calibrator.py", line 51 in __init__
  File ".../python3.6/site-packages/tensorflow_core/lite/python/lite.py", line 236 in _calibrate_quantize_model
  File ".../python3.6/site-packages/tensorflow_core/lite/python/lite.py", line 993 in convert
  File ".../convert_model_to_tflite_int8.py", line 97 in <module>
  File "<string>", line 1 in <module>
  File "/usr/lib/python3.6/bdb.py", line 434 in run
  File "/usr/lib/python3.6/pdb.py", line 1548 in _runscript
  File "/usr/lib/python3.6/pdb.py", line 1667 in main
  File "/usr/lib/python3.6/pdb.py", line 1694 in <module>
  File "/usr/lib/python3.6/runpy.py", line 85 in _run_code
  File "/usr/lib/python3.6/runpy.py", line 193 in _run_module_as_main
[1]    17668 segmentation fault (core dumped)  python -m pdb convert_model_to_tflite_int8.py

Let me know if this is unrelated and I will open a new issue. I can't find any other issues more similar to my problem than this issue.
Any other debugging tips to get to the root of my problem would be appreciated :)

@hvico We are checking to see if you still need help on this issue, as you are using an older version of tensorflow(1.x) which is not actively supported. We recommend that you upgrade to 2.4 or later version and let us know if the issue still persists in newer versions.Thanks!


Last message of the chat between sender and receiver will be showing

We will show last message as shown in screenshot:

@Ijaiswalshivam but whoever is doing fcm notification feature they can pull the repository and easily add this code right?

@Ijaiswalshivam ???


Crash while generating seed in linux version

Finally got some time to test the linux port, hit generate, and the error message thingy popped up. Doing this under devuan daedalus on amd64, appending log
smz3-cas-20241207.log

I'll take a look when I can tomorrow  Not too familiar with that distro, but if you got the error popup it must be somewhat trying to work.

@doomcup Have you tried possibly re-opening it and seeing if it works now? It looks like there may be some timing issues, and it tried to load all of the tracker profiles before they were put in the proper location. I'll have to look into why that may be the case. But, it looks like it did move it to the correct location after, so I think they may be where they need to be for you now.

Looks like you're right. I've tested it since and it's done a couple more seeds without issue.
Also Devuan's just a fork of Debian without systemd, by a bunch of former Debian devs.


[SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL when driver crashes

What changes were proposed in this pull request?
There is a race condition introduced in SPARK-11141 which could cause data loss.
The problem is that ReceivedBlockTracker.insertAllocatedBatch function assumes that all blocks from streamIdToUnallocatedBlockQueues allocated to the batch and clears the queue.
In this PR only the allocated blocks will be removed from the queue which will prevent data loss.
How was this patch tested?
Additional unit test + manually.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@jose-torres @tdas @zsxwing could you take a look at this please?

ok to test

Test build #87489 has started for PR 20620 at commit 152fec4.

Test build #87489 has finished for PR 20620 at commit 152fec4.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87489/
Test FAILed.

Test build #87494 has started for PR 20620 at commit bd46d1c.

Test build #87494 has finished for PR 20620 at commit bd46d1c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87494/
Test PASSed.

Test build #87519 has started for PR 20620 at commit 23e0204.

Test build #87519 has finished for PR 20620 at commit 23e0204.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87519/
Test FAILed.

Seems like unrelated issue.

retest this please.

Test build #87522 has started for PR 20620 at commit 23e0204.

Test build #87522 has finished for PR 20620 at commit 23e0204.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87522/
Test PASSed.

LGTM.

Merging to master, will try back to 2.0.

(Argh, the wifi here is horrible, I'll need to manually merge things, so hang on a sec...)


React + ASPNET Core + Prerendering: An attempt was made to access a socket in a way forbidden by its access permissions

(Also posted on Stack Overflow)
I have a .NET Core + React app using aspnet-prerendering. I created the app using dotnet new reactredux. I published this to Azure, and routed to that app using AWS CloudFront.
The app has worked in all environments (locally, CI, prestaging...) but when I deployed it to production, it worked for a few days and then I started getting a lot of An attempt was made to access a socket in a way forbidden by its access permissions errors. This is happening when a POST is made to <IP_ADDRESS>:<randomPort>. I do not make this POST in my code, and I've been told it has to do with how react is prerendered. These errors usually only start about 1-2 days after deploying. So when I first deploy, everything looks good, but then when the errors start, they continue until I re-deploy or restart the app in Azure.

Things I've tried:

Restarting the app fixes the problem for a little while.
I have heard that this is caused by react hot reloading, so I made sure to add an appsetting in Azure: ASPNETCORE_ENVIRONMENT=Production. This did not solve the problem.
I tried commenting out the hot reloading portion of Startup.cs entirely to check this hypothesis…but the errors still started again 2 days after deploying that code.

A sample list of the ports that fail vs. succeed (seemingly random...?):
success	count_	data
FALSE	3493	http://<IP_ADDRESS>:53571
FALSE	1353	http://<IP_ADDRESS>:49988
FALSE	535	    http://<IP_ADDRESS>:55453
FALSE	484	    http://<IP_ADDRESS>:53144
FALSE	13	    http://<IP_ADDRESS>:52428
FALSE	7	    http://<IP_ADDRESS>:49583
TRUE	11247	http://<IP_ADDRESS>:56790
TRUE	10960	http://<IP_ADDRESS>:55806
TRUE	10920	http://<IP_ADDRESS>:55227
TRUE	9300	http://<IP_ADDRESS>:55453
TRUE	8472	http://<IP_ADDRESS>:51734
TRUE	5602	http://<IP_ADDRESS>:53571
TRUE	5429	http://<IP_ADDRESS>:56860

Code snippets:
Startup:
public void ConfigureServices(IServiceCollection services)
    {    services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);

        // In production, the React files will be served from this directory
        services.AddSpaStaticFiles(configuration =>
        {
            configuration.RootPath = "ClientApp/build";
        });
        …
    }

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
            app.UseWebpackDevMiddleware(new WebpackDevMiddlewareOptions
            {
                HotModuleReplacement = true,
                ReactHotModuleReplacement = true,
                // This configuration suppresses the warnings that the server code doesn't match the client code, which happens on reload.
                HotModuleReplacementClientOptions = new Dictionary<string, string>()
                {
                    { "warn", "false" }
                }
            });
        }
        else
        {
        app.UseExceptionHandler("/Error");
        app.UseHsts();
        }

        //app.UseHttpsRedirection();
        app.UseStaticFiles();
        …
    }

Index.cshtml
@using Microsoft.Extensions.Configuration
@inject Microsoft.AspNetCore.Hosting.IHostingEnvironment hostingEnv
@inject IConfiguration Configuration

<script>
    window.data = @Html.Raw(Json.Serialize(@Model));
</script>

<div id="col-app"
     asp-prerender-module="./wwwroot/cost-of-living-calculator/dist/main-server"
     asp-prerender-webpack-config="webpack.config.server.js"
<EMAIL_ADDRESS>
@if (hostingEnv.EnvironmentName == "Production")
{
    <script src="/cost-of-living-calculator/dist/main.min.js" asp-append-version="true"></script>
}
else
{
    <script src="/cost-of-living-calculator/dist/main.js" asp-append-version="true"></script>
}

Package.json
...
"aspnet-prerendering": "3.0.1",
"aspnet-webpack": "3.0.0",
"aspnet-webpack-react": "3.0.0",
...

NOTE: I have a boot-server and a boot-client, as well as a server webpack config and a client webpack config. This is due to differing needs for server/client compilation.
Boot-server.js
import React from 'react';
import { createServerRenderer } from 'aspnet-prerendering';
import { renderToString } from 'react-dom/server';
import App from './App';

export default createServerRenderer(
  params =>
    new Promise((resolve, reject) => {
      const data = params.data;
      const result = renderToString(React.createElement(App, data));

      resolve({ html: result });
    })
);

Boot-client.js
import React from 'react';
import ReactDOM from 'react-dom';
import App from './App';

const element = document.getElementById('col-app');
ReactDOM.render(<App {...window.data} />, element);

// This is required to trigger the hot reloading functionality
if (module.hot) {
  module.hot.accept('./App', () => {
    const NewApp = require('./App').default;
    ReactDOM.render(<NewApp {...window.data} />, element);
  });
}

Webpack.config.js
var path = require('path');
var webpack = require('webpack');
const UglifyJsPlugin = require("uglifyjs-webpack-plugin");
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
const OptimizeCSSAssetsPlugin = require("optimize-css-assets-webpack-plugin");
const isDevelopment = process.env.ASPNETCORE_ENVIRONMENT === 'Development';

module.exports = () => {
  return {
    devtool: 'inline-source-map',
    entry: {
    'main': ['babel-polyfill', './ClientApp/boot-client.js', './ClientApp/index.scss'],
},
optimization: {
  minimizer: [
    new UglifyJsPlugin({
      cache: true,
      parallel: true,
      sourceMap: true // set to true if you want JS source maps
    }),
    new OptimizeCSSAssetsPlugin({})
  ]
},
output: {
  path: path.join(__dirname, './wwwroot/cost-of-living-calculator/dist'),
  publicPath: '/cost-of-living-calculator/dist/',
  filename: isDevelopment ? "[name].js" : "[name].min.js"
},
plugins: [
  new MiniCssExtractPlugin({
    // Options similar to the same options in webpackOptions.output
    // both options are optional
    filename: "[name].css",
    chunkFilename: "[id].css"
  })
],
module: {
  rules: [
    {
      // JavaScript files, which will be transpiled
      // based on .babelrc
      test: /\.(js|jsx)$/,
      exclude: [/node_modules/],
      loader: ['babel-loader', 'source-map-loader']
    },
    {
      test: /\.s?css$/,
      use: isDevelopment ? [
        'style-loader',
        'css-loader',
        'sass-loader',
      ] : [
        MiniCssExtractPlugin.loader,
        'css-loader',
        'sass-loader'
      ]
    }
  ]
},
mode: isDevelopment ? 'development' : 'production'
  }
};

Webpack.config.server.js
var path = require('path');
const isDevelopment = process.env.ASPNETCORE_ENVIRONMENT === 'Development';

module.exports = () => {
  return {
    devtool: 'inline-source-map',
    entry: {
      'main-server': ['babel-polyfill', './ClientApp/boot-server.js']
    },
    output: {
      path: path.join(__dirname, './wwwroot/cost-of-living-calculator/dist'),
      publicPath: '/cost-of-living-calculator/dist/',
      filename: "[name].js",
      // commonjs is required for server-side compilation
      libraryTarget: 'commonjs'
    },
    module: {
      rules: [
        {
          test: /\.js$/,
          exclude: [/node_modules/],
          loader: ['babel-loader', 'source-map-loader']
        }
      ]
    },
    target: 'node',
    mode: isDevelopment ? 'development' : 'production'
  }
};

My question is - Why is this happening? How do I prevent it?

My team is also having this issue. Is is possible there might be a problem with disposing sockets? From what I have found, there is a limit on sockets you can open, so if a socket is disposed but there is a problem and another one is opened, after a few days, you reach the limit of sockets and errors occur. Just an hypothesis, nothing for sure.
If anyone finds a solution, would be great :)

Thanks for contacting us. We're closing this issue as this doesn't align with our long-term plans in this area. You can read more details about our vision for this area at https://github.com/aspnet/AspNetCore/issues/12890.


Initial gogo prototype

This is a working prototype for implementing constraints for gogo-generated protobufs.
This is sufficient for handling envoyproxy/data-plane-api:

supports gogoproto.nullable annotation
supports gogoproto.stdduration annotation

Signed-off-by: Kuat Yessenov<EMAIL_ADDRESS>

Partially addresses #52

@rodaine I want to hear your thoughts how best to structure this. I can fix the build once we agree how to get this started.

Thanks @rodaine. Can you help me bring in gogo protoc plugin into the CI docker image?

Build fixed!

PTAL, added docs and separate templates into go/gogo as requested.


Not receiving success or fail callback from Branch.initSession()

I'm currently attempting to update our Branch SDK to the latest version available.
Functionality is now breaking in the app because I am no longer receiving a callback from Branch.initSession() call.
After extensive debugging I can see the Intent in logcat with the correct referral information that was embedded in the link from the referrer
I also log this in the activity that receives the intent, so I know the information is being received correctly at the Activity level.
Now the problem occurs when I call Branch.initSession(BranchInitCallBack) from a Unity script.
I have traced the logs all the way to the  BranchAndroidWrapper.cs then at this point the code crosses the AndroidUnity bridge via the call to
_getBranchClass().CallStatic("initSession", callbackId);
At this point I receive nothing back, no error or success callback, I would assume if something was wrong with my setup I would be receiving the error callback?
Before I updated the SDK, the callback was pretty much instantaneous however now it seems to never call.
Any ideas / gotchas or suggestions in what I could be missing would be appreciated.
Thanks

@Callum-Abele thanks for the report again. The issue should be resolved in the new release (0.6.0)

Hi @bklastaitis-branch thanks for the update, after importing the latest plugin I am now receiving an error object back from the callback
ReferralManager: Error = Warning. Session initialization already happened. To force a new session, set intent extra, "branch_force_new_session", to true.
This sounds like it needs to be set from the Android side of the plugin? is their a c# accessor function for this?

@Callum-Abele what is version of SDK?

@antonargunov  0.6.0

@Callum-Abele can you write steps to reproduce? what did you do? just open the app? open and then click the link? etc.

I'll check this in your sample app to make sure its not something on my setup will keep you informed

@antonargunov Hi Anton, I've been using the UnityTestbed from the latest sdk and it all seems to be working, so it seems to be a fault with our setup.
I have been looking into how the error is created on the Android plugin and as far as I can understand this should only trigger if the app is already in the foreground and a new intent is passed to the app?
I have been testing our app with the app closed so I am unsure why I am receiving this error.
Is their an API from the Unity app to  set intent extra, "branch_force_new_session", to true?

@antonargunov @bklastaitis-branch
I have not managed to replicate my issue on the test bed Unity project.
The issue occurs if I implement my own subclass of Application and reference this in the AndroidManifest.xml
I call Branch.getAutoInstance(getApplicationContext()) on my Application's onCreate call.
However when I launch the test bed I see the same error I get on my app
see attached screenshot

I was under the impression this was the only call I had to make for sub-classing Application. Is this not correct now for the latest version of the SDK?

@antonargunov @bklastaitis-branch
I have now tried subclassing BranchUnityApp and calling super.onCreate. This does not work.
I have tried called BranchUtil.setPluginType(BranchUtil.PluginType.Unity) before Branch.getAutoInstance(this.getApplicationContext());
After adding this call I no longer get the error returned from the callback,
'Error = Warning. Session initialization already happened. To force a new session, set intent extra, "branch_force_new_session", to true.'
Is this the correct call to make now for custom applications as I do not see any references to this in your documentation.
Unfortunately even though the error is no longer being returned I am still not receiving my referral data in the callback, all I get is an object with 2 keys
{ +clicked_branch_link=False  +is_first_session=False }
Any help / confirmation of this issue your side would be greatly appreciated.

@Callum-Abele I understood, look please at this:
https://github.com/BranchMetrics/unity-branch-deep-linking-attribution/blob/master/AndroidWrapperProject/app/src/main/java/io/branch/unity/BranchUnityApp.java
https://github.com/BranchMetrics/unity-branch-deep-linking-attribution/blob/master/AndroidWrapperProject/app/src/main/java/io/branch/unity/BranchUnityActivity.java
If you need to do the same rows to your app and activity classes.

@antonargunov Hi Anton
It seems adding the BranchUtil.setPluginType(BranchUtil.PluginType.Unity) seems to be working for me now.
I think I had some caching issue occurring yesterday, after running a clean build and calling that function in my Application subclass onCreate, it seems to be working correctly now

@Callum-Abele we have uploaded new version with support of Unity 2019.
Add please in you application OnCreate method the next calls:
BranchUtil.setPluginType(BranchUtil.PluginType.Unity);
Branch.getAutoInstance(this.getApplicationContext());
Branch.disableInstantDeepLinking(true);

If you will have any trouble - create please new ticket.

At the moment I have the same error, the callback is not called

Any updates on this? I'm having the same issue with Unity 2020.3.29f1 + Android Pixel XL + Branch Unity SDK v0.6.7

FYI, I can replicate this with the BranchSDK_Demo scene. The UI is visible, but the callback never returns. Ie. I don't see any of the following debug:
public void CallbackWithBranchUniversalObject(BranchUniversalObject universalObject, BranchLinkProperties linkProperties, string error) {
		if (error != null) {
			Debug.LogError("Branch Error: " + error);
		} else {
			Debug.Log("Branch initialization completed: ");

			Debug.Log("Universal Object: " + universalObject.ToJsonString());
			Debug.Log("Link Properties: " + linkProperties.ToJsonString());
		}
	}

Quick update - I was able to get it working in a vanilla Unity 2020.3.29f1 project. I will try to figure out why it's behaving differently on my project.

I am also experiencing this issue


what is it connecting to ?

not sure why is this trying to connect to Amazon server .
$db = "(DESCRIPTION=(ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = nirbhaylms.c3x2dwubess3.us-west-2.rds.amazonaws.com)(PORT = 1521)))(CONNECT_DATA=(SID=orcl)))";

You can connect it to your own rds if you're using Amazon. Otherwise however you want to connect to your db. (Not maintaining this repo anymore)

Is there somewhere copy of db?  Will save some time :)


build(deps): bump github.com/onsi/gomega from 1.33.1 to 1.34.1

Description of the change:
Motivation for the change:
Architectural changes:

Testing remarks:

Reviewer Checklist

[ ] Implementation matches the proposed design, or proposal is updated to match implementation
[ ] Sufficient unit test coverage
[ ] Sufficient end-to-end test coverage
[ ] Bug fixes are accompanied by regression test(s)
[ ] e2e tests and flake fixes are accompanied evidence of flake testing, e.g. executing the test 100(0) times
[ ] tech debt/todo is accompanied by issue link(s) in comments in the surrounding code
[ ] Tests are comprehensible, e.g. Ginkgo DSL is being used appropriately
[ ] Docs updated or added to /doc
[ ] Commit messages sensible and descriptive
[ ] Tests marked as [FLAKE] are truly flaky and have an issue
[ ] Code is properly formatted

Fixes #3351


chore: migrate publish related commands

chore: migrate publish related commands
This PR:

Moves the bulk of the logic of dendron publish <cmd> to pods-core
Slightly improves consistency of cli messages
Implements Dendron: Publish Export and Dendron: Publish Dev, that corresponds to dendron publish export and dendron publish dev

The options that are passed in the cli version are prompted with a quickpick in vscode.

Fixes issue in dendron publish export where _handlePublishTarget pre-emptively ran because npm run export didn't block the execution and piped the output over the cli prompt.

NOTE: I had to update an outdated dependency innextjs-template (caniuse-lite is outdate), because this was failing npm run export
DOCS: https://github.com/dendronhq/dendron-site/pull/260
Manually checked

[x] dendron publish init works as intended
[x] dendron publish build works as intended
[x] dendron publish export works as intended
[x] dendron publish dev works as intended
[x] Dendron: Publish Export prompts necessary options and works successfully exports
[x] Dendron: Publish Dev prompts necessary options and serves to localhost:3000, and could be un-served by clicking on the toast notification's button

Pull Request Checklist
You can go to dendron pull requests to see full details for items in this checklist.
General
Quality Assurance

[ ] add a test for the new feature
[x] make sure all the existing tests pass
[x] do a spot check by running your feature with our test workspace
[ ] after you submit your pull request, check the output of our integration test and make sure all tests pass

NOTE: if you running MacOS or Linux, pay special attention to the Windows output and vice versa if you are developing on Windows

Special Cases

[~] if your tests changes an existing snapshot, make sure that snapshots are updated
[~] if you are adding a new language feature (graphically visible in VS Code/preview/publishing), make sure that it is included in test-workspace. We use this to manually inspect new changes and for auto regression testing

Docs

[x] Make sure that the PR title follows our commit style
[x] Please summarize the feature or impact in 1-2 lines in the PR description
[x] If your change reflects documentation changes, also submit a PR to dendron-site and mention the doc PR link in your current PR

Analytics

[~] if you are adding analytics related changes, make sure the Telemetry docs are updated

Fixed the broken tests and passed in an optional progressCb from the cli.
Stubs were acting a little weird... quirks of Sinon.

checking if next was initialized was omitted during dev and export. fixed.

all looks good - lets remove the progressCb in pods and we should be good to go!

sorry. the actual push of the changes were omitted 😅


All extension traits need to be sealed

They are currently unsealed which allows users to implement them even though it's pretty obviously misguided. If we keep them unsealed then formally adding any function to them is a breaking change which is silly.

I think if they are blanket-impled then users can't implement them? e.g. if we have ReadExt impled for all T: Read then nobody except stdlib will be able to implement ReadExt since theoretically stdlib could add their own blanket impl that'd cover the user's type?
If it's possible to do, I wouldn't say it's "obviously misguided". But it's not what the extension traits are meant for, and I agree we should seal them to give ourselves more API freedom at the expense of preventing it.

LOL, I mean all type extension traits that we're adding lately. Trait extensions are indeed fine.

Oh, yes, definitely agreed! And ok, yes, in that case it'd be "obviously misguided" for users to implement them :).

If we keep them unsealed then formally adding any function to them is a breaking change which is silly.

This doesn't matter because bitcoin 1.0.0 is years away.
 I think we should close this.


\Illuminate\Database\Eloquent\Relations\Relation not treated as \Illuminate\Database\Eloquent\Builder

Larastan Version: master at https://github.com/nunomaduro/larastan/commit/15e8d17cbccbf83bcebadbf581a037920803f510
--level used: 6

Description
I think the issue description is the one I want to go for, not exactly sure TBH; this is the reported error:

 Parameter #1 $builder of method Repo::addSomeImportentConditions() expects Illuminate\Database\Eloquent\Builder, Illuminate\Database\Eloquent\Relations\HasManyThrough<App\Table2> given.

Laravel code where the issue was found
I've a helper method defined like this:
    /**
     * @param Builder|QueryBuilder $builder
     * @return Builder|QueryBuilder
     */
    private function addSomeImportentConditions($builder)
    {
        return $builder
            ->where('table1.deleted', '=', false)
            ->where('table2.deleted', '=', false)
            ->whereNull('table2.parent_id')
            ->whereIn('table2.type', ['foo', 'bar']);
    }

and I use it from code like this:
    public function someImportantMethod(Model $model): int
    {
        return (int) $this
            ->addSomeImportentConditions($model->thisIsAHasManyThroughRelation())
            ->sum('table2.some_aggregate_column');
    }

The thisIsAHasManyThroughRelation is defined as:
    public function thisIsAHasManyThroughRelation(): HasManyThrough
    {
        return $this->hasManyThrough(Table2::class, Table1::class);
    }

Now, HasManyThrough is a Relation defined as:
/**
 * @mixin \Illuminate\Database\Eloquent\Builder
 */
abstract class Relation

I.e. internally it forwards certain calls to the builder, see the __call method:
    /**
     * Handle dynamic method calls to the relationship.
     *
     * @param  string  $method
     * @param  array  $parameters
     * @return mixed
     */
    public function __call($method, $parameters)
    {
        if (static::hasMacro($method)) {
            return $this->macroCall($method, $parameters);
        }

        $result = $this->forwardCallTo($this->query, $method, $parameters);

        if ($result === $this->query) {
            return $this;
        }

        return $result;
    }

This means I can technically use the relation directly and call ->where() etc. on it which I'm doing in addSomeImportentConditions.
Workaround
I can write this code and it works, i.e. adding ->getQuery() on the relationship:
    public function someImportantMethod(Model $model): int
    {
        return (int) $this
            ->addSomeImportentConditions($model->thisIsAHasManyThroughRelation()->getQuery())
            ->sum('table2.some_aggregate_column');
    }

Hi,
I think this is normal behavior. You are expecting Builder|QueryBuilder type in addSomeImportentConditions method, but you are passing Relation (HasManyThrough in this case) Doesn't matter if relation forwards calls to the query builder. Types don't match.
So I think having Builder|QueryBuilder|Relation as the type of the parameter would fix it.

testing my signed email :)
Idézem/Quoting Can Vural<EMAIL_ADDRESS>
Hi,
I think this is normal behavior. You are expecting
Builder|QueryBuilder type in addSomeImportentConditions method,
but you are passing Relation (HasManyThrough in this case)
Doesn't matter if relation forwards calls to the query builder.
Types don't match.
So I think having Builder|QueryBuilder|Relation as the type of the
parameter would fix it.

SZÉPE Viktor, webes alkalmazás üzemeltetés / Running your application
https://github.com/szepeviktor/debian-server-tools/blob/master/CV.md
ügyelet/hotline: +36-20-4242498<EMAIL_ADDRESS>skype: szepe.viktor
Budapest, III. kerület

Hmm. You're right, works! Strange why I didn't thought of it.

Strange why I didn't thought of it.

Well, I think because I expected the @mixin to tell phpstan/larastan that "Relation is-a Builder".
Acceptable solution for me 😛 😏 Thanks!


calculator outputs entered number

Coverage decreased (-100.0%) to 0.0% when pulling 8f5ee53f5c7c92619dfc7d9b2ad01d2fbaa9e7fd on print_input into ee2b07f93ea0a40e7160fcd8904969c3a2c457e9 on master.


rPackages.rJava: set JAVA_HOME, fixes darwin

Sets JAVA_HOME as it was empty. This was not an issue on linux, but caused rPackages.rJava not to load on Darwin.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@ofborg build rPackages.rJava

Other than that, it works on x86_64-linux.
It's a bit weird that it doesn't work on darwin despite the explicit export JAVA_HOME. Is that the traditional way to set enc vars on darwin? do the other packages that manupulate JAVA_HOME work on darwin?
[ins] r$> library(rJava)

[ins] r$> example(.jcompare)

.jcmprr$> ## Don't show:
.jcmprr$> .jinit()

.jcmprr$> ## End(Don't show)
.jcmprr$> s <- .jnew("java/lang/String", "foo")

.jcmprr$> .jequals(s, "foo") # TRUE
[1] TRUE

.jcmprr$> .jequals(s, "foo", strict=TRUE) # FALSE - "foo" is not a Java object
[1] FALSE

.jcmprr$> t <- s

.jcmprr$> .jequals(s, t, strict=TRUE) # TRUE
[1] TRUE

.jcmprr$> s=="foo" # TRUE
[1] TRUE

.jcmprr$> ## Don't show:
.jcmprr$>
.jcmprr$> 	stopifnot(
.jcmpr+ 		.jequals(s, "foo"),
.jcmpr+ 		!.jequals(s, "foo", strict=TRUE),
.jcmpr+ 		.jequals(s, t, strict=TRUE),
.jcmpr+ 		s == "foo"
.jcmpr+ 	)

.jcmprr$> ## End(Don't show)
.jcmprr$>
.jcmprr$> Double <- J("java.lang.Double")

.jcmprr$> d1 <- new( Double, 0.0 )

.jcmprr$> d2 <- new( Double, 1.0 )

.jcmprr$> d3 <- new( Double, 0.0 )

.jcmprr$> d1 < d2
[1] TRUE

.jcmprr$> d1 <= d3
[1] TRUE

.jcmprr$> d1 >= d3
[1] TRUE

.jcmprr$> d1 > d2
[1] FALSE

.jcmprr$> # cannot compare a Double and a String
.jcmprr$> try( d1 < "foo" )
Error in .jcall("RJavaComparator", "I", "compare", .jcast(a), .jcast(b)) :
  NotComparableException: objects of class java.lang.Double and java.lang.String are not comparable

.jcmprr$> # but can compare a Double and an Integer
.jcmprr$> d1 < 10L
[1] TRUE

.jcmprr$> ## Don't show:
.jcmprr$> 	stopifnot(
.jcmpr+ 		d1 < d2       ,
.jcmpr+ 		d1 <= d3      ,
.jcmpr+ 		d1 >= d3      ,
.jcmpr+ 		! (d1 > d2 )  ,
.jcmpr+ 		inherits( try( d1 < "foo", silent = TRUE ), "try-error" ),
.jcmpr+ 		d1 < 10L )

.jcmprr$> ## End(Don't show)
.jcmprr$>
.jcmprr$>
.jcmprr$>
.jcmprr$>
.jcmprr$>

[ins] r$> Sys.getenv("JAVA_HOME")
[1] "/nix/store/3vhlpdyqkrwk9arf84iaxk935r71rxvx-openjdk-21.0.5+11"

I'm surprised it even worked with the env-var only an being set at build time.
(I haven't looked at the code, maybe its expected to work like that)


adding test for issue #196

test for https://github.com/showdownjs/showdown/issues/196
but it is not passing

caution: depending on the fix, the test might need to be updated: changing ">" to ">" in the expected result.


Add pre-commit.ci badge

Let's keep the linting job for a while until we are fully satisfied with pre-commit.ci. 👍

cc @asottile 🤗

total time to completion: 5.3s zoooooom 🚗 💨


Fix Styling & Alignment on Search

When a search is done and results are returned, the cards are scattered all over the place.
We need to make sure the alignment remains the same as that before search while still only showing results matching the search.

Resolved by #85


[Workspaces] Taskbar Icon Order Not Preserved

Microsoft PowerToys version
0.84.0
Installation method
GitHub
Running as admin
Yes
Area(s) with issue?
Workspaces
Steps to reproduce
Open the apps you want for your workspace so you have those icons on the taskbar. Save your workspace layout.
When you load that layout the taskbar icon order is not preserved.
✔ Expected Behavior
Taskbar icons should maintain their original position when loading a saved workspace.
Taskbar icons should maintain their position when rearranged by user and switching between virtual desktops.
❌ Actual Behavior
Taskbar icon order is not preserved.
Loading a saved workspace results in icons for fastest opening programs appearing first on the taskbar.
Taskbar icon order is also not preserved when switching between virtual desktops if those icons were rearranged.
Other Software
No response

Regarding the order of the apps there is no ordering implemented so you can take it as random. (At the point of creating a workspace we get a list of apps from the system, where I suppose the last modified window is on the top. We store it with no modification on the order and when launching we take this list).
However if it helps you, you can manually modify the workspace file, which can be found in
c:\Users\ Yours \AppData\Local\Microsoft\PowerToys\Workspaces\workspaces,json
If you look into the file, search for your workspace (there are all the workspaces) and under the "applications" you can freely modify the order of your apps. Just beware of changing the syntax, I mean watch out for the commas and so.
To be honest the issue (or feature request) to enable the ordering in the editor will be low prio due to having more disturbing issues. Thanks for your understanding, best regards


ADJUST_RESIZE validation

Hi,
thank you for such a useful lib!
I am not sure that code below correctly check that activity has WindowManager.LayoutParams.SOFT_INPUT_ADJUST_RESIZE
may be
WindowManager.LayoutParams.SOFT_INPUT_ADJUST_RESIZE & softInputMethod  != 0

?
in my case  softInputMethod == 275 which tells us that activity is ADJUST_RESIZE
KeyboardVisibilityEvent.java
        int softInputMethod = activity.getWindow().getAttributes().softInputMode;
        if (WindowManager.LayoutParams.SOFT_INPUT_ADJUST_UNSPECIFIED != softInputMethod &&
                (WindowManager.LayoutParams.SOFT_INPUT_ADJUST_RESIZE > softInputMethod ||
                        WindowManager.LayoutParams.SOFT_INPUT_ADJUST_PAN <= softInputMethod)) {
            throw new IllegalArgumentException("Parameter:activity window SoftInputMethod is not ADJUST_RESIZE");
        }

thisnk you

Indeed this looks more correct approach to a bit flag.
Would you like to submit a PR?

this may also fix #37 ?

this may also fix #37 ?

Delaying did not fix

I mean checking the flag this way may fix #37.

Hi, Do you plan to merge my pull request? I experience crashes in my prod env right now(

give me few hours, I'm reviewing now.

I've just released v2.3.0.
Thanks for your contribution!


IIS metrics not scraped

I used below thing to get iis metrics, but i am not able to see iis metrics are not getting
msiexec /i  ENABLED_COLLECTORS=os,iis LISTEN_PORT=9182
I used wmi_exporter-0.4.3-386 and prometheus-2.3.2

Hi @anjaneyuludesu,
Can you see the metrics by accessing localhost:9182/metrics from the machine where you installed the exporter? Have you checked if there are any errors logged?

I used the below commandline the i am able to see
wmi_exporter.exe --collectors.enabled "cpu,iis"


Issue with simulation context definition in optimization goal

There doesn't seem to be an argument in add_optimization() and other related optimization functions, that accepts simulation context (e.g. Infinite Sphere). Only 'domain' is available, which probably should be simulation context instead.
Once this is corrected, would need to know how to add multiple 'Calc. Range',  for eg. Theta(-10deg:10deg), Phi("0deg"),Freq(10GHz) in the goal.

@zebanaqviWAT  I'm refactoring Optimetrics Classes in Pyaedt. I'll inform when done


Fix compile error for GHC 7.10.1

I have no idea why flip does not work here.  I suspect a GHC bug.  Anyway, this seems like a practical workaround.

Here it is explained what is going on: http://stackoverflow.com/questions/29712710/polymorphic-flip-fails-in-7-10

It seems compilation fails on 7.6 in Travis so I couldn't merge this commit. But I confirmed your solution works well, so I followed your fix in commit d8db2fa. Thanks!


[Request] support for www.gfycat.com

Ripme version: 17.74
Java version: 1.8.0_1.9.1 Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Operating system:
Windows 10 Enterprise N edition version 1809 OS Build 17763.195
Exact URL you were trying to rip when the problem occurred: https://gfycat.com
Please include any additional information about how to reproduce the problem:

aditional download in bulk downloader in whole profile, example link
https://gfycat.com/@golbanstorage
aditional configuration save without folder file

Please make all site requests in #38