Publish pre-compiled wheels to PyPI

didn't find any previous issue on this but apologies if this was discussed before

are there are blockers (technical or development time) for publishing pre-compiled wheels for phono3py to PyPI? would be great to be able to install with pip/uv

there's a very nice project directly from the python packaging authority PyPA) https://github.com/pypa/cibuildwheel which handles a lot of the platform-specific complexity of building wheels for every OS and Python version. i've used it on a few projects and had good experiences. i don't know if it handles all the needs of phono3py but if it wasn't tried yet, probably worth a shot

Thanks, @janosh! I used it a long time ago, but back then, it would sometimes stop working unexpectedly for reasons I couldn't figure out. I often spent a day or more troubleshooting since I didn’t fully understand how it worked. Eventually, I decided, 'This isn’t really my job,' and let it go.
Since users sometimes express complaints, and software development continues to be unfairly undervalued in academia, I need to be careful not to exhaust myself to avoid losing the passion to continue scientific software development.

Lapacke is a bit tricky and most likely I can't make it performant, but I can port the spglib build system here. I have one project to attend to this week, but I can add this ond to my todo for next week. @atztogo all ok sith refactoring the build system here?

@atztogo all ok sith refactoring the build system here?

Sure, thanks. But please don't feel too obligated.

No worries, this is my bread-and-butter and my new job at Qt is also cmake oriented.

Having this would be really useful for my new atomate2 anharmonic phonons workflow

are there are blockers (technical or development time) for publishing pre-compiled wheels for phono3py to PyPI? would be great to be able to install with pip/uv


Can you add to the readme of what this project is about?

Hello
I am a Flutter developer. I have worked with a couple of projects before as part of a small internship and an open source program. Can you let me know what this project is about so that I can see what I can contribute to it as part of cross winter of code?
Thanks!

Check in this I have explain all the topic https://youtu.be/t1Fo3KUx1_k


fix: disable subtitles by default

Disable subtitles by default.
Subtitles will be shown only after player initialization

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Facebook doesn't accept OAuth Redirect URI specified by tutorial

Per the instructions in the readme, I tried to register a new OAuth Redirect URI via Facebook's dev site. Here's the relevant part:

Before we leave the Facebook settings, we need to tell Facebook we have a new redirect url to register. Navigate to the bottom of the page and add the following into the bundle ID field:
fb{YOUR_APP_ID}
We'll need to create a new URL scheme for Facebook and (this is a weird bug on the Facebook side) the facebook redirect URL scheme must be the first one in the list. The URL scheme needs to be the same id as the Bundle ID copied from above.

Facebook won't accept the new redirect URL scheme. It says:

[0] should represent a valid URL

I know this isn't exactly a bug in react-native-oauth, but it certainly suggests the instructions in this repo are no longer accurate, which seems like a valid doc issue.

I needed to use
http://localhost/fb314432522286765


Waystone Config

The current config has an option to make players toggle a waystone public or not which doesn't seem to work for the owners of the waystone, would it be possible to either update that one to be all players that aren't opped or a separate one for the waystone owners? Thanks in advance.

I improved the configs in general :) Now finally with modmenu support.


[Bug]: The command line in prerequisite can involve a typo

Issue Type
Documentation
Modules Involved
Documentation/Tutorial/Example, Others
Have you reproduced the bug with SPU HEAD?
Yes
Have you searched existing issues?
Yes
SPU Version
spu 0.9.0
OS Platform and Distribution
Linux Ubuntu 22.04
Python Version
3.10
Compiler Version
GCC 11.4
Current Behavior?
This line sets $BAZEL_ARCH = arm64 in both of the control flows.
if [ "$TARGETPLATFORM" = "linux/arm64" ] ; then BAZEL_ARCH=arm64 ; else BAZEL_ARCH=amd64 ; fi \

Standalone code to reproduce the issue
Nothing more here.

Relevant log output
No response

link is here


📢 Azure/cli v2 Release Announcement

👀 What’s new in azure/cli@v2?

The default runtime of azure/cli is updated from Node16 to Node20 due to the upcoming deprecation of Node16.

⚠ This is breaking change

If you're using azure/cli with a self-hosted runner, ensure it's running Actions Runner image v2.308.0 or later.

To apply this change, please update the version of azure/cli to v2 in your workflow files.
    - name: Run Azure CLI Action
      uses: azure/cli@v2
      with:
        azcliversion: latest
        inlineScript: |
          az version

I just upgraded to v2 from v1, I can confirm it works fine. the warning around node 16 is gone.
with v1

with v2
No node 16 warning


Create an intro

We really need some basic intro information asap!

My name is Nikolay and I really want to become a developer 😃 (backend or frontend, as for my plans, i'm trying myself in both directions). I have recently finished a GOIT html+css+javascript course, i want to practice more and grow as a developer. I'm fond of developing and getting new knowledge. I also want to learn some frameworks, for right now my main target is react.js and all its pitfalls 😄
I hope that after completing this course, I will master all the necessary skills.

Task has been done.


Should needsEmailVerification be considered an error?

Both useSignUpEmailPassword and useSignInEmailPassword have a needsEmailVerification property.
Moreover, useSignInEmailPassword also have a needsMfaOtp property.
However, when their value turns to be true, it is not flagged as an error, and isSuccess will return true when the handler has been called.
I think it makes sense when authenticating with useSignUpEmailPassword, as a user has been indeed added, but they still need to verify their email to authenticate. In that sense, it is not an error.
But it is less obvious with useSignInEmailPassword: the authentication actually failed if the email needs to be verified, and we should consider it an error.
To correct this behaviour will however introduce a breaking change, although minor.
Other hook using email verification: useChangeEmail.
@szilarddoro @elitan what do you think?

It depends, because some apps are not restricting access to some of their services if the user has not verified themselves. Is it technically possible now to access features without verifying your email address?

yes, if email verification is not activated in the backend, then  authentication/registration succeeds straight ahead: user is returned, isSuceess is true and needsEmailVerification is false.
Note: needsEmailVerification will be true, but the email will still be flagged as unverified in the DB


[Feature Request] Enforce std/core when there's an option

for some set of imports like this:
use core::marker::PhantomData;
use std::collections::HashMap;
use std::pin::Pin;

it would be nice to have an fmt option to enforce either the lowest possible import, or always use std.
use core::marker::PhantomData;
use core::pin::Pin;
use std::collections::HashMap;

or
use std::marker::PhantomData;
use std::collections::HashMap;
use std::pin::Pin;

not sure how this would interact with alloc

this could also apply to qualified names

Could you expand on what specifically you are asking for? It's not entirely clear what "lowest possible import" means. It'd also be helpful to explain your ask in relation to relevant existing configuration options, e.g. reorder_imports, group_imports, etc.

Lowest as in most specific. Really the setting would be a lint to specify how importing items available in core should be written. Should they be imported from std or core?

Additionally, this lint would happen before those two lints so sorting works correctly, and grouping is more effective.

Gotcha thanks for the clarification. However, I am going to close as that's outside the scope of style/formatting and is too far beyond our core remit for me to be willing to consider incorporating in rustfmt.
Thanks again for the suggestion though!


_onReady event not firing anymore in Chrome

I am aware that this is most likely not a problem by the react-youtube player component itself.
The error occurs in this file that is used by the react-youtube player:
https://www.youtube.com/yts/jsbin/player-de_DE-vflM_cNgA/base.js
I just wanted to check if anybody else is experiencing this problem...
This is the error I get upon initialization of the react-youtube component:
Uncaught TypeError: a.g.setActionHandler is not a function
    at J_ (https://www.youtube.com/yts/jsbin/player-de_DE-vflM_cNgA/base.js:3784:24)
    at new K_ (https://www.youtube.com/yts/jsbin/player-de_DE-vflM_cNgA/base.js:3779:111)
    at new Y1 (https://www.youtube.com/yts/jsbin/player-de_DE-vflM_cNgA/base.js:3974:367)
    at new d2 (https://www.youtube.com/yts/jsbin/player-de_DE-vflM_cNgA/base.js:3995:210)
    at d2.create (https://www.youtube.com/yts/jsbin/player-de_DE-vflM_cNgA/base.js:6993:333)
    at Gi.<anonymous> (https://www.youtube.com/yts/jsbin/www-embed-player-vfl-i_LLs/www-embed-player.js:507:269)
    at Gi.l.jb (https://www.youtube.com/yts/jsbin/www-embed-player-vfl-i_LLs/www-embed-player.js:507:308)
    at Ki (https://www.youtube.com/yts/jsbin/www-embed-player-vfl-i_LLs/www-embed-player.js:505:143)
    at Gi.l.oa (https://www.youtube.com/yts/jsbin/www-embed-player-vfl-i_LLs/www-embed-player.js:498:176)
    at https://www.youtube.com/yts/jsbin/www-embed-player-vfl-i_LLs/www-embed-player.js:557:378

Will check if the error persists, and update this issue accordingly.

Okay, this went away... must have been a temporary error in one of the youtube-api scripts...

Yea the error stack seems to indicate that this problem was caused on YouTube's end.
You already closed the issue but if it persists in the future feel free to reopen it.

base.js:30 Uncaught TypeError: Cannot read property 'prototype' of undefined
at Object.g.p (base.js:30)
at remote.js:79
at remote.js:162
g.p @ base.js:30
(anonymous) @ remote.js:79
(anonymous) @ remote.js:162
annotations_module.js:419 Uncaught TypeError: Cannot read property 'prototype' of undefined
at annotations_module.js:419
at annotations_module.js:697


Latch incorrectly inferred on initialized local variable

Hello.
I found a small bug that inline-initialization of local variables does not prevent latch inference.
Version: 2023-11-01-b746752
module latch_test2 (
    input logic a,
    input logic b
);

    always_comb begin
        logic temp = 0; // inline initialization does not prevent latch
        if (a)
            temp = b;
    end

endmodule

Error:
2.2.8. Executing PROC_DLATCH pass (convert process syncs to latches).
ERROR: Latch inferred for signal `\latch_test2.$unnamed_block$0.temp' from always_comb process `\latch_test2.$proc$latch_test2.sv:6$1'.

However, this implementation works.

    always_comb begin
        logic temp;
        temp = 0; // this correctly prevents a latch
        if (a)
            temp = b;
    end

AST with inline initialization (latch inferred)
AST_MODULE <latch_test2.sv:1.1-12.10> [0x561763f33030] str='\latch_test2'
  AST_WIRE <latch_test2.sv:2.17-2.18> [0x5617640b34d0] str='\a' input logic port=1 multirange=[ 0 1 ] multirange_swapped=[ 0 ]
    AST_RANGE <latch_test2.sv:0.0-0.0> [0x561763db39e0] basic_prep range=[0:0] in_param
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x561763f00910] bits='00000000000000000000000000000000'(32) signed basic_prep range=[31:0] in_param
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x561763fffbb0] bits='00000000000000000000000000000000'(32) signed basic_prep range=[31:0] in_param
  AST_WIRE <latch_test2.sv:3.17-3.18> [0x561763f37de0] str='\b' input logic port=2 multirange=[ 0 1 ] multirange_swapped=[ 0 ]
    AST_RANGE <latch_test2.sv:0.0-0.0> [0x56176410a5d0] basic_prep range=[0:0] in_param
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x561763e9d800] bits='00000000000000000000000000000000'(32) signed basic_prep range=[31:0] in_param
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x56176400c510] bits='00000000000000000000000000000000'(32) signed basic_prep range=[31:0] in_param
  AST_ALWAYS <latch_test2.sv:6.5-10.8> [0x561764004b20]
    ATTR \always_comb:
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x561763fda590] bits='00000000000000000000000000000001'(32) range=[31:0] int=1 in_param
    AST_BLOCK <latch_test2.sv:6.17-10.8> [0x561763db17c0]
      AST_BLOCK <latch_test2.sv:6.17-10.8> [0x561763edeb30] str='$unnamed_block$0'
        AST_WIRE <latch_test2.sv:7.15-7.19> [0x561763dbe280] str='\temp' logic
          AST_RANGE <latch_test2.sv:0.0-0.0> [0x561763fa3a40] range=[0:0] in_param
            AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x561763ec1e70] bits='00000000000000000000000000000000'(32) signed range=[31:0] in_param
            AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x56176409de70] bits='00000000000000000000000000000000'(32) signed range=[31:0] in_param
        AST_ASSIGN_LE <latch_test2.sv:7.15-7.23> [0x561763dcdf50]
          AST_IDENTIFIER <latch_test2.sv:7.15-7.19> [0x561763dce100] str='\temp' logic in_lvalue
          AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x561763dce2b0] bits='00000000000000000000000000000000'(32) signed range=[31:0]
        AST_CASE <latch_test2.sv:8.9-9.22> [0x561763e87b70]
          AST_REDUCE_BOOL <latch_test2.sv:0.0-0.0> [0x561763dc51b0]
            AST_IDENTIFIER <latch_test2.sv:8.13-8.14> [0x561763ec13a0] str='\a'
          AST_COND <latch_test2.sv:0.0-0.0> [0x561763ddf9f0]
            AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x561763dceb50] bits='1'(1) range=[0:0] int=1
            AST_BLOCK <latch_test2.sv:0.0-0.0> [0x5617640a2540]
              AST_ASSIGN_EQ <latch_test2.sv:9.13-9.21> [0x5617640bf6a0]
                AST_IDENTIFIER <latch_test2.sv:9.13-9.17> [0x561763eb8830] str='\temp' in_lvalue
                AST_IDENTIFIER <latch_test2.sv:9.20-9.21> [0x561763ef0dc0] str='\b'

AST with inilization on different line (no latch inferred)
AST_MODULE <latch_test2.sv:1.1-19.10> [0x55e02e93dfc0] str='\latch_test2'
  AST_WIRE <latch_test2.sv:2.17-2.18> [0x55e02e8be180] str='\a' input logic port=1 multirange=[ 0 1 ] multirange_swapped=[ 0 ]
    AST_RANGE <latch_test2.sv:0.0-0.0> [0x55e02e6e4050] basic_prep range=[0:0] in_param
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02e5bbce0] bits='00000000000000000000000000000000'(32) signed basic_prep range=[31:0] in_param
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02e607d80] bits='00000000000000000000000000000000'(32) signed basic_prep range=[31:0] in_param
  AST_WIRE <latch_test2.sv:3.17-3.18> [0x55e02ea49130] str='\b' input logic port=2 multirange=[ 0 1 ] multirange_swapped=[ 0 ]
    AST_RANGE <latch_test2.sv:0.0-0.0> [0x55e02e8450f0] basic_prep range=[0:0] in_param
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02e89cc60] bits='00000000000000000000000000000000'(32) signed basic_prep range=[31:0] in_param
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02ec1bd80] bits='00000000000000000000000000000000'(32) signed basic_prep range=[31:0] in_param
  AST_ALWAYS <latch_test2.sv:12.5-17.8> [0x55e02ef31770]
    ATTR \always_comb:
      AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02e8bd270] bits='00000000000000000000000000000001'(32) range=[31:0] int=1 in_param
    AST_BLOCK <latch_test2.sv:12.17-17.8> [0x55e02ee6f790]
      AST_BLOCK <latch_test2.sv:12.17-17.8> [0x55e02ee74060] str='$unnamed_block$0'
        AST_WIRE <latch_test2.sv:13.15-13.19> [0x55e02edaa9b0] str='\temp' logic
          AST_RANGE <latch_test2.sv:0.0-0.0> [0x55e02e7153d0] range=[0:0] in_param
            AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02e62bc30] bits='00000000000000000000000000000000'(32) signed range=[31:0] in_param
            AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02e6525d0] bits='00000000000000000000000000000000'(32) signed range=[31:0] in_param
        AST_ASSIGN_EQ <latch_test2.sv:14.9-14.17> [0x55e02ee80bf0]
          AST_IDENTIFIER <latch_test2.sv:14.9-14.13> [0x55e02ed69820] str='\temp' in_lvalue
          AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02ed55f30] bits='00000000000000000000000000000000'(32) signed range=[31:0]
        AST_CASE <latch_test2.sv:15.9-16.22> [0x55e02edafef0]
          AST_REDUCE_BOOL <latch_test2.sv:0.0-0.0> [0x55e02ef2be50]
            AST_IDENTIFIER <latch_test2.sv:15.13-15.14> [0x55e02ef20e60] str='\a'
          AST_COND <latch_test2.sv:0.0-0.0> [0x55e02e8b2400]
            AST_CONSTANT <latch_test2.sv:0.0-0.0> [0x55e02e94c620] bits='1'(1) range=[0:0] int=1
            AST_BLOCK <latch_test2.sv:0.0-0.0> [0x55e02ec1d810]
              AST_ASSIGN_EQ <latch_test2.sv:16.13-16.21> [0x55e02e6d4130]
                AST_IDENTIFIER <latch_test2.sv:16.13-16.17> [0x55e02eac2a10] str='\temp' in_lvalue
                AST_IDENTIFIER <latch_test2.sv:16.20-16.21> [0x55e02ec1c5c0] str='\b'

Fixed in https://github.com/alaindargelas/synlig/tree/stable_rs


hasNewTasksEvent signal results in infinite look on Apple Silicon

Hey, I noticed that the example project does not work on Apple Silicon. It causes an infinite loop in MTScheduler.cpp
line 719 (the context.hasNewTasksEvent.Signal(); takes infinite time).
The Run(MT::FiberContext& context) function is empty so it should finish immediately.
Im using the latest version from Github.
MacOS version: 11.2.2

gogo fix, i need for Metal Raytracing Game Engine

Same on my MacBook Air M1 :)

it's a shame that there was absolutely no response. I assume that means the library is no longer being developed. Still, it would have been decent to at least communicate that clearly.
I can recommend Marl from Google to anyone who will be reading this, it's a very good library.

@DaeBasduhr It seems like he is not very active on github at the moment, If you really want to fix it you can fork it and fix it yourself.

Sorry guys, due to a few recent GitHub mail storms, most of the emails from GitHub end up in the spam folder.
I plan to retrofit TaskSceduler later this year. I'll be switching to CMake, adding support for M1/M2, and faster fibers. I don't have a specific timeline for this yet, as I'm busy with my main job. But I'm excited to make these updates and improve the project overall!
P.S. Pull Requests are welcome!

@SergeyMakeev are u fine if I already port the scheduler to cmake?


Add serialization/deserialization hook support for structs/unions

Similarly to how std.fmt supports adding a format() function to a struct or union, s2s should support it for custom serialization/deserialization or for when a type isn't fully supported. This would help with unsupported types that can't easily be serialized due to recursiveness.

I want to change s2s format in a breaking way so that arbitrary graphs can be serialized instead of a pure tree structure, so this issue will be resolved. Then, all runtime viable types are actually serializable with s2s except for a [*]T


Error 500 when requesting another connector's description

Dear DataspaceConnector-Team,
we've setup two connectors on a server. After succesfully registering a data resource and corresponding representation on one, we tried accessing it from the other one (which runs on another port). First, we tried requesting the connector's description using POST/admin/api/request/description, passing the url http://{ip-address}:{port}/api/ids/data. We configured the relevant files in the repository for http to work.
This worked in older versions. With the latest commit however, we obtain an error message in the response body:
Failed to send description request message.
Did anyone else encounter this yet?

Hello Vincent,
we also encountered the problem and commitet a fix (a87fa468ad21ad8da15bc72a923d8c5181648d7c). Please check if the fix solves your problem.


1.4.3 shipping with old compressed templatetags

1.4.0 marked the old templatetags as backwards incompatible, but it appears that at least 1.4.3 ships with the old templatetags, as well as the new ones - making the change rather backwards compatible.
Since the file is not present in the Git repository anymore, I suspect a small error in the packaging.

Thanks!
Fixed in <IP_ADDRESS>.


silent microphone

Issue description
I am using pulseaudio, audio output works but input doesn't.
Microphones are detected (running as shown by pacmd list-source), source selected, channels aren't muted, volume up.
I don't really know how to check if it comes from pulseaudio or not actually and don't know what kind of information I should provide here.
Steps to reproduce
hardware.pulseaudio = {
  enable = true;
  support32Bit = true;
};

Technical details

System: 17.03.1188.9fd6417e27 (Gorilla)
Nix version: nix-env (Nix) 1.11.8
Nixpkgs version: "17.03.1188.9fd6417e27"
Sandboxing enabled: build-use-sandbox = false

@lambdax-x Have you also checked the settings in alsamixer?

Also, there is sometimes a magic invocation with amixer sset [name] cap

Everything seems okay in alsamixer (nothing muted, volumes up).
Also I found an option to enable the microphone in the bios but it's set up correctly.

Sorry, closed the issue unintentionally...

It should be ok with amixer :
$ amixer
Simple mixer control 'Master',0
  Capabilities: pvolume pvolume-joined pswitch pswitch-joined
  Playback channels: Mono
  Limits: Playback 0 - 74
  Mono: Playback 67 [91%] [-7.00dB] [on]
Simple mixer control 'Headphone',0
  Capabilities: pvolume pswitch
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 74
  Mono:
  Front Left: Playback 74 [100%] [0.00dB] [on]
  Front Right: Playback 74 [100%] [0.00dB] [on]
Simple mixer control 'Speaker',0
  Capabilities: pvolume pswitch
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 74
  Mono:
  Front Left: Playback 65 [88%] [-9.00dB] [on]
  Front Right: Playback 65 [88%] [-9.00dB] [on]
Simple mixer control 'PCM',0
  Capabilities: pvolume
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 255
  Mono:
  Front Left: Playback 255 [100%] [0.00dB]
  Front Right: Playback 255 [100%] [0.00dB]
Simple mixer control 'Mic Boost',0
  Capabilities: volume
  Playback channels: Front Left - Front Right
  Capture channels: Front Left - Front Right
  Limits: 0 - 4
  Front Left: 4 [100%] [40.00dB]
  Front Right: 4 [100%] [40.00dB]
Simple mixer control 'IEC958',0
  Capabilities: pswitch pswitch-joined
  Playback channels: Mono
  Mono: Playback [off]
Simple mixer control 'Beep',0
  Capabilities: pvolume pvolume-joined pswitch pswitch-joined
  Playback channels: Mono
  Limits: Playback 0 - 7
  Mono: Playback 1 [14%] [-24.00dB] [off]
Simple mixer control 'Capture',0
  Capabilities: cvolume cswitch
  Capture channels: Front Left - Front Right
  Limits: Capture 0 - 80
  Front Left: Capture 80 [100%] [6.00dB] [on]
  Front Right: Capture 80 [100%] [6.00dB] [on]
Simple mixer control 'Auto-Mute Mode',0
  Capabilities: enum
  Items: 'Disabled' 'Enabled'
  Item0: 'Enabled'
Simple mixer control 'Digital',0
  Capabilities: cvolume
  Capture channels: Front Left - Front Right
  Limits: Capture 0 - 120
  Front Left: Capture 120 [100%] [30.00dB]
  Front Right: Capture 120 [100%] [30.00dB]
Simple mixer control 'Internal Mic Boost',0
  Capabilities: volume
  Playback channels: Front Left - Front Right
  Capture channels: Front Left - Front Right
  Limits: 0 - 4
  Front Left: 4 [100%] [48.00dB]
  Front Right: 4 [100%] [48.00dB]

Please run amixer sset Capture cap, from the output I see that you do need to run it.

I don't think I need to:
Simple mixer control 'Capture',0
  Capabilities: cvolume cswitch
  Capture channels: Front Left - Front Right
  Limits: Capture 0 - 80
  Front Left: Capture 80 [100%] [6.00dB] [on]   # <-- is on
  Front Right: Capture 80 [100%] [6.00dB] [on]  # <-- also

on/off, numeric scale and cap/nocap are three different dimensions, you
reference the lines showing the first two of them.

Could you please elaborate on how you see that? Because I can't see and experience any difference after executing this command.

cswitch + similarity to the layout of my card where I had a lot of headbanging with that.
Hm. Are there any hardware buttons that could be capture-related? Also, do you have a combined jack with headphones without microphone inserted? Also, what amixer contents say, it is more detailed than amixer…

It's kind of obscure but after the night, the mic input is no longer muted...
Looks like there has been a sacrifice for the alsa devil.

My mic was auto muted and capture audio was off. I went through alsamixer and just messed with it.. You have to make sure auto mute is off and capture audio is on(the higher, the better your mic pics up). To see all sound options use f1 to see what sound card options you have(im running headphones with mic on them). f6 for sound card then see what other options you have for sound cards. If any options, mess around with them, find mic options and capture and that should do it. I fixed it without having to mess with any code.


Errors in reading BNGL files should be reported consistently

bionetgen.bngmodel raises errors for some problems, but prints messages to stdout for others. It would be helpful for errors to be reported in a consistent way.
Example:
https://github.com/ASinanSaglam/PyBioNetGen/blob/378d1d639efaa96c2be2f2a11c646e7a5328b379/bionetgen/xmlapi/bngparser.py#L59
Could be replaced with the following to be consistent with the report of other errors
raise ValueError("XML file couldn't be generated")

Added more error raising. Along with #5 and #6, this will likely be an ongoing effort.


Proxy-man not found in production build

[x] I have checked that the [SDK documentation][sdk-docs] doesn't solve my issue.
[x] I have checked that the [API documentation][api-docs] doesn't solve my issue.
[x] I have searched the [Box Developer Forums][dev-forums] and my issue isn't already reported (or if it has been reported, I have attached a link to it, for reference).
[x] I have searched [Issues in this repo][github-repo] and my issue isn't already reported.

Description of the Issue
When running a production Next.js application that uses the SDK, I receive an error indicating the box sdk cannot access proxy-agent.
The app works fine in development mode but fails after bundling the application into a production Docker container.
I am using a developer token for authentication and testing against the uploadBigFile function. As mentioned, this works successfully in development, but not in a production build. I've tested adding proxy-man to package.json directly but the issue is not resolved.
Error Message, Including Stack Trace
  ▲ Next.js 14.2.14
  - Local:        http://localhost:3000
  - Network:      http://<IP_ADDRESS>:3000

 ✓ Starting...
 ✓ Ready in 184ms
File upload error: Error: Cannot find module 'proxy-agent'
Require stack:
- /app/node_modules/box-typescript-sdk-gen/lib/internal/utils.js
- /app/node_modules/box-typescript-sdk-gen/lib/networking/network.generated.js
- /app/node_modules/box-typescript-sdk-gen/lib/box/ccgAuth.generated.js
- /app/node_modules/box-typescript-sdk-gen/lib/box/index.js
- /app/node_modules/box-typescript-sdk-gen/lib/index.js
- /app/.next/server/pages/api/upload.js
- /app/node_modules/next/dist/server/future/helpers/module-loader/node-module-loader.js
- /app/node_modules/next/dist/server/future/helpers/module-loader/route-module-loader.js
- /app/node_modules/next/dist/server/next-server.js
- /app/node_modules/next/dist/server/next.js
- /app/server.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1140:15)
    at /app/node_modules/next/dist/server/require-hook.js:55:36
    at Module._load (node:internal/modules/cjs/loader:981:27)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at mod.require (/app/node_modules/next/dist/server/require-hook.js:65:28)
    at require (node:internal/modules/helpers:177:18)
    at createAgent (/app/node_modules/box-typescript-sdk-gen/lib/internal/utils.js:376:39)
    at new NetworkSession (/app/node_modules/box-typescript-sdk-gen/lib/networking/network.generated.js:11:49)
    at new BoxClient (/app/node_modules/box-typescript-sdk-gen/lib/client.generated.js:78:31)
    at u (/app/.next/server/pages/api/upload.js:1:1533) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/app/node_modules/box-typescript-sdk-gen/lib/internal/utils.js',
    '/app/node_modules/box-typescript-sdk-gen/lib/networking/network.generated.js',
    '/app/node_modules/box-typescript-sdk-gen/lib/box/ccgAuth.generated.js',
    '/app/node_modules/box-typescript-sdk-gen/lib/box/index.js',
    '/app/node_modules/box-typescript-sdk-gen/lib/index.js',
    '/app/.next/server/pages/api/upload.js',
    '/app/node_modules/next/dist/server/future/helpers/module-loader/node-module-loader.js',
    '/app/node_modules/next/dist/server/future/helpers/module-loader/route-module-loader.js',
    '/app/node_modules/next/dist/server/next-server.js',
    '/app/node_modules/next/dist/server/next.js',
    '/app/server.js'
  ]
}

Versions Used
Typescript SDK: 1.6.0
Platform: Node 18, Next 14.2.14 (Pages router, using /api)

Hi @trevordebard
Please take a look at this solution and let us know if it's working for you: https://github.com/box/box-typescript-sdk-gen/issues/213#issuecomment-2285387965
Bests,
Minh


How can I accept bitcoin using NBitcoin?

I have read your book and I know how to send coin and work with block chain.
I am wondering how can I accept bitcoin?
I know it sound silly that I just need to give the payer my address then he can send me the coin.
But I want to provide service for my website to accept bitcoin, how can I query the blockchain to confirm the transaction is OK and I am ready to give the payer my product or service in a short time? If it need 15 minutes or hours more might be not good for payer to wait.
I am sorry I hit the button to create the title but it just post the message.

Thank you so much for your fast reply and your great work to let me understand the bitcoin programming in a few days.
I'd like build service like GoURL.io for my website.
So I need to create  P2PKH address for my clients to pay then I use NBXp to track the UTXOs status, Correct?
Is there any method I can leverage to instant confirm the transaction to prevent my client wait 15 mins or more? Or it's still not possible in current environment?
Another question is do I need to rebuild your ProgrammingBitcoinFunding website to learn more about the bticoin coding? I found the datebase ProgrammingBitcoinFunding-20150114124016.mdf is not included in your files.
Thanks again for your kind help.

No that's fine you don't need this.
Yes you use NBx to track UTXOs. You can't instant confirm, the merchant should decide to take the risk to be double spent if he wishes.
If you just want to help a merchant, just use BTCPay which is a service self hostable for merchant (it uses nbx under the hood):

https://www.youtube.com/channel/UCpG9WL6TJuoNfFVkaDMp9ug
https://github.com/btcpayserver/btcpayserver

@ScofieldYeh
If I'd be you, I'd just use a public API: https://www.smartbit.com.au/api
Alongside with my KeyManager: https://github.com/nopara73/MagicalCryptoWallet/blob/master/MagicalCryptoWallet/KeyManagement/KeyManager.cs

Public API can't query over a xpub. You need to query all addresses of your wallets to detect payments.

Many of them can, even the one you wrote (QBitNinja.) That being said, it's not like it'd be needed in this case, he can periodically query only one address, that's just about to get paid. Or even better, SmartBit has websocket notifications for addresses.

Generate new address.
Subscribe to websocket notification for the address.
Wait until the customer pays.
Unsubscribe.

NBxplorer has also websocket notification. (https://github.com/dgarage/NBXplorer/blob/master/docs/API.md)

But you must run a full node with that, don't you?

yes indeed. So this depends on whether this is on the table or not.

I am curious about why we use address to monitor or query the transaction results instead of the transaction IDs? Especially I would like to know how many blocks of a transaction is confirmed?

how can the payment server know about the transaction id?

       BroadcastResponse broadcastResponse = client.Broadcast(transaction).Result;

        if (!broadcastResponse.Success)
        {
            Console.Error.WriteLine("ErrorCode: " + broadcastResponse.Error.ErrorCode);
            Console.Error.WriteLine("Error message: " + broadcastResponse.Error.Reason);
        }
        else
        {
            Console.WriteLine("Success! You can check out the hash of the transaciton in any block explorer:");
            Console.WriteLine(transaction.GetHash());
        }

from your code can we use transaction.GetHash() to query how many confirmed it get?

yes


clarify compile distinction from preview.

pr to clarify compile distinction from preview. in response to this pr comment from @davidharting
What are you changing in this pull request and why?

Checklist

[ ] Review the Content style guide and About versioning so my content adheres to these guidelines.
[ ] Add a checklist item for anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch."

Adding new pages (delete if not applicable):

[ ] Add page to website/sidebars.js
[ ] Provide a unique filename for the new page

Removing or renaming existing pages (delete if not applicable):

[ ] Remove page from website/sidebars.js
[ ] Add an entry _redirects
[ ] Ran link testing to update the links that point to the deleted page

thank you very much for checking this and for flagging as well 🙏  @davidharting !


Autofixing quote is commenting out code

We have a few cases in our code base of multi line template strings like this:
{
  html: '\
      <iframe class="classes"\
              src="foo<%= foo %>">\
      </iframe>\
    '
}

When running the autofix for validateQuoteMark, it returns this:
{
  html: '\
/*  <iframe class="classes"\
              src="foo<%= foo %>*/">\
/*  </iframe>*/\
    '
}

How did those comment tags get in there?

Thanks for reporting @TheSavior. Any help investigating this is appreciated.


New Rust code generator

This PR provides full SBE code generation support for the Rust programming language.

generates 100% safe rust code
the rust code generated has zero dependencies on any external rust libs
both lib.rs and Cargo.toml files are generated so projects can easily add codec as a dependency
Fixes #813

Testing
# following will write generated code to "./generated/rust"
./gradlew generateRustCodec

# assuming you have "rust" and "cargo" already installed
cd ./rust
cargo test

This is failing Checkstyle.  Could you have a look at the error and provide a correction.  There appears to be some sort of whitespace problem.

This is failing Checkstyle. Please could you have a look at the error and provide a correction. There appears to be some sort of whitespace problem.

should be fixed

Overall this looks good.  I would like to see a few more tests.  Have a look at some of the other language bindings and some of the additional tests they have in place.  E.g. there are specific ones for groups and vardata.  There should definitely be a test showing that big endian byte order works.
@mward What are your plans for the Rust support moving forward?  Have you looked at an on-the-fly (OTF) decoder for Rust as well?

Overall this looks good. I would like to see a few more tests. Have a look at some of the other language bindings and some of the additional tests they have in place. E.g. there are specific ones for groups and vardata. There should definitely be a test showing that big endian byte order works.
@mward What are your plans for the Rust support moving forward? Have you looked at an on-the-fly (OTF) decoder for Rust as well?

@mikeb01 - I added a test for big endian byte ordering as well as a test for "issue 435".
I am using the Rust SBE implementation I've submitted, so I'm very interested in ensuring it is supported moving forward.  I have not yet needed an OTF decoder for Rust, but I'd be willing to take a look at implementing that after this PR is accepted. Ideally it would be great if I can convince some others that are interested in Rust and SBE to work on that with me.
Thanks!

I added a “rust-build” job to the CI workflow

@mikeb01 - are there any issues preventing this PR from being accepted?
Thanks

Are there any updates on this pull request? My team is also interested in SBE for a Rust application.

Are there any updates on this pull request? My team is also interested in SBE for a Rust application.

@joakimthun happy to see that others are interested in this PR. Have you had a chance to try it out? Let me know if you notice any issues.
Hopefully they'll accept this PR soon.
Thanks!

@mward not yet, will let you know if I find any issues when I do. Should be able to give it a try during next week.

@joakimthun Glad to hear it's working for you!
I figured it would be easier to have generated code be formatted via rust-fmt instead of trying to conform to those rules within SBE code generator.  Additionally, I did not want to be dependent on rust/cargo/rust-fmt being available when generating the code from the SBE tool.  Regardless, I'll see if I can improve format of generated code.
I'll fix that needless_borrow that clippy detected.  Must be a newer lint because I do run clippy often and don't recall seeing that.
Thanks again for the feedback!

Thanks @mward, lgtm!

Thanks! Merged.


Luckily the fix was easy :)

Could you include screenshots for smaller sized screens? Also the changes seem duplicate. Since they should be styled in a parent, should we put this in rite-app instead?

The change is duplicate to prevent the scrollbar from appearing in the middle of your screen. If you want to see how it works, just make your browser window smaller.

LGTM. Could not test for the full-HD size, since I do not have such a screen :smile:


[Aviraj] Added Ink Boxes follow up grant

Project Abstract
We are introducing NFT marketplace Ink Box as part of Ink Boxes. People will be able to spin up their own ink box fast.
Grant level

[x] Level 1:  Up to $10,000, 2 approvals
[ ] Level 2:  Up to $30,000, 3 approvals
[ ] Level 3:  Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

[x] The application template has been copied and aptly renamed (project_name.md).
[x] I have read the application guidelines.
[x] A BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (aUSD/USDT) address for the payment of the milestones is provided in the application.
[x] The software delivered for this grant will be released under an open-source license specified in the application.
[x] The initial PR contains only one commit (squash and force-push if needed).
[x] The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
[ ] I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hey @Noc2, Merry Christmas. I hope you are doing great.
I have completed the Ink Boxes grant. I have opened up a follow-up grant to create an NFT marketplace separately because creating an NFT marketplace is a huge task.
Can you please take a look at it today, if possible?
Thanks

Just saw the README. Merry Christmas and happy new year :)

@Noc2 , I am having one more doubt. Can I create follow up grants?
If not, I will close this and work on existing grants.

Sure. I will close this grant for now. Since I have money now, I will hire a front-end developer to make the website more pretty.


[BugFix] Fix wrong state of 'is_result_sink' of query context

Introduced by #29901
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[x] Yes, this PR will result in a change in behavior.
[ ] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.1
[ ] 3.0
[ ] 2.5
[ ] 2.4

@Mergifyio backport branch-2.5

@Mergifyio backport branch-3.0

@Mergifyio backport branch-3.1


python3.6 cannot use pymatting and report pymatting_dot cannot found in ubuntu [Question❓]

python3.6 cannot use pymatting and report pymatting_dot cannot found in ubuntu, but python3.8 can work

Closing this issue since there has been no response in 3 years.


proposal: text/template: Move text/template/parse into an internal package

For #10608, the text/template/parse package was changed in a way that was a breaking change in 28c1ad9d35f27b3b57afff4ee78faac746a8ed0a. The justification of this is that text/template/parse would have been an internal package if that had existed before and users shouldn't import it. I agree with that reasoning, but I think if we are going to treat text/template/parse as internal and break its API, we may as well make it internal.
My proposal is that in Go 1.11, we add documentation explaining that text/template/parse will be moved to an internal package and in 1.12, text/template/parse is moved. If we want to be more aggressive, we could move it sooner, since 1.11 is already a breaking change to anyone relying on text/template/parse.

I don't think #10608 was a breaking change, but this issue definitely would be one.

@carlmjohnson I use the text/template parse tree. So... aside from breaking the Go1 compatibility promise which would prevent this, this is a really bad idea.

If we want parse to follow the Go 1 guarantee, someone can go back and fix it, but as is, the guarantee is broken by various token renamings.

I think if someone introduces type VariableNode = AssignNode that would fix the breakage.

The doc comment suggests not to use this package at all. It was created before internal packages were available but if they were, it would have been made one.
So it's in an odd state now. It has had several breaking changes, and there may be more, because it's really not a package you should be using. But people do use it, so we can't just make it internal.

It's not at all clear we can do this in Go 2 either, but we certainly can't do it in Go 1.

I use it for a Go Template to Go transpiler using the same AST, I would use the other packages, however I've found that they can be fairly... slow in comparison.

#25875 also ran into issues with changes to text/template/parse.
What about making a new internal package and using that in text/template and just leaving the 1.10 version of text/template/parse around as a fossil? That seems less likely to break people, if that's the worry.

Look like the same suggestion was made at https://github.com/golang/go/issues/25968#issuecomment-398938510.

Perhaps a template parsing package should be available as a third party package.  It doesn't seem that we want to make that package be part of the (non-internal) standard library, since we've had to change the API in the past and may have to change it in the future.

Actually i would go the opposite way. I have use case for requiring to be able to list a template's complete parse tree because i want to list all fields that are accessed in it.
But the package does obfuscation through the Node interface preventing casting to proper node type (ActionNode etc.) Unless there is some obscure way to to this which i haven't figure out.
I understand why someone might want to create true black box implementation but i don't how this would serve any purpose.
my 2 cents.

Based on the discussion over on #34652, this sounds like it should be rejected.

Based on the discussion above, this is a likely decline.
Leaving open for four weeks for final comments.

No further comments.  Closing.


New ai quickstarts

Merge new AI quickstarts release branch.

Internal previews

📄 File
🔗 Preview link

docs/ai/azure-ai-for-dotnet-developers.md
Develop .NET apps that use Azure AI services

docs/ai/get-started-app-chat-template.md
CustomerIntent: As a .NET developer new to Azure OpenAI, I want deploy and use sample code to interact with app infused with my own business data so that learn from the sample code.

docs/ai/quickstarts/get-started-azure-openai.md
Quickstart - Build an Azure AI chat app with .NET

docs/ai/quickstarts/quickstart-ai-chat-with-data.md
CustomerIntent: As a .NET developer new to Azure OpenAI, I want deploy and use sample code and data to interact to learn from the sample code.

docs/ai/quickstarts/quickstart-azure-openai-tool.md
Extend Azure AI using Tools and execute a local Function with .NET

docs/ai/quickstarts/quickstart-openai-generate-images.md
docs/ai/quickstarts/quickstart-openai-generate-images

docs/ai/quickstarts/quickstart-openai-summarize-text.md
CustomerIntent: As a .NET developer new to Azure OpenAI, I want deploy and use sample code to interact to learn from the sample code to summarize text.

closing and reopening for a fresh build.


Fix Stack Information Leakage

Stack Information Leakage is a complex problem unique to multi-layer stack ensembling.
I first discovered this problem in 2021 during benchmark results analysis where I found that in certain datasets AutoGluon would produce very poor results due to heavy validation score overfitting without an obvious reason. I have since identified the cause and named it "Stack Information Leakage".
Here is a Colab Notebook that describes and demonstrates the issue.
AutoGluon-Tabular currently suffers from stack information leakage, reducing the predictive quality of binary and regression problems, and introducing extreme worst-case scenarios of intensely overfit models.
Despite this, AutoGluon-Tabular remains a very strong predictive model. If this problem were fixed, I suspect it would result in a new state-of-the-art for tabular data.
Ideas:

[ ] Incorporate into the https://github.com/Innixma/autogluon-zeroshot project. AutoGluon-Zeroshot will be able to provide out-of-fold prediction probabilities of many models across a large suite of datasets. These can be used to rapidly test and evaluate stack information leakage mitigation strategies.

Is this issue present if tuning_data is passed to fit together with use_bag_holdout=True?

Is this issue present if tuning_data is passed to fit together with use_bag_holdout=True?

When use_bag_holdout=True, the validation scores of the models, their ensembling, and the selection of the best model will use the holdout data and thus not be overfit. However, the multi-layer stack ensemble will still suffer reduced test (and therefore holdout) performance if the leak occurs, since they are still fed the out-of-fold predictions during training that is unrelated to the holdout data.
use_bag_holdout=True is a method to detect the leak, but it does not fix the leak. Detection is still very useful however, as it avoids the worst-case scenario of picking the model that suffered the most from the leak (and thus has very poor test score)

Sounds promising! Is there a timeline for solving this issue?

Hi @Innixma ,
Interesting problem, I think I have a potential solution for binary classification case that prevents overfitting, it is easy to implement and independent from model type. Basically, we need to round OOF probabilities.  Here is an example, based on your notebook (section 'potential fix'):  https://colab.research.google.com/gist/mglowacki100/fd8e28cb4e19ff8a1b21703de7866472/autogluoncomplexdataleak_v07_leak_fix.ipynb ):
ROUND = 0.1
l1_oof2 = l1_oof.copy()
l1_test2 = l1_test.copy()
l1_oof2[' >50K'] = l1_oof2[' >50K'].apply(lambda x: round(x / ROUND) * ROUND)
l1_test2[' >50K'] = l1_test2[' >50K'].apply(lambda x: round(x / ROUND) * ROUND)

#VERY IMPORTANT, drop or still leak:
l1_oof2 = l1_oof2.drop(columns=[' <=50K'])
l1_test2 = l1_test2.drop(columns=[' <=50K'])

predictor_gbm_leak2 = TabularPredictor(eval_metric='roc_auc', label=label, verbosity=0).fit(
    train_data=l1_oof2,
    hyperparameters={
        'GBM':
            [
                {},
            ]
    },
    fit_weighted_ensemble=False,
)

leaderboard_gbm_leak2 = predictor_gbm_leak2.leaderboard(l1_test2, silent=True)
leaderboard_gbm_leak2

model	  score_test	score_val	pred_time_test	pred_time_val	fit_time	pred_time_test_marginal	pred_time_val_marginal	fit_time_marginal	stack_level	can_infer	fit_order
LightGBM  0.693445	0.704503	0.030098	0.003197	0.376164	0.030098	0.003197	0.376164	1	True	1

Now score_test 0.69... is very close to score_val 0.70..
A few remarks:

I didn't test it on other datasets, I don't have a full picture of the problem
Method works not only for lightGBM but for other models too
Not sure about model final performance, maybe this workaround prevents overfitting, but introduces underfitting
Rounding 'level' could be finetuned (see snippet below), btw I've a clue that there is some rough relation between rounding level and number of rows in training dataset
If this method is promising and works, we could check an additional variant, namely instead of rounding, we could use bucketizing (it is a little more complicated, instead of just rounding level, we need to provide 'bucket function' from train and apply it on test data)
For regression potential solution: predictions could be scaled to [0,1] range (maybe additional winsorization), so now fix from classification could be used, subseqeuntly scaling back

Here is snippet to calculate performance depending on model type and rounding level:
ROUNDS = [0.01, 0.02, 0.05, 0.075, 0.1, 0.125, 0.15, 0.20, 0.25]
MODELS = ['GBM', 'CAT', 'XGB', 'KNN', 'RF', 'XT']
for mdl in MODELS:
  print('#########################################')
  print('#########################################')
  print(f'model type: {mdl}')
  for r in ROUNDS:
    l1_oof2 = l1_oof.copy()
    l1_test2 = l1_test.copy()
    l1_oof2[' >50K'] = l1_oof2[' >50K'].apply(lambda x: round(x / r) * r)
    l1_test2[' >50K'] = l1_test2[' >50K'].apply(lambda x: round(x / r) * r)
    l1_oof2 = l1_oof2.drop(columns=[' <=50K'])
    l1_test2 = l1_test2.drop(columns=[' <=50K'])

    predictor_leak2 = TabularPredictor(eval_metric='roc_auc', label=label, verbosity=0).fit(
      train_data=l1_oof2,
      hyperparameters={
          mdl:
              [
                  {},
              ]
      },
      fit_weighted_ensemble=False,
    )
    leaderboard_leak2 = predictor_leak2.leaderboard(l1_test2, silent=True)
    print('#########################################')
    print(f"Rounding r: {r}")
    print(leaderboard_leak2[['model', 'score_test', 'score_val']])

#########################################
#########################################
model type: GBM
#########################################
Rounding r: 0.01
      model  score_test  score_val
0  LightGBM    0.665311   0.835742
#########################################
Rounding r: 0.02
      model  score_test  score_val
0  LightGBM    0.693943   0.775804
#########################################
Rounding r: 0.05
      model  score_test  score_val
0  LightGBM    0.686461   0.700132
#########################################
Rounding r: 0.075
      model  score_test  score_val
0  LightGBM     0.70193   0.681797
#########################################
Rounding r: 0.1
      model  score_test  score_val
0  LightGBM    0.693445   0.704503
#########################################
Rounding r: 0.125
      model  score_test  score_val
0  LightGBM    0.698761   0.696609
#########################################
Rounding r: 0.15
      model  score_test  score_val
0  LightGBM    0.695758    0.68394
#########################################
Rounding r: 0.2
      model  score_test  score_val
0  LightGBM    0.691416   0.703941
#########################################
Rounding r: 0.25
      model  score_test  score_val
0  LightGBM    0.688187   0.676481
#########################################
#########################################
model type: CAT
#########################################
Rounding r: 0.01
      model  score_test  score_val
0  CatBoost     0.66719   0.833885
#########################################
Rounding r: 0.02
      model  score_test  score_val
0  CatBoost    0.692683   0.770966
#########################################
Rounding r: 0.05
      model  score_test  score_val
0  CatBoost    0.689956   0.697564
#########################################
Rounding r: 0.075
      model  score_test  score_val
0  CatBoost    0.702874   0.690168
#########################################
Rounding r: 0.1
      model  score_test  score_val
0  CatBoost    0.693445   0.704503
#########################################
Rounding r: 0.125
      model  score_test  score_val
0  CatBoost    0.699377   0.704185
#########################################
Rounding r: 0.15
      model  score_test  score_val
0  CatBoost    0.695758    0.68394
#########################################
Rounding r: 0.2
      model  score_test  score_val
0  CatBoost    0.691416   0.703941
#########################################
Rounding r: 0.25
      model  score_test  score_val
0  CatBoost    0.688187   0.676481
#########################################
#########################################
model type: XGB
#########################################
Rounding r: 0.01
     model  score_test  score_val
0  XGBoost    0.671387    0.83466
#########################################
Rounding r: 0.02
     model  score_test  score_val
0  XGBoost    0.692673   0.770924
#########################################
Rounding r: 0.05
     model  score_test  score_val
0  XGBoost    0.686461   0.700132
#########################################
Rounding r: 0.075
     model  score_test  score_val
0  XGBoost    0.702874   0.690168
#########################################
Rounding r: 0.1
     model  score_test  score_val
0  XGBoost    0.693445   0.704503
#########################################
Rounding r: 0.125
     model  score_test  score_val
0  XGBoost    0.699364   0.704057
#########################################
Rounding r: 0.15
     model  score_test  score_val
0  XGBoost    0.695758    0.68394
#########################################
Rounding r: 0.2
     model  score_test  score_val
0  XGBoost    0.691416   0.703941
#########################################
Rounding r: 0.25
     model  score_test  score_val
0  XGBoost    0.688187   0.676481
#########################################
#########################################
model type: KNN
#########################################
Rounding r: 0.01
        model  score_test  score_val
0  KNeighbors    0.627423   0.793969
#########################################
Rounding r: 0.02
        model  score_test  score_val
0  KNeighbors    0.640205   0.696662
#########################################
Rounding r: 0.05
        model  score_test  score_val
0  KNeighbors    0.651572   0.664258
#########################################
Rounding r: 0.075
        model  score_test  score_val
0  KNeighbors    0.623913   0.605647
#########################################
Rounding r: 0.1
        model  score_test  score_val
0  KNeighbors    0.644587   0.619695
#########################################
Rounding r: 0.125
        model  score_test  score_val
0  KNeighbors    0.657459   0.678646
#########################################
Rounding r: 0.15
        model  score_test  score_val
0  KNeighbors    0.682628   0.669765
#########################################
Rounding r: 0.2
        model  score_test  score_val
0  KNeighbors    0.644182    0.64743
#########################################
Rounding r: 0.25
        model  score_test  score_val
0  KNeighbors    0.688187   0.676481
#########################################
#########################################
model type: RF
#########################################
Rounding r: 0.01
          model  score_test  score_val
0  RandomForest    0.663497   0.835031
#########################################
Rounding r: 0.02
          model  score_test  score_val
0  RandomForest    0.692433   0.770966
#########################################
Rounding r: 0.05
          model  score_test  score_val
0  RandomForest    0.687053   0.700132
#########################################
Rounding r: 0.075
          model  score_test  score_val
0  RandomForest     0.70288   0.690296
#########################################
Rounding r: 0.1
          model  score_test  score_val
0  RandomForest    0.693445   0.704503
#########################################
Rounding r: 0.125
          model  score_test  score_val
0  RandomForest    0.699377   0.704185
#########################################
Rounding r: 0.15
          model  score_test  score_val
0  RandomForest    0.695758    0.68394
#########################################
Rounding r: 0.2
          model  score_test  score_val
0  RandomForest    0.691416   0.703941
#########################################
Rounding r: 0.25
          model  score_test  score_val
0  RandomForest    0.688187   0.676481
#########################################
#########################################
model type: XT
#########################################
Rounding r: 0.01
        model  score_test  score_val
0  ExtraTrees    0.663518    0.83501
#########################################
Rounding r: 0.02
        model  score_test  score_val
0  ExtraTrees    0.692433   0.770966
#########################################
Rounding r: 0.05
        model  score_test  score_val
0  ExtraTrees     0.68672   0.700132
#########################################
Rounding r: 0.075
        model  score_test  score_val
0  ExtraTrees     0.70288   0.690296
#########################################
Rounding r: 0.1
        model  score_test  score_val
0  ExtraTrees    0.693445   0.704503
#########################################
Rounding r: 0.125
        model  score_test  score_val
0  ExtraTrees    0.699377   0.704185
#########################################
Rounding r: 0.15
        model  score_test  score_val
0  ExtraTrees    0.695758    0.68394
#########################################
Rounding r: 0.2
        model  score_test  score_val
0  ExtraTrees    0.691416   0.703941
#########################################
Rounding r: 0.25
        model  score_test  score_val
0  ExtraTrees    0.688187   0.676481

What do you think about it?

Sounds promising! Is there a timeline for solving this issue?

The plan is to address this issue as one of the major features of AutoGluon v1.0 release, which is planned for ~November/December 2023, and we hope to have a working fix available in pre-release in August. Currently we are working on AutoGluon-Zeroshot which is a large-scale ensemble simulation suite. Once it is complete, we can simulate stack information leakage fixes across 1000's of tasks to see if we can find a generalizable solution that leads to improved performance.

What do you think about it?

Hey @mglowacki100, this is great! Indeed this is one of the ideas I thought of early on in my investigation last year, although I did not specifically implement rounding, so it is cool that you implemented it, as I think it should work pretty well. You can see some of the other methods I implemented here: https://github.com/Innixma/stack-info-leak . In general, AddNoise performed the best (of the ones I implemented), and AddRound might outperform AddNoise, but I haven't tested.
I think the trickiest part is finding the right value of ROUNDS or NOISE based on the task.  We can find a good value by doing nested cross-validation, but that would be expensive, so the question is if we can find a good value without increasing training time significantly.

Hi @Innixma,
I'll check how AddRound compares to AddNoise.
Btw. for a moment I thought that reason of leak is:
l1_oof[' >50K'].value_counts(dropna=False)
0.000000    520
0.451982      2
...

but, removal of "0 rows" from train, or from train and test doesn't remove leak. I've also tried to inject small noise but it doesn't help. Also, I think this quirk specific for adult data set, but if this quirk is general, then we'd have heurestics for bucketing size, namely: len(l1_oof[' >50K']) 5000 need to be divided by 520 (l1_oof[' >50K'].value_counts(dropna=False)[0]) and rounded down which gives 9, next bucketing:
l1_oof2[' >50K'] = pd.qcut(l1_oof2[' >50K'], 9)
l1_test2[' >50K'] = pd.cut(l1_test2[' >50K'], bins=l1_oof2[' >50K'].cat.categories)

Regarding value of ROUNDS I think it could also be impacted by roc_auc; higher roc_auc less rounding. Maybe there is some rule of thumb for it.

This is an example of fixing the problem of seeing oof data due to early stopping etc. Code quality is low because it is my personal code.
The problem of seeing oof data in ensemble optimization has not been resolved.
https://github.com/autogluon/autogluon/compare/master...richmanbtc:autogluon:feature/strict_val

Hi @Innixma,
I've recently look at autogluon 0.8.2 and tested with this old kaggle competition (regression, metric r_2, small set): https://www.kaggle.com/competitions/mercedes-benz-greener-manufacturing/overview
Unfortunately, autogluon is overoptimistic regarding internal cv-validation and fails on public and private leaderboards (on private I've never get more than 0.545, I've checked combinations of best_quality, experimental_zeroshot_hpo_hybrid on raw data, data with with additional features like pca, selected features - generic stuff).
I think leak still occurs as it is possible to get decent result (silver medal) with automl on raw data: lightgbm tuned with optuna (https://www.kaggle.com/code/mt77pp/mljar-automl-mercedes-optimize-r2/notebook).
Another interesting but extremally small dataset for regression is https://www.kaggle.com/competitions/restaurant-revenue-prediction/overview here is also overfit, but I didn't look closer into it.
On bright side for competition: https://www.kaggle.com/competitions/bioresponse/overview on 'experimental...' gives top7 result, but if you change sample_weight to balance_weight  then top4. Note that, this is classification problem

@mglowacki100 Wow!! Thanks for the analysis and dataset references. We have an intern joining for the fall next week who will be working on investigating and implementing fixes for this problem, and your comment will be a great resource!
@richmanbtc Thanks for the code examples and details! Yes, in general this problem becomes more severe as num_bag_sets increases, with more overfitting for each additional set. We will be taking a deeper look at your code + comment in a few weeks as part of the intern's ramp-up and investigation to the problem.

Heyho everyone,
I would like to provide a short update on our effort regarding this issue. I am the intern that @Innixma mentioned. We are still working on fixing stack information leakage. In the past weeks, we tested several fixes and formalized our current understanding of why stack information leakage occurs.
To clarify, we define stack information leakage as a failure case of traditional multi-layer stacking (as used in AutoGluon). In this failure case, we obtain very good validation scores for the stacking models (denoted as L2 models) but their test scores are worse than non-stacking models (denoted as L1 models). The following provides an overview of stack information leakage for a (reproducible) example when stacking Random Forest (RF) models:

In a first study, we observed that the leak occurs in ~25% (63 out of 244) of all datasets across all problem types (binary, multi-class, and regression). Moreover, we were able to formalize several factors that influence whether the leak occurs. To pick up one such factor that was already mentioned in this issue, we found that more resampling during the validation procedure (e.g., more folds during cross-validation or more repetitions / higher num_bag_sets) makes it more likely that the leak occurs.
Besides several other ideas, we tested all fixes proposed above and mentioned in the Colab Notebook. We also explored some solutions from Kaggle users and machine learning partitioners that seem to have encountered stack information leakage before. We are currently exploring alternative approaches as none of them were promising enough or suitable for integration into AutoGluon. Since we did not have the time to test all solutions and their potential variations in detail, I would not rule them out entirely.
We implemented one promising approach with PR https://github.com/autogluon/autogluon/pull/3494 and tested it more extensively. We found that it can avoid stack info leakage somewhat successfully (e.g., see @mglowacki100’s similar findings). Yet, we also found that when the leak does not occur, employing this fix weakness AutoGluon’s performance; making it not suitable for integration into AutoGluon. Hence, we also explored in how far we can detect if the leak occurs. Our current best-performing fix is based on such an approach.
As a fix for stack information leakage is planed to be part of AutoGluon 1.0, we are currently focusing on finding a working solution. Therefore, we plan to share the findings of our research in more detail after the release of AutoGluon 1.0.

Hi @LennartPurucker and @Innixma,
I've recently been pondering this problem and have come up with a simple idea that may be worth trying if it hasn't been explored already. Specifically, I suggest looking at the feature importance at the L2 level. My hypothesis is that "L1 predictions" will be at the top, dominating other features for datasets that exhibit leakage. I believe this can be achieved by using the feature_importances_ attribute with a direct call to the scikit-learn random forest interface.
If this hypothesis holds true, it could provide ways to address the leaking issue, such as introducing enough noise to "L1 predictions" to align their importance with non-leaking datasets. Even if there is no such relationship, a negative result would still be valuable, especially if tested across a large number of datasets. Unfortunately, I lack access to the necessary infrastructure to conduct this testing.
Additionally, a couple of days ago, I came across a related article: Which of Your Features Are Overfitting. It involves Shapley values and may require more computational resources, but the method appears intriguing

Hi @mglowacki100,
Thank you for your continued help and the suggestion!
We have not explicitly analyzed the feature importance at L2 across many datasets so far. We shortly looked at the relation between the quality and importance of L1 features and the leak; but I have no tangible takeaways from this. I think feature importance could be a promising approach to either remove the leak or, more generally, spot the leak and then, for example, remove likely harmful L1 predictions from the L2 features.
One potential downside of relying on high feature importance is that L2 models could justifiably put a lot of importance on the L1 predictions even if a dataset does not leak. As the L1 predictions represent (in theory) high quality features and are (potentially) highly correlated to the target variable, an L2 model might focus mostly on L1 predictions too. So, I think the hard part of this would be to determine or meta-learn when the L2 model justifiably has a large feature importance for the L1 predictions and when not.
I agree that something like an approach based on Shapley values or permutation importance would likely be better than the feature importance of Random Forest. As AutoGluon already supports permutation importance, I went ahead and computed the feature importance of L2 features (on holdout data) for a selection of binary classification datasets using this (for a small budget with AutoGluon).
Thereby, I found that by dropping the top 5 most important L1 prediction features and then re-training, the leak does generally not go away. Instead it got worse in several cases.
However, I found that by dropping L1 prediction features with a low permutation importance (i.e., features that are considered harmful or even useless for training) the leak goes away for some datasets that leak (e.g., the datasets meta and blood-transfusion-service-center)! For datasets that do not leak the results are not conclusive so far and I am planning more in-depth benchmarks for this.
The results for my small benchmark are definitely promising enough to spend more time on this. I will report back once I have more conclusive results and a solution suitable to integrate into AutoGluon.
Thank you again for bringing this to our attention!

Hi @LennartPurucker,
I'd like to discuss feature importance further. I believe it's crucial to assess feature importance for both the training dataset and the holdout set. I presume that a leak occurs when a feature (L1 prediction) is important in the training set but not important in the holdout set. In cases where importance is low for both the training and holdout sets, removing the feature shouldn't significantly impact the results. When both importances are high, there should be no leakage. If importance is low on the training set but high on the holdout set, it's a peculiar case, and it suggests the presence of other issues (other features also leaking?) ?
Additionally, upon revisiting @Innixma's notebook, I've come across another fix proposal. It involves controlling the max_depth parameter (the current default value is None, which allows for very deep trees). This approach differs from other fixes, such as monotonic constraints and rounding L1 predictions, which attempt to address this indirectly.
For reference, here's a Colab notebook with experiments conducted on the adult dataset: Colab Experiment. The essence of the findings is that, as max_depth increases, score_val also increases. The score_test reaches its maximum at max_depth 2 and then starts to degrade significantly after max_depth 10. Please bear in mind that this observation is based on just one dataset.
predictor2 = TabularPredictor(label=label, verbosity=0, eval_metric='roc_auc').fit(
    train_data=train_data,
    num_bag_folds=8,
    num_stack_levels=1,
    hyperparameters={'RF': {'max_depth': 2}},
    fit_weighted_ensemble=False,
)

# Get model leaderboard
leaderboard2 = predictor2.leaderboard(test_data, silent=True)
leaderboard2

model
score_test
score_val
pred_time_test
pred_time_val
fit_time
pred_time_test_marginal
pred_time_val_marginal
fit_time_marginal
stack_level
can_infer
fit_order

RandomForest_BAG_L1
0.712969
0.694710
0.170580
0.139401
0.563640
0.170580
0.139401
0.563640
1
True
1

RandomForest_BAG_L2
0.711966
0.722436
0.324468
0.287209
1.161739
0.153888
0.147808
0.598099
2
True
2

max depth 10:

model
score_test
score_val
pred_time_test
pred_time_val
fit_time
pred_time_test_marginal
pred_time_val_marginal
fit_time_marginal
stack_level
can_infer
fit_order

RandomForest_BAG_L1
0.707334
0.703999
0.324493
0.198804
0.609666
0.324493
0.198804
0.609666
1
True
1

RandomForest_BAG_L2
0.701778
0.962194
0.680128
0.440659
1.873010
0.355635
0.241855
1.263344
2
True
2

max_depth 25:

model
score_test
score_val
pred_time_test
pred_time_val
fit_time
pred_time_test_marginal
pred_time_val_marginal
fit_time_marginal
stack_level
can_infer
fit_order

RandomForest_BAG_L1
0.706670
0.703578
0.381530
0.257632
0.980403
0.381530
0.257632
0.980403
1
True
1

RandomForest_BAG_L2
0.649768
0.953570
0.903723
0.600602
2.538800
0.522193
0.342970
1.558397
2
True
2

It's important to note that I've set max_depth for all random forest models. It might be worth exploring whether setting max_depth on L2 level models exclusively would be sufficient.
If controlling max_depth proves effective for other datasets: It would be beneficial if a sensible default for all datasets could be established (although this might be a low probability). Hyper-tuning could be considered. Training decision tree solely on "L1 predictions" should be fast and provide a good max_depth value without interference from other features. This max_depth could then be used for the entire random forest and other tree-based methods.
I appreciate your attention to this matter and eagerly await your results!
Best regards,
Marek

Heyho @mglowacki100,
again thank you vey much for sharing your insights and being part of this ongoing discussion!

I believe it's crucial to assess feature importance for both the training dataset and the holdout set.

We came to the very same conclusion and implemented this yesterday. Therefore, we had to get the "out-of-fold feature importance" to work correctly such that we have an unbiased training importance for cross-validation. Moreover, we defined your last suggestion, importance is low on the training set but high on the holdout set, also as a sign of leakage. We then filtered alls L1 prediction features based on low importance on the holdout set and I tested a simple heuristics to remove features that have a large gap between holdout and training.
After setting up this implementation with AutoGluon to start testing the idea, I found that it works but is highly influenced by the number of repeats done by AutoGluon. For the dataset blood-transfusion-service-center (which is a really good test dataset for the leak), it works as long as we do not go above 4 repeats without the heuristic. Afterwards, we need the heuristic that filters based on the gap.
Next, I am trying to figure out why this is the case and whether we can measure importance per repeat or similar thoughts. Moreover, I need more time to think about a good heuristic.

controlling the max_depth parameter

We explored the influence of max_depth among other regularization hyperparameters of L1 models early on in our study and found similar results. In detail, in your example a larger max depth creates more unique OOF prediction values (and a larger tree with more leaves) which the L2 models can exploit more, i.e., the L2 models have more opportunities to associate the leaking target label to the OOF predictions. Interestingly, it behaves (with respect to the OOF predictions) similar to increasing the number of bags, i.e., increasing the number of folds or repetitions.
However, we also found that a default value was not consistent. Hence, we would require the optimization/tuning of such regularization hyperparameters as we drastically hurts the performance of L1 or L2 models otherwise. We could use a proxy model for tuning, as you described,  but the proxy would need to be representative - which is hard to do in itself.
Finally, if I recall correctly, we also found that sometimes no value for max_depth (or other regularization hyperparameters) was able to remove the leak. In your provided data, you can see that we already (very minimally) leak for max_depth=2. My working theory is that overfitting of L1 models amplifies the leak but is not the reason for the leak. Hence, even if we remove "all overfitting" from L1 models, we might still leak.
Lastly, regarding regularizing L2 models (e.g., setting max_depth at L2): this can work as long as the regularization is powerful enough and we can tune it correctly. For neural networks the regularization is so powerful that we can more easily avoid the leak but for tree-base models it is a bit harder (maybe too hard) due their intrinsic bias.

I hope I was able provide you with some more results and insights Marek.
I will explore the feature importance approach further and report back.
Best regards,
Lennart

Hi @LennartPurucker,
I've come across an article that might interest you: United We Stand: Using Epoch-wise Agreement of Ensembles to Combat Overfit. It discusses neural networks, but I believe the proposed methodology could be extended to Gradient Boosting Machines (GBMs) since they also have "epochs".
Regarding tree methods, I agree that their intrinsic bias could be hard to overcame (greedy approach with focusing on the single feature at time). Maybe some modification of decision split criteria could help, but such customization will be hard to productionize in the short term.
Regarding the "repeats" problem, in addition to "filtering within repeats," I believe a valuable addition would be "filtering across repeats." Specifically, a feature may perform well with respect to both the training and holdout datasets across all repeats, but if its importance varies significantly, such as being in the first decile in the one of the repeatr and the last decile in the other it should raise suspicion.
Thank you for providing detailed explanations!

Hi,
First of all: congratz for the 1.0 release!
Out of curiosity: Did you investigate nested CV (sometimes called "careful" CV aka CCV at kaggle) as potential solution to prevent this type of leakage and if so, what were your findings?
Cheers,
Faron

Thanks for the quick and detailed response @LennartPurucker!


implement net emissions token network smart contracts

See https://wiki.hyperledger.org/display/CASIG/Net+Emissions+Tokens+Network
We'll be implementing these smart contracts based on the code from the eThaler project.

Done.  See https://www.youtube.com/watch?v=C-cUjQLDGJw


Feature/8 add sample standard files for open source projects

close #8

Codecov Report
Patch coverage has no change and project coverage change: +87.37 :tada:

Comparison is base (ef864a4) 0.00% compared to head (673cf4d) 87.37%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff            @@
##           main      #10       +/-   ##
=========================================
+ Coverage      0   87.37%   +87.37%
=========================================
  Files         0        6        +6
  Lines         0      198      +198
=========================================
+ Hits          0      173      +173
- Misses        0       25       +25

Flag
Coverage Δ

Linux
87.37% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 6 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


reducing png image sizes using pngquant

Thanks for the pull request @danielwhatmuff!

:x: Missing CLA.

Please send in a Contributor License Agreement (CLA) and comment back here to let us know to check this!

:grey_question: CHANGES.md was not updated.

If this change updates the public API in any way, please add a bullet point to CHANGES.md.

:grey_question: Unit tests were not updated.

Make sure you've updated tests to reflect your changes, added tests for any new code, and ran the code coverage tool.

Reviewers, don't forget to make sure that:

[ ] Cesium Viewer works.
[ ] Works in 2D/CV.
[ ] Works (or fails gracefully) in IE11.

Thanks for the awesome work @danielwhatmuff! Your contribution is about to launch to millions of users with the next release. :rocket:
Do you mind if we tweet about it? CC @OmarShehata @Slchow.

Reopened after bringing our fork up to date with upstream master here #8169. Tweets always welcome @OmarShehata @slchow  - our handle is @watchkeeper.


[Feature]: AC input Relay open/close feature

Contact Details
<EMAIL_ADDRESS>Description
Allow a toggle switch so that DESS can control the AC input relay.
I.e to prevent grid pulls at peak rates or whilst there is available SOC%
But, turn on grid is overload situation occurs (grid support).
Adding this feature will stop all minor grid pulls during peak hours, and save users money.
Suggested Solution
Adding this feature will stop all minor grid pulls during peak hours, and save users money.
Alternatives
Currently running node red to open relay during peak hours, with an overload watch.
Additional Context
No response

Any general advise how to control AC-INPUT relay?
Is it possible to control via MQTT ?


chat templates?

thanks for building this. looks nice on quick first try. but for some models, you might want to check the prompt template / stop parameters / parse yaml template files like eg ooba etc do, because cf. eg:

(sorry for quick lazy issue note, am short of time atm)

Thank you @CrispStrobe , can you specific which model is this?

sorry should have been in the screenshot, think this was NousHermes-2-Mistral-7B-DPO (4bit)

let me take a look really quick

I will look into it. Thank you for reporting


Add ba_ref field to Location model

I had a look at the data with Dan today and I was wondering what is preventing me from being able to start importing Cambridge data. We found out that ba_ref is not available in the Location model. Even if not a mandatory field, we need to import it when available, otherwise we can't lookup the VOA data.

Seems fine for where it exists, but we need to handle when it doesn't exist for some users.  We said UPRN was mandatory for new import files, but need to check that we said BA_REF is too.


Feat/this is for you bill gates

closes #160, #154

@mihaiblaga89
testing on win machine I get this error:

? Do you want to install android SDK? (Y/n) y                                                         ? Do you want to install android SDK? Yes                                                             ℹ  $ rnv run -p android - NOTE: Platform win32                                                        ℹ  $ rnv run -p android - NOTE: Checking wget is installed                                            ℹ  $ rnv run -p android - NOTE: Installing Scoop...                                                   & : The term 'C:\Users\beher\AppData\Roaming\npm\node_modules\rnv/scripts/installPackageManager.ps1'  is not recognized                                                                                as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name,  or if a path was                                                                                included, verify that the path is correct and try again.                                              At line:1 char:3                                                                                + & "C:\Users\beher\AppData\Roaming\npm\node_modules\rnv/scripts/instal ...                           +   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                   + CategoryInfo          : ObjectNotFound: (C:\Users\beher\...kageManager.ps1:String) [], CommandN otFoundException                                                                                + FullyQualifiedErrorId : CommandNotFoundException                                                                                exec: & : The term 'C:/Users/beher/AppData/Roaming/npm/node_modules/rnv/scripts/installPackageManager .ps1' is not recognized                                                                               as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name,  or if a path was                                                                                included, verify that the path is correct and try again.                                              At line:1 char:3                                                                                + & "C:/Users/beher/AppData/Roaming/npm/node_modules/rnv/scripts/instal ...                           +   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                   + CategoryInfo          : ObjectNotFound: (C:/Users/beher/...kageManager.ps1:String) [], CommandN otFoundException                                                                                + FullyQualifiedErrorId : CommandNotFoundException                                                                                'C:\Users\beher\AppData\Roaming\npm\node_modules\rnv/scripts/resetPath.vbs' is not recognized as an i nternal or external command,                                                                          operable program or batch file.                                                                       exec: 'C:/Users/beher/AppData/Roaming/npm/node_modules/rnv/scripts/resetPath.vbs' is not recognized a s an internal or external command,                                                                    operable program or batch file.                                                                                'C:\Users\beher\AppData\Local\Temp/resetvars.bat' is not recognized as an internal or external comman d,                                                                                operable program or batch file.                                                                       exec: 'C:/Users/beher/AppData/Local/Temp/resetvars.bat' is not recognized as an internal or external  command,                                                                                operable program or batch file.                                                                                ℹ  $ rnv run -p android - NOTE: Looks like you don't have wget or curl installed. We'll install wget  for you                                                                                The system cannot find the path specified.                                                            exec: The system cannot find the path specified.                                                                                'C:\Users\beher\AppData\Roaming\npm\node_modules\rnv/scripts/resetPath.vbs' is not recognized as an i nternal or external command,                                                                          operable program or batch file.                                                                       exec: 'C:/Users/beher/AppData/Roaming/npm/node_modules/rnv/scripts/resetPath.vbs' is not recognized a s an internal or external command,                                                                    operable program or batch file.                                                                                'C:\Users\beher\AppData\Local\Temp/resetvars.bat' is not recognized as an internal or external comman d,                                                                                operable program or batch file.                                                                       exec: 'C:/Users/beher/AppData/Local/Temp/resetvars.bat' is not recognized as an internal or external  command,                                                                                operable program or batch file.                                                                                ℹ  $ rnv run -p android - NOTE: Looks like you don't have java installed. We'll install it for you    The system cannot find the path specified.                                                            exec: The system cannot find the path specified.                                                                                'C:\Users\beher\AppData\Roaming\npm\node_modules\rnv/scripts/resetPath.vbs' is not recognized as an i nternal or external command,                                                                          operable program or batch file.                                                                       exec: 'C:/Users/beher/AppData/Roaming/npm/node_modules/rnv/scripts/resetPath.vbs' is not recognized a s an internal or external command,                                                                    operable program or batch file.                                                                                'C:\Users\beher\AppData\Local\Temp/resetvars.bat' is not recognized as an internal or external comman d,                                                                                operable program or batch file.                                                                       exec: 'C:/Users/beher/AppData/Local/Temp/resetvars.bat' is not recognized as an internal or external  command,                                                                                operable program or batch file.                                                                                The system cannot find the path specified.                                                            exec: The system cannot find the path specified.                                                                                The system cannot find the path specified.                                                            exec: The system cannot find the path specified.                                                                                'C:\Users\beher\AppData\Roaming\npm\node_modules\rnv/scripts/resetPath.vbs' is not recognized as an i nternal or external command,                                                                          operable program or batch file.                                                                       exec: 'C:/Users/beher/AppData/Roaming/npm/node_modules/rnv/scripts/resetPath.vbs' is not recognized a s an internal or external command,                                                                    operable program or batch file.                                                                                'C:\Users\beher\AppData\Local\Temp/resetvars.bat' is not recognized as an internal or external comman d,                                                                                operable program or batch file.                                                                       exec: 'C:/Users/beher/AppData/Local/Temp/resetvars.bat' is not recognized as an internal or external  command,                                                                                operable program or batch file.                                                                                rm: no such file or directory: C:/Users/beher/sdk-tools-windows-4333796.zip                           ℹ  $ rnv run -p android - NOTE: Downloading android SDK...                                            'wget' is not recognized as an internal or external command,                                          operable program or batch file.                                                                       exec: 'wget' is not recognized as an internal or external command,                                    operable program or batch file.                                                                                unzip:  cannot find or open C:\Users\beher/sdk-tools-windows-4333796.zip, C:\Users\beher/sdk-tools-wi ndows-4333796.zip.zip or C:\Users\beher/sdk-tools-windows-4333796.zip.ZIP.                            exec: unzip:  cannot find or open C:/Users/beher/sdk-tools-windows-4333796.zip, C:/Users/beher/sdk-to ols-windows-4333796.zip.zip or C:/Users/beher/sdk-tools-windows-4333796.zip.ZIP.                                                                                ┌──────────────────────────────────────────────────────────────────────────────┐                      │  🚀  SUMMARY \| 8/10/2019, 11:59:15 AM                                                            │  │  $ rnv run -p android                                                                            │  ├──────────────────────────────────────────────────────────────────────────────┤                      │  Project Name: renative-wrapper-app                                                                         │                                                                                │  Project Version: 0.24.0-alpha1                                                                             │                                                                                │  App Config: helloWorld                                                                                │                                                                                │  Supported Platfroms: ios, android, androidtv, androidwear, web,                                                      │                                                                                │  tizen, tvos, webos, macos, windows, tizenwatch, tizenmobile,                                               │                                                                                │  kaios, firefoxos, firefoxtv                                                                                │                                                                                │  Master Template: renative-template-hello-world                                                             │                                                                                │  Env Info: win32 \| x64 \| node v10.15.0 \| rnv v0.24.0-alpha1                                                 │                                                                                │  Platform: android                                                                                │                                                                                │  Executed Time: 0h:0m:6s:177ms                                                                              │                                                                                │                                                                                │            │ ⚠  $ rnv run -p android - WARNING: You're in ReNative wrapper mode. plugin renative will stay as lo cal dep!                                                                                │ 🛑  $ rnv run -p android - ERRROR! android requires SDK to be installed. check your C:\Users\beher\ .rnv\config.json file if you SDK path is correct                                                      │ 🛑  $ rnv run -p android - ERRROR! process errored with Error: spawn C:\Users\beher/Android/tools/b in/sdkmanager.bat ENOENT                                                                              Error: process errored with Error: spawn C:\Users\beher/Android/tools/bin/sdkmanager.bat ENOENT           at ChildProcess. (C:\DEV\Code\renative\packages\rnv\dist\systemTools\exec.js:86:8)         at ChildProcess.emit (events.js:182:13)                                                               at Process.ChildProcess._handle.onexit (internal/child_process.js:238:12)                             at onErrorNT (internal/child_process.js:407:16)                                                       at process._tickCallback (internal/process/next_tick.js:63:19)                                    │                                                                                │            └──────────────────────────────────────────────────────────────────────────────┘                      Killing child process C:\Users\beher/Android/tools/bin/sdkmanager.bat --licenses
--


build!: update to latest version of gts and typescript

I used this PR as reference for the rest-params usage: https://github.com/googleapis/nodejs-bigtable/pull/682/files