Add ability to set the request headers

Hi, was wondering to add the ability to set request headers:
https://github.com/itchio/httpkit/blob/50f60be27c88c7d8a3cdc01ea2029d465a830ceb/htfs/conn.go#L115
For example the external site expects some headers and rejects all requests without it.
The actual use-case is setting a cookie.

func (c *conn) tryConnect(offset int64) error {
	hf := c.file

	req, err := http.NewRequest("GET", hf.currentURL, nil)
	if err != nil {
		return err
	}

        key := os.Getenv("EXAMPLE_API_KEY") #change
        req.Header.Set("Cookie", "owidev=" + key) #change

	byteRange := fmt.Sprintf("bytes=%d-", offset)
	req.Header.Set("Range", byteRange)

	res, err := hf.client.Do(req)
	if err != nil {
		return err
	}

This is what I got.

The way I'd go about it would be to add a
ModifyRequest func (req *http.Request)

in htfs.Settings:
https://github.com/itchio/httpkit/blob/a507e904f82f9fc90c40eec62bb64eb453cfbc18/htfs/file.go#L114-L122
Then File would be responsible to pass it down to conn, which would call it in tryConnect (if it's non-nil).
Do you want to give it a shot?

This seems better. Can push it down the chain more, but wanted review.
diff --git a/vendor/github.com/itchio/httpkit/htfs/conn.go b/vendor/github.com/itchio/httpkit/htfs/conn.go
index 9cfbed7..1ae66d1 100644
--- a/vendor/github.com/itchio/httpkit/htfs/conn.go
+++ b/vendor/github.com/itchio/httpkit/htfs/conn.go
@@ -24,6 +24,7 @@ type conn struct {
 	currentURL string

 	header        http.Header
+	modifyRequest func (req *http.Request)
 	requestURL    *url.URL
 	statusCode    int
 	contentLength int64
@@ -120,6 +121,10 @@ func (c *conn) tryConnect(offset int64) error {
 		return err
 	}

+	if hf.modifyRequest != nil {
+		hf.modifyRequest(req)
+	}
+
 	byteRange := fmt.Sprintf("bytes=%d-", offset)
 	req.Header.Set("Range", byteRange)

diff --git a/vendor/github.com/itchio/httpkit/htfs/file.go b/vendor/github.com/itchio/httpkit/htfs/file.go
index caa7fb6..e105d0d 100644
--- a/vendor/github.com/itchio/httpkit/htfs/file.go
+++ b/vendor/github.com/itchio/httpkit/htfs/file.go
@@ -72,6 +72,7 @@ type File struct {
 	getURL        GetURLFunc
 	needsRenewal  NeedsRenewalFunc
 	client        *http.Client
+	modifyRequest func (req *http.Request)
 	retrySettings *retrycontext.Settings

 	Log      LogFunc
@@ -114,6 +115,7 @@ var _ io.Closer = (*File)(nil)
 // Settings allows passing additional settings to an File
 type Settings struct {
 	Client             *http.Client
+	ModifyRequest      func (req *http.Request)
 	RetrySettings      *retrycontext.Settings
 	Log                LogFunc
 	LogLevel           int
@@ -139,6 +141,7 @@ func Open(getURL GetURLFunc, needsRenewal NeedsRenewalFunc, settings *Settings)
 		retrySettings: &retryCtx.Settings,
 		needsRenewal:  needsRenewal,
 		client:        client,
+		modifyRequest: settings.ModifyRequest,
 		name:          "<remote file>",

 		conns: make(map[string]*conn),
diff --git a/vendor/github.com/itchio/wharf/eos/eos.go b/vendor/github.com/itchio/wharf/eos/eos.go
index 44f5a21..820c290 100644
--- a/vendor/github.com/itchio/wharf/eos/eos.go
+++ b/vendor/github.com/itchio/wharf/eos/eos.go
@@ -136,7 +136,15 @@ func realOpen(name string, opts ...option.Option) (File, error) {
 	switch u.Scheme {
 	case "http", "https":
 		res := &simpleHTTPResource{name}
-		hf, err := htfs.Open(res.GetURL, res.NeedsRenewal, htfsSettings())
+
+	   	key := os.Getenv("EXAMPLE_API_KEY")
+
+		settings := htfsSettings()
+		settings.ModifyRequest = func (req *http.Request) {
+        		req.Header.Set("Cookie", "example=" + key)
+    		}
+
+		hf, err := htfs.Open(res.GetURL, res.NeedsRenewal, settings)

 		if err != nil {
 			return nil, err


Play well with other SPI devices

Developers like myself over at https://github.com/MarlinFirmware/Marlin/issues/1226 are scratching our heads how to get SD card reading to work alongside SPI-driven thermocouples and other SPI devices. I'm not personally familiar with the use of chip-select lines, or whether these other devices are designed to play well on SPI buses either, but perhaps you can provide some insight to guide us in the right direction....?
https://github.com/MarlinFirmware/Marlin/issues/1227

https://learn.sparkfun.com/tutorials/serial-peripheral-interface-spi

Studying your code is also very helpful, thanks!


Support both password and link-based sign-in with email

According to Firebase documentation, it should be possible to support both password and link-based sign-in with email.
REF: https://firebase.google.com/docs/auth/android/email-link-auth?authuser=1&hl=en#differentiating_emailpassword_from_email_link
Calling setAvailableProviders with two EmailBuilder doesn't work:
Intent signInIntent =
        AuthUI.getInstance()
                .createSignInIntentBuilder()
                .setAvailableProviders(Arrays.asList(
                        new AuthUI.IdpConfig.GoogleBuilder().build(),
                        new AuthUI.IdpConfig.EmailBuilder().build(),
                        new AuthUI.IdpConfig.EmailBuilder().enableEmailLinkSignIn()
                        .setActionCodeSettings(actionCodeSettings).build()))
                .build();

How we can support both password and link-based sign-in with email using FirebaseUI?

Hi @zjamshidi,
Thank you for filing this issue. Can you please add more information on what exactly doesn't work and what error you are seeing?
Here are some relevant docs on adding FirebaseUI to your Android app: https://firebase.google.com/docs/auth/android/firebaseui#before_you_begin

Hi @ssbushi,
Thanks for your comment. I successfully set up Firebase UI auth in my android app providing email/password and Google login. Now I want to add support for email-link auth.  Since some users are currently using the email/password login, I want to support both password and link-based sign-in with email in my app. Here is my code:
List<AuthUI.IdpConfig> providers = Arrays.asList(
                        new AuthUI.IdpConfig.EmailBuilder().setAllowNewAccounts(false).build(),
                        new AuthUI.IdpConfig.GoogleBuilder().build(),
                        new AuthUI.IdpConfig.EmailBuilder()
                                .enableEmailLinkSignIn()
                                .setActionCodeSettings(actionCodeSettings).build()
                );

If I comment out the first EmailBuilder, the app supports email-link auth. If I comment out the second EmailBuilder, the app supports email/password auth. If I keep both, both buttons will act the same providing email/password auth for current users.

Any update on this issue?

@zjamshidi thanks for filing this issue. It seems like you're having the same problem as #1735
We'll have to do a behavior change in FirebaseUI to support both email sign in methods, that's why the issue was marked as a feature request.
I'm closing this as a duplicate of #1735 - you can track the progress of this feature request on that issue instead.


HTTP protocol

[ ] GET, POST and DELETE methods
[ ] Status codes
[ ] Play and analize with response headers (with REST Client Visual Studio Code extension)


bad indent for math equation

#import "@preview/physica:0.9.0": *
#import "@preview/ctheorems:1.1.2": thmrules, thmbox

#let problem = thmbox(
  "problem",
  "Solution to Problem",
)
#show: thmrules

#problem("1")[
  $
    pdv(,t)braket(phi.alt (t), psi(t))=hbar / i
    mel(
    phi.alt(t),
    (-H^dagger (hat(x),hat(p), t)+H(hat(x),hat(p),t))
    ,psi(t)
  ),
  $
]

this is because currently we dont format func call in math equations


Polish feature revisions

Came in via email:
Hello,
Today, while browsing the Grambank database, I have noticed that Polish has been classified as exhibiting a marker for sensory evidentiality in feature BG322. I decided to check the source, and unfortunately page 180 of Swan's grammar doesn't seem to report anything even remotely resembling evidentiality. The neighboring feature BG323 has some actual evidence supporting it on page 408, but I doubt the distinction between "że" and "czy" differs that much from e.g. English  "that" vs "if/whether" or Russian "что" vs "ли". I would like suggesting that someone take a look at these two and possibly correct them.
Kind regards,
Diego Ostoja-Kowalski

Hiya @HedvigS,
Thank you Diego Ostoja-Kowalski for spotting this!
I have had a look through the grammar and can confirm that this is an incorrect coding.
On page 180 I cannot see any mention of evidentiality. The page is focused on the usage of adjectives and pronouns and about comparison, identification etc. On page 408, the examples mainly discuss "że" and "czy" after reporting verbs; there doesn’t seem to be any grammatical markers in the authors' discussions or examples that mark evidentiality.
I'll go ahead and correct this coding to 0 in the sheet.


refactor: remove/update comments

:tada: This PR is included in version 0.82.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


bug: getCurrentUser returns undefined user

I'm making an Android app with framework7 and I'm stuck. I think it worked a few days ago so I'm afraid om doing something wrong. So probably not a bug. :|
Plugin version:

@robingenz/capacitor-firebase-authentication": "^0.4.0
firebase": "^9.6.3
Platform(s):

Android 11
minSdkVersion = 26
compileSdkVersion = 30
targetSdkVersion = 30
Current behavior:

The user clicks a link, my app should check if the user is logged in. but this fails and user is redirected to a login page, because the user is undefined.
EDIT: The logout is also not working. It looked like it worked, but if call firebase/auth directly I'm still logged on.
Expected behavior:

The user clicks a link, my app should check if the user is logged in. If the user isn't logged on they should be redirect to login page, otherwise the content page if the content is bought.
Related code:

import { initializeApp } from 'firebase/app';
import { FirebaseAuthentication } from '@robingenz/capacitor-firebase-authentication';
import { getAuth, browserLocalPersistence, GoogleAuthProvider, setPersistence, signInWithCredential,} from "firebase/auth";
const firebaseConfig = {
    apiKey: "xxxx",
    authDomain: "xxxx",
    projectId: "xxxx",
    storageBucket: "xxxx",
    messagingSenderId: "xxxx",
    appId: "xxxx"
  };

const FirebaseApp = initializeApp(firebaseConfig);

const getCurrentUser = async () => {
  const result = await FirebaseAuthentication.getCurrentUser();
  return result.user;
};

 $('#kopen-of-tonen').on('click', () => {
        getCurrentUser().then((user) =>{
        alert('user: '+user); //user: undefied

        alert('user: '+ getAuth().currentUser.email); //succesfully returns emailaddress
        if (user != null) {
            if((true)){
              //load content
              request();
            }else {
              //buy access to content
              store.order('oefeningen');
        };
        } else {
            // No user is signed in
            app.views.main.router.navigate('/register/');
        }
      });
     })

toegang

Other information:

When I run my app in the webbrowser I don't encounter problems(user is null if not logged on). I can also login succesfully in the webbrowser and in my android app.
I was thinking maybe it has something to with my Content-Security-Policy. This looks like this:
<meta http-equiv="Content-Security-Policy" content="default-src 'self' 'unsafe-inline' 'unsafe-eval' gap: data: https://validator.fovea.cc https://ssl.gstatic.com https://blablabla.be https://*.google.com https://fonts.gstatic.com https://5e2c7.firebaseapp.com https://*.googleapis.com https://cdnjs.cloudflare.com https://unpkg.com https://cdn.firebase.com https://*.firebaseio.com  https://securetoken.googleapis.com;">

Capacitor doctor:

Latest Dependencies:

  @capacitor/cli: 3.4.0
  @capacitor/core: 3.4.0
  @capacitor/android: 3.4.0
  @capacitor/ios: 3.4.0

Installed Dependencies:

  @capacitor/ios: not installed
  @capacitor/cli: 3.4.0
  @capacitor/android: 3.4.0
  @capacitor/core: 3.4.0

Hi @kabouter,
thank you for your request.
From what I see in your code, it should work.
I would have to look at your code and configuration to be able to help you further.
Please provide a Minimal, Reproducible Example (see How to create a Minimal, Reproducible Example).

I was thinking maybe it has something to with my Content-Security-Policy. This looks like this:

Does an error appear anywhere in the Developer Console or the Network tab?
Can you remove this tag for testing?

I stripped everything and had only auth things left. I still had the problem.
I changed from firebase web to Firebase Android.
Now it is working. User is still undefied when not logged on, but that is not a real problem. When logged on the is no longer undefied.


Use errors.Is to check for a specific error

Signed-off-by: Fish-pro<EMAIL_ADDRESS>Comparing with == will fail on wrapped errors. Use errors.Is to check for a specific error

Hi @Fish-pro ,
Have you considered using errorlint to ensure the conformity to this rule ?

Hi @Fish-pro , Have you considered using errorlint to ensure the conformity to this rule ?

@mmorel-35 Yes, I have, but new tests need to be introduced


Sampling

This implements prior and posterior sampling. I also included testing scripts for several functionalities. Those are not unittests, but if you find the time, have a look at test_sampling.py and test_write.py, maybe also test_pymagglobal.py if you have pymagglobal installed...

Is your comment regarding the errors still valid? I can address the issue if necessary.

I already took care of it :)
Op do 4 apr 2024 om 15:37 schreef M. Arthus Schanner <
@.***>

Is your comment regarding the errors still valid? I can address the issue
if necessary.
—
Reply to this email directly, view it on GitHub
https://github.com/outfrenk/geomagnetic_field_inversions/pull/13#issuecomment-2037250930,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANZZXPEIRXU27UBFY74CHS3Y3VJSXAVCNFSM6AAAAABFW4B2EOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZXGI2TAOJTGA
.
You are receiving this because you modified the open/close state.Message
ID: @.***>


Compatibility with newer versions of cryptography.

Firstly, I just discovered mitmproxy and I'm pretty excited to make use of it.  Thank you for maintaining it.
Problem Description
I'm trying to use mitmproxy as part of an integration test in a project that depends on cryptography, specifically version 40.0.0.  I see that mitmproxy requires cryptography at <38.1.  I can automate the creation of a separate venv for the mitmproxy stuff, but that's kind of clunky.  I'd rather just make it a dependency and let poetry handle it.  For that to happen, mitmproxy needs to be compatible with higher versions of cryptography.
I recognize that the <38.1 constraint might be there for a good reason, but not knowing that reason I figured I'd file an issue and cross my fingers.  Thanks for at least considering it.
Steps to reproduce the behavior:

Have a pyproject.toml like this:

...

[tool.poetry.dependencies]
python = "^3.10"
cryptography = "^40.0.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Run poetry add mitmproxy
See error:

Because no versions of mitmproxy match >9.0.1,<10.0.0
 and mitmproxy (9.0.1) depends on cryptography (>=38.0,<38.1), mitmproxy (>=9.0.1,<10.0.0) requires cryptography (>=38.0,<38.1).
So, because foo depends on both cryptography (^40.0.0) and mitmproxy (^9.0.1), version solving failed.

System Information
Mitmproxy: 9.0.1
Python:    3.10.9
OpenSSL:   OpenSSL 1.1.1s  1 Nov 2022
Platform:  Linux-5.15.86-x86_64-with-glibc2.35

We already upgraded, you have to wait for the next release.
https://github.com/mitmproxy/mitmproxy/blob/be02b1e298dfb1b218007f824c4cf345c53b0cee/pyproject.toml#L37
https://github.com/mitmproxy/mitmproxy/commit/8c6ec5cb56fbf4961806ed27b4974c440df59e87
https://github.com/search?q=repo%3Amitmproxy%2Fmitmproxy+cryptography&type=issues
https://github.com/mitmproxy/mitmproxy/issues/5966
Feel free to re-open if I miss something. I have no idea what poetry is.

https://github.com/mitmproxy/mitmproxy/issues/5966#issuecomment-1455045917

I don't think either of the posted vulnerabilities affects mitmproxy users in a significant way, so I don't think we will ship a patch release just for that. :)

@Prinzhorn Do you have a target date for the next release?  Our customers in regulated industries (banking, federal, etc.) are prohibited by regulations from using s/w with known High and Critical severity vulnerabilities. I'm trying to formulate our case for waiting, but need some kind of target date.

@FrugalGuy: If you are interested in timely patch releases to fix your company's compliance requirements, I'm happy to set up a support contract. Email is on my profile. :-)

@Prinzhorn Do you have a target date for the next release? Our customers in regulated industries (banking, federal, etc.) are prohibited by regulations from using s/w with known High and Critical severity vulnerabilities. I'm trying to formulate our case for waiting, but need some kind of target date.

Yeah, no, also, welcome to the front page of Hacker News, IBM! It bet the optics look great with the controversy around Red Hat. I worked at IBM for seven years after my startup was acquired and can say; IBM has a terrible track record of monetizing FOSS and ruining it for the source projects.
May want to do some reflection and check into IBM's SLA agreements and support contracts before accusing others of "thinly veiled extortion".
Why don't you log back into ServiceNow and let these customers (and surely, the uncoordinated internal IBM teams fighting) know that IBM is at fault here and that mitmproxy awaits your team's PRs?

Since this has been picked up by some news sites I'd like to provide some additional context here: I don't mind the release question at all, it's valid! But the context should ideally be along the lines of "we have an interest in this, how can we help make it happen" (contributions or $) and not "you are causing problems for our customers". I don't want the requestor to have a miserable time because of a badly-worded comment, I want large companies to have a healthy relationship with FOSS.

@FrugalGuy has just sent me genuine apology, which I truly appreciate. Please be nice and assume good intentions. ❤


Bug:  hasChanged() always returns true even if a field of boolean type is reassigned with the same value without change.

PHP Version
7.4
CodeIgniter4 Version
4.1.8
CodeIgniter4 Installation Method
Manual (zip or tar.gz)
Which operating systems have you tested for this bug?
Windows
Which server did you use?
apache
Database
MySQL 8.0.27
What happened?
I am currently working on a project where there is a boolean type column (state) in a table, I am working on this with an Entity, when I make a post to update a record in that table, I make the query with ->find() and I reassign the information of the inputs through the function ->fill(), example:
-- In Agent_Entity.php :
namespace App\Entities;
use CodeIgniter\Entity;
class Agent_Entity extends Entity
{
    protected $attributes = [
      ..
      ..
      "status" => true, //default in the DB table
      "name" => null
    ];

    protected $casts = [
      ..
      ..
      "status" => "boolean",
      "name" => "?string"
    ];
}

-- In Agent_Model.php :
namespace App\Models;
use CodeIgniter\Model;
class Agent_Model extends Model
{
    protected $table = "agent";
    protected $primaryKey = "id_agent";
    ..
    ..
    protected $returnType = \App\Entities\Agent_Entity::class
}

-- In Agent_Controller.php :
namespace App\Controllers;
class Agent_Controller extends Base_Controller
{
    function update($_id) {
        $name = trim($this->request->getPost("name", FILTER_SANITIZE_STRING));
        $status = (bool) $this->request->getPost("status", FILTER_UNSAFE_RAW);
        ..
        ..
        $agent = model("agent_model")->find($_id); //this returns an Agent_Entity()
        ..
        ..
        $agent->fill([
            ..
            ..
            "name" => $name,
            "status" => $status
        ]);

        //here is the bug
        //here it always returns true, even if the same information that is registered in the database is assigned
        if ( $agent->hasChanged() ) {
             //send to save to database
        }
        else {
            echo "without changes"; exit;
        }
   }
}

When I call the function ->hasChange() making it ->fill() with the same data that is registered (without changes), ->hasChanged() always returns true, but when I comment it, it means:
//in this case, ->hasChanged() it returns false (what is expected)
$agent->fill([
    ..
    ..
    "name" => $name,
    //"status" => $status
]);

sending the same registered information (without making changes) returns false.
The error is in this latest version because this project was working very well in version 4.1.5 or 4.1.6 I don't remember which one it had
Steps to Reproduce
->hasChange()
It must return false since sometimes changes are not made in the name or in the state and neither in the other fields.
Expected Output
false when there is no change even when a boolean field is included
Anything else?
No response

Call dd($agent); after filling.
The attributes and original properties must be the same.

In the current implementation, it seems $attributes has raw values from the database when fetched from the database.
https://github.com/codeigniter4/CodeIgniter4/blob/90eef161c1e7cb4af6165dfa87e8d4be074092fa/system/Database/MySQLi/Result.php#L145

What's the validation problem?

Hello everyone, look at the problem is the following:
When I send a POST form with the same information registered in the database, that is, without changes, it is supposed that when doing the validation with hasChanged() it should throw me False, because no change was made, this is not happening since the hasChanged() function takes the Status field (boolean) as if it always changed, but when I comment, the Status field, that is, I don't add it in the Fill(), there is if it works correctly.

MySQL does not have a boolean data type. Any boolean value is converted to 1 or 0.
MySQL php driver returns all data as a string.
This means that the "status" field received from the database will have the value "1" or "0".
Here you convert the value to boolean
 $status = (bool) $this->request->getPost("status", FILTER_UNSAFE_RAW);
        $agent->fill([
            //...
            "status" => $status
        ]);

Can you guess what the result of the expression bool === string will be?

@iRedds Thanks for your clarification, but in the model it returns an Entity (Agent_Entity), which converts this field (Status) into boolean, which I suppose hasChanged() will find of the same data type.
namespace App\Entities;
use CodeIgniter\Entity;
class Agent_Entity extends Entity
{
    protected $attributes = [
      ..
      ..
      "status" => true, //default in the DB table
      "name" => null
    ];

    protected $casts = [
      ..
      ..
      "status" => "boolean",
      "name" => "?string"
    ];
}

protected $returnType = \App\Entities\Agent_Entity::class

I performed the following validation before doing the fill() on the
Controller and the result was True, this confirms that if there is any problem in the hasChanged()
$agent = model("agent_model")->find($_id); //this returns an Agent_Entity()
echo gettype($agent->status); // this returned Boolean
echo json_encode($agent->status === (bool) $status); // this returned True, This means that there are no changes, which is correct
exit;

The "boolean" type casting only works for getting, not for setting.
That is, when you call $agent->status
That is, the original value does not change.
As I said above

Call dd($agent); after filling.
The attributes and original properties must be the same.

hi @iRedds, You are right, I just assigned in the fill() the state with type string ("1" or "0") and it works correctly.
The code is as follows in the Controller:
$status= (bool) $this->request->getPost("status", FILTER_UNSAFE_RAW); //checkbox value, if checked is true, otherwise (null) is false
..
..
$agent = model("agent_model")->find($_id); //this returns an Agent_Entity()
..
..
$agent->fill([
    ..
    ..
    "name" => $name,
    "status" => ($status) ? "1" : "0"
]);
..
..
if ( $agent->hasChanged() ) { // this returned False, is correct!!
     //send to save to database
}
...

Thank @iRedds, and thanks to all, I close the topic.

You can extend the CodeIgniter\Entity\Cast\BooleanCast class by adding your own setter.
https://codeigniter.com/user_guide/models/entities.html#custom-casting

But the current behavior is difficult to understand, and inconsistent:

$attributes has raw values from the database when fetched from the database.
when you create an Entity and fill() data, $attributes values are casted.


Guevera

Want to win

Yes, I want to win. Please help me out
On Sun, Jul 14, 2024, 2:10 AM Muralicgv @.***> wrote:

Want to win
—
Reply to this email directly, view it on GitHub
https://github.com/ton-blockchain/wallet-contract-v5/issues/32, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/BJWNMMWBEQKUJ36DBET6X33ZMHFXFAVCNFSM6AAAAABK2W56H6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGQYDOMRRHE2DMMA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Revisit gather_statistics in read_parquet

Loading metadata from parquet files in the client can be costly; parsing the statistics and looking for ordering can also be costly.

We should limit when we load metadata at all, either from a global file, otherwise from the first data file
If the pandas metadata says there is an index, or the user specifies an index, we need to parse the statistics for that column only; do not try to guess an index
While parsing, we can know early that ordering won't work, so bail early

As stated, this implies that any filters will be applied at read time, so we don't proactively cut down on partitions, but some partitions might come back empty (but the work is done on the worker).

@rjzamora , does this reflect the gist of what we decided?

Thanks for summarizing @martindurant !
This pretty much covers the metadata/filtering component of our discussion.  The one grey area is that we don't necessarily need the filtering to be applied at read time.  Rather, we would like to avoid applying filters on the client using python code in dask.  That is, I think we can/should still allow the pyarrow.datset API to filter out row-group fragments during dataset discovery.

That is, I think we can/should still allow the pyarrow.datset API to filter out row-group fragments during dataset discovery.

Yes, totally happy to pass a filters= when setting up the graph, but I don't think we want or have a way to transform df[filter] into such clauses and rewriting the graph via an optimisation - we'll just end up with some empty partitions instead. We currently only do this where we have an index with known "division" values from the statistics metadata.
Did I understand correctly?
(note the interesting example of https://github.com/holoviz/spatialpandas , which does apply limits on columns to rewrite the graph, specialised to lat/lon for datasets where the partitioning strategy is well known example)

Closing as stale, #8058 better reflects current metadata handling thoughts.


unable to install nvm on windows systems: python not found

Operating system and version:
Windows, Version	10.0.19042 Build 19042
nvm debug output:

$ nvm debug
bash: nvm: command not found

nvm ls output:

How did you install nvm?

I installed using: curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
What steps did you perform?
I installed nvm using the above curl command. WHen I verified using: command -v nvm it gave the output as nvm. After this I tried to install node using the command: nvm install node # "node" is an alias for the latest version
What happened?
I am getting the below error:
$ nvm install node # "node" is an alias for the latest version
Can not determine how many core(s) are available, running in single-threaded mode.
Please report an issue on GitHub to help us make nvm run faster on your computer!
Local cache found: ${NVM_DIR}/.cache/src/node-v16.10.0/node-v16.10.0.tar.xz
Checksums match! Using existing downloaded archive ${NVM_DIR}/.cache/src/node-v16.10.0/node-v16.10.0.tar.xz
$>./configure --prefix=/c/Users/PrarthanaReddy/.nvm/versions/node/v16.10.0 <
./configure: line 6: exec: python: not found
nvm: install v16.10.0 failed!
What did you expect to happen?
Is there anything in any of your profile files that modifies the PATH?

I cannot see any of these files.

If you are having installation issues, or getting "N/A", what does curl -I --compressed -v https://nodejs.org/dist/ print out?

 curl -I --compressed -v https://nodejs.org/dist/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying <IP_ADDRESS>:443...
* Connected to nodejs.org (<IP_ADDRESS>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
*  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [19 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4904 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [520 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: OU=Domain Control Validated; OU=PositiveSSL Wildcard; CN=*.nodejs.org
*  start date: Oct 21 00:00:00 2019 GMT
*  expire date: Jan 18 23:59:59 2022 GMT
*  subjectAltName: host "nodejs.org" matched cert's "nodejs.org"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x2ed5bab1ae0)
} [5 bytes data]
> HEAD /dist/ HTTP/2
> Host: nodejs.org
> user-agent: curl/7.75.0
> accept: */*
> accept-encoding: deflate, gzip, br, zstd
>
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [230 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [230 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
} [5 bytes data]
< HTTP/2 200
< date: Mon, 04 Oct 2021 10:22:11 GMT
< content-type: text/html
< last-modified: Mon, 04 Oct 2021 07:05:03 GMT
< cache-control: max-age=14400
< cf-cache-status: HIT
< age: 3884
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< vary: Accept-Encoding
< server: cloudflare
< cf-ray: 698db009ee223c0c-BLR
< content-encoding: br
<
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0HTTP/2 200
date: Mon, 04 Oct 2021 10:22:11 GMT
content-type: text/html
last-modified: Mon, 04 Oct 2021 07:05:03 GMT
cache-control: max-age=14400
cf-cache-status: HIT
age: 3884
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
vary: Accept-Encoding
server: cloudflare
cf-ray: 698db009ee223c0c-BLR
content-encoding: br

* Connection #0 to host nodejs.org left intact

@prarredd not sure how you run the shell on Windows, have you checked this section https://github.com/nvm-sh/nvm#important-notes ?

The latest released version of nvm does not support Windows outside of WSL.
The next version, however (which is what the readme on master describes) will work on most Windows environments.

@prarredd not sure how you run the shell on Windows, have you checked this section https://github.com/nvm-sh/nvm#important-notes ?

Hello.. I have used GitBash to run the command on windows.

@prarredd nvm does not support Git Bash yet. It will, however, in the next version.
For now, use WSL2.


[zh-cn]sync readme.md --Update GO address link

golang.org->golang.google.cn(official website in China address link)
I'm not sure if this should be changed to the official website of mainland China？

/language zh

/cc @tengqm


Refactor evohome to prepare for water_heater

Breaking Change:
None.
Description:
Refactor evohome in preparation for adding a water_heater component.

moved/renamed the base class to __init__.py
add const.py
change order of methods/properties to match
add client library to top of file

Related issue (if applicable): N/A
Pull request in home-assistant.io with documentation (if applicable): N/A
Checklist:

[x] The code change is tested and works locally.
[ ] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.

If user exposed functionality or configuration variables are added/changed: N/A
If the code communicates with devices, web services, or third-party tools:

[x] The manifest file has all fields filled out correctly (example).
[x] New dependencies have been added to requirements in the manifest (example).
[x] New or updated dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[x] New files were added to .coveragerc.

If the code does not interact with devices: N/A

@MartinHjelmare do you have time to review this?
I am sorry about re-ordering the properties, but they are in the same order as the base classes now, and is much easier to work with - will put them back in the original order if you insist.


[big-refactor] Accelerate launch FSDP Runtime Error

Hi when running accelerate launch with FSDP I run into the following error:
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: 'weight' must be 2-D

I am running eval on 2 GPUs, the error message is replicate on both GPUs. Typically one batch is completed on one of the GPUs before erroring out.

What is the exact command you are running?

Same issue on Nvidia L4 x 2
Command: accelerate launch -m lm_eval --model hf --model_args pretrained=meta-llama/Llama-2-7b-chat-hf --tasks arc_challenge --batch_size 1 --num_fewshot=25
Accelerate conf:
compute_environment: LOCAL_MACHINE
debug: false
distributed_type: FSDP
downcast_bf16: 'no'
fsdp_config:
  fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP
  fsdp_backward_prefetch_policy: BACKWARD_PRE
  fsdp_cpu_ram_efficient_loading: true
  fsdp_forward_prefetch: true
  fsdp_offload_params: false
  fsdp_sharding_strategy: 1
  fsdp_state_dict_type: SHARDED_STATE_DICT
  fsdp_sync_module_states: true
  fsdp_use_orig_params: true
machine_rank: 0
main_training_function: main
mixed_precision: bf16
num_machines: 1
num_processes: 2
rdzv_backend: static
same_network: true
tpu_env: []
tpu_use_cluster: false
tpu_use_sudo: false
use_cpu: false

Error:
    ......
    File "/opt/conda/envs/eval/lib/python3.10/site-packages/torch/nn/functional.py", line 2233, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: 'weight' must be 2-D

When i change num_processes to 1 it works.
Thanks

Using SIZE_BASED_WRAP it works (the memory allocated for each gpu is higher), is it normal?
I thought it was possible to use LLama2 with TRANSFORMER_BASED_WRAP.

facing the same issue with 8xA6000

I have this issue on main branch (release 4.0) on 8xa100s 40gb when trying to eval 70B models.
Trying SIZED_BASED_WRAP gets me another issue:
File "/usr/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/transformers/models/llama/modeling_llama.py", line 107, in forward
    return self.weight * hidden_states.to(input_dtype)
RuntimeError: The size of tensor a (0) must match the size of tensor b (8192) at non-singleton dimension 2

No wrap would just OOM.
My config is almost identical to above, just num gpu difference.

Same issue here as well with TRANSFORMER_BASED_WRAP: RuntimeError: 'weight' must be 2-D
SIZED_BASED_WRAP seems to work but then NCCL timeouts (30minutes) on the last request batch. It is hanging on some processing.

We now recommend using vLLM instead of FSDP for fast / big model generation where possible.
#1520 may also fix the NCCL timeouts due to a padding bug?


@prisma/engines weights 113 MB and causes maximum size limit error on Vercel

Bug description
When I try to deploy a Next.js + GraphQL + Prisma application on Vercel I receive an error about exceeding the lambda function limit.
23:15:50.190  	Max serverless function size of 50 MB compressed or 250 MB uncompressed hit
23:15:50.190  	Serverless Function's page: api/graphql.js
23:15:50.191  	Large Dependencies                          Uncompressed size  Compressed size
23:15:50.191  	node_modules/@prisma/engines                           113 MB          39.6 MB
23:15:50.191  	node_modules/.prisma/client                           45.9 MB          16.1 MB
23:15:50.191  	node_modules/prisma/build                             5.27 MB          1.09 MB

Full log here: https://gist.github.com/talentlessguy/74f58664cd5af05407037be15b3b25a1
How to reproduce

Create a new Next.js project with these files: https://gist.github.com/talentlessguy/9cac915be4fa6a1ea911c7f0ead5fa2b

Expected behavior
I expected a Next.js app to build and launch as it usually does (it did run fine 1-2 weeks ago)
Prisma information
Schema: https://gist.github.com/talentlessguy/9cac915be4fa6a1ea911c7f0ead5fa2b#file-graphql_schema-ts
Environment & setup

OS: Vercel Linux runtime
Database: PostgreSQL
Node.js version: 14.16
Prisma version: 2.22.0

prisma               : 2.22.0
@prisma/client       : 2.22.0
Current platform     : debian-openssl-1.1.x
Query Engine         : query-engine 60cc71d884972ab4e897f0277c4b84383dddaf6c (at node_modules/.pnpm/@prisma+engines<EMAIL_ADDRESS>Migration Engine     : migration-engine-cli 60cc71d884972ab4e897f0277c4b84383dddaf6c (at node_modules/.pnpm/@prisma+engines<EMAIL_ADDRESS>Introspection Engine : introspection-core 60cc71d884972ab4e897f0277c4b84383dddaf6c (at node_modules/.pnpm/@prisma+engines<EMAIL_ADDRESS>Format Binary        : prisma-fmt 60cc71d884972ab4e897f0277c4b84383dddaf6c (at node_modules/.pnpm/@prisma+engines<EMAIL_ADDRESS>Default Engines Hash : 60cc71d884972ab4e897f0277c4b84383dddaf6c
Studio               : 0.379.0

You should not be uploading the Prisma CLI and the Engines dependency to Vercel. How are you deploying your app?

@janpio I deploy it using vercel github integration

FYI, it works fine with prisma and @prisma/client v2.21.2

Yes, the binary size might have increased a tiny bit, putting you above the limit that is now being broken. That is unfortunate, but only makes the problem visible that existed before. I'll try to understand why it is uploading all these dependencies although it should not.
You could help me by creating a "Create a new Next.js project with these files: gist.github.com/talentlessguy/9cac915be4fa6a1ea911c7f0ead5fa2b" and put that into a new reproduction repository that I can easily fork and connect to a test Vercel project. That would save me a lot of time.

@janpio ok, please wait a few minutes while I'm bootrstraping a repo

@janpio here it is: https://github.com/talentlessguy/prisma-limit-repro
let me know if something is missing in there

Vercel deploy outputs this:
...
23:13:10 | It looks like you're trying to use TypeScript but do not have the required package(s) installed.
-- | --
23:13:10 | Please install @types/react by running:
23:13:10 | yarn add --dev @types/react
23:13:10 | If you are not trying to use TypeScript, please remove the tsconfig.json file from your package root (and any TypeScript files in your pages directory).
23:13:10 | error Command failed with exit code 1.

@janpio oops my bad
fixed
please pull from latest commit

lib/prisma seems to be missing now :)

@janpio oops sorry just added it

23:40:44 | Failed to compile.
-- | --
23:40:44 | ./graphql/resolvers.ts:5:7
23:40:44 | Type error: Property 'crud' does not exist on type 'ObjectDefinitionBlock<"Query">'.
23:40:44 | 3 \| export const Query = queryType({
23:40:44 | 4 \|   definition(t) {
23:40:44 | > 5 \|     t.crud.posts()
23:40:44 | \|       ^
23:40:44 | 6 \|     t.crud.post()
23:40:44 | 7 \|     t.crud.user()
23:40:44 | 8 \|     t.crud.images()
23:40:44 | error Command failed with exit code 1.

@janpio you need to generate nexus:
pnpm generate:nexus

Even if I modify the deploy command in Vercel manually, I then get this:
...
00:42:53 | Running "pnpm generate:nexus & next build"
-- | --
00:42:53 | sh: pnpm: command not found
00:42:54 | info  - Using webpack 5. Reason: future.webpack5 option enabled https://nextjs.org/docs/messages/webpack
...

Even if I install pnpm manually before, I do not get to success unfortunately: npm i -g<EMAIL_ADDRESS>--unsafe-perm && pnpm generate:nexus && next build:

10:50:22 | Running "npm i -g<EMAIL_ADDRESS>--unsafe-perm && pnpm generate:nexus && next build"
-- | --
10:50:23 | /node14/bin/pnpm -> /node14/lib/node_modules/pnpm/bin/pnpm.js
10:50:23 | /node14/bin/pnpx -> /node14/lib/node_modules/pnpm/bin/pnpx.js
10:50:23 | +<EMAIL_ADDRESS>10:50:23 | added 1 package from 1 contributor in 0.717s
10:50:24 | ><EMAIL_ADDRESS>generate:nexus /vercel/path0
10:50:24 | > ts-node --transpile-only -P nexus.tsconfig.json pages/api/graphql.ts
10:50:24 | Warning<EMAIL_ADDRESS>does not support<EMAIL_ADDRESS>The supported range is: `2.21.x`. This could lead to undefined behaviors and bugs.
10:50:25 | info  - Using webpack 5. Reason: future.webpack5 option enabled https://nextjs.org/docs/messages/webpack5
10:50:25 | info  - Checking validity of types...
10:50:25 | Attention: Next.js now collects completely anonymous telemetry regarding usage.
10:50:25 | This information is used to shape Next.js' roadmap and prioritize features.
10:50:25 | You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
10:50:25 | https://nextjs.org/telemetry
10:50:28 | Failed to compile.
10:50:28 | ./graphql/resolvers.ts:5:7
10:50:28 | Type error: Property 'crud' does not exist on type 'ObjectDefinitionBlock<"Query">'.
10:50:28 | 3 \| export const Query = queryType({
10:50:28 | 4 \|   definition(t) {
10:50:28 | > 5 \|     t.crud.posts()
10:50:28 | \|       ^
10:50:28 | 6 \|     t.crud.post()
10:50:28 | 7 \|     t.crud.user()
10:50:28 | 8 \|     t.crud.images()
10:50:28 | Error: Command "npm i -g<EMAIL_ADDRESS>--unsafe-perm && pnpm generate:nexus && next build" exited with 1

@janpio huge apologies for repeatedly forgetting to add missing files but now I think it's finally all set

I can reproduce with your repo:
<html>
<body>
<!--StartFragment-->

- | -
-- | --
17:00:19 | Max serverless function size of 50 MB compressed or 250 MB uncompressed hit
17:00:19 | Serverless Function's page: api/graphql.js
17:00:19 | Large Dependencies                          Uncompressed size  Compressed size
17:00:19 | node_modules/@prisma/engines                           113 MB          39.6 MB
17:00:19 | node_modules/.prisma/client                           45.9 MB          16.1 MB
17:00:19 | node_modules/prisma/build                             5.27 MB          1.09 MB
17:00:19 | node_modules/@prisma/client                           3.93 MB           822 kB
17:00:19 | node_modules/busboy/deps                               618 kB           196 kB
17:00:19 | node_modules/micro/node_modules                        360 kB           190 kB
17:00:19 | node_modules/encoding/node_modules                     328 kB           179 kB
17:00:19 | node_modules/lodash/lodash.js                          544 kB          96.4 kB
17:00:19 | All dependencies                                       173 MB          59.1 MB
17:00:19 | Max serverless function size was exceeded for 1 function
17:00:20 | Created all serverless functions in: 36.748s
17:00:32 | Failed to process build result for "api/graphql". Data: {"type":"Lambda","zipBuffer":{"type":"Buffer","data":[80,75,3,4,20,0,0,8,8,0,9,120,165,82,104,52,206,48,15,6,0,0,57,24,0,0,31,0,0,0,46,110,101,120,116,47,115,101,114,118,101,114,47,119,101,98,112,97,99,107,45,114,117,110,116,105,109,101,46,106,115,173,88,2.
17:00:32 | Error: The Serverless Function "api/graphql" is 57.26mb which exceeds the maximum size limit of 50mb. Learn More: https://vercel.link/serverless-function-size
``

@janpio so this means you can take a look at the issue?

Yes, now I have a reproduction that I can poke.

Had the same issue. Works with Prisma 2.21.2.

@janpio we'll probably be impacted as well in the future... The binary size has slightly grow (80kb) but there's something I'm wondering that could help.
Why do we have 2 times the query-engine-debian-openssl-1.1.x in 2 different directories ?
in ./node_modules/.prisma/client/
 ls -lah ./node_modules/.prisma/client/
total 49M
drwxrwxr-x 2 sebastien sebastien 4,0K mai    6 15:59 .
drwxrwxr-x 3 sebastien sebastien 4,0K mars  29 17:34 ..
-rw-rw-r-- 1 sebastien sebastien  19K mai    6 15:59 index-browser.js
-rw-rw-r-- 1 sebastien sebastien 2,3M mai    6 15:59 index.d.ts
-rw-rw-r-- 1 sebastien sebastien 5,3M mai    6 15:59 index.js
-rw-rw-r-- 1 sebastien sebastien  110 mai    6 15:59 package.json
-rwxrwxr-x 1 sebastien sebastien  41M mai    6 15:59 query-engine-debian-openssl-1.1.x
-rw-rw-r-- 1 sebastien sebastien  39K mai    6 15:59 schema.prisma

in ./node_modules/@prisma/engines
ls -lah ./node_modules/@prisma/engines
total 99M
drwxrwxr-x 4 sebastien sebastien 4,0K mai    6 15:59 .
drwxrwxr-x 5 sebastien sebastien 4,0K avril  7 15:24 ..
drwxrwxr-x 2 sebastien sebastien 4,0K mai    6 15:58 dist
drwxrwxr-x 2 sebastien sebastien 4,0K mai    6 15:58 download
-rwxrwxr-x 1 sebastien sebastien  19M mai    6 15:58 introspection-engine-debian-openssl-1.1.x
-rwxrwxr-x 1 sebastien sebastien  33M mai    6 15:58 migration-engine-debian-openssl-1.1.x
-rw-r--r-- 1 sebastien sebastien  646 avril 14 15:18 package.json
-rwxrwxr-x 1 sebastien sebastien 7,4M mai    6 15:58 prisma-fmt-debian-openssl-1.1.x
-rwxrwxr-x 1 sebastien sebastien  41M mai    6 15:58 query-engine-debian-openssl-1.1.x

Also, can those one be moved in a dev package that we can exclude in prod.
Just wondering, I guess the problem is more complex as always :smile:
-rwxrwxr-x 1 sebastien sebastien  19M mai    6 15:58 introspection-engine-debian-openssl-1.1.x
-rwxrwxr-x 1 sebastien sebastien  33M mai    6 15:58 migration-engine-debian-openssl-1.1.x

@talentlessguy can you try to remove one of them after install to see ?
ie:
   "build": "next build",
   "vercel-build": "yarn hackprisma && yarn build",
   "hackprisma": "rm<EMAIL_ADDRESS>
We never know...

Why do we have 2 times the query-engine-debian-openssl-1.1.x in 2 different directories ?

You should only have that locally, but @prisma/engines should not be included in the uploaded archive in the first place, as it is a dependency of a dev dependency, prisma CLI. Something seems to be messing with that detection for some reason.
(That is also the reason why the file exists twice - the generated client needs to be self contained as you can also generate to another location via output in the generator block of your schema, and some people then publish that folder as an Npm package, upload it as an artifact or many similar things).
So indeed, complex.

Thanks @janpio for the update. In the meantime enabling napi fixed it for me

One more data point:
Was unable to upload to Vercel on version 2.20.0. Able to download with:
    "@prisma/client": "^2.23.0-dev.17",
    "nexus-plugin-prisma": "^0.34.1",

Another data point here: I was facing the same issue, but by installing<EMAIL_ADDRESS>and<EMAIL_ADDRESS>it was fixed!
Thanks for your great work for the community 👏 👏 👏

Hi! Today i made it work with prisma and @prisma/client on version 2.23.0-dev.27 hope to see this on production!

This will definitely land in production very soon so you can switch back to latest of Prisma CLI and Client 👍 (Expect it later today.)

Ok, so we figured out what is going on here and I need to document that for later:

This was indeed caused by a change in Prisma CLI (Prisma Studio more exactly) between 2.22.0-dev.39 and 2.22.0-dev.40 - and was shipped with 2.22.0.
Our end to end tests did not trigger the failure, as it only appears in relation with nexus-plugin-prisma - which you are all using in your projects.
We could easily fix the problem in Prisma CLI already which we did in early 2.23.0-dev, which will of course also be included in 2.23.0.
Left us with the problem that we did not understand what is going on in nexus-plugin-prisma what triggered this bug in the CLI to become active.
After help from Vercel we could confirm that 1 specific line in nexus-plugin-prisma led to Prisma Client Postinstall and via that Prisma CLI and via that Prisma studio was triggered: https://github.com/vercel/nft/issues/203#issuecomment-840541429
We could also confirm that a simple change in nexus-plugin-prisma could break that inclusion: https://github.com/graphql-nexus/nexus-plugin-prisma/pull/1089/files Our reproduction project can successfully be published on Vercel when we change the nexus-plugin-prisma dependency to that PR version: https://github.com/janpio/prisma-limit-repro/pull/11/files

Case closed. 📕
Left to do now is to maybe clean up nexus-plugin-prisma a bit and certainly add our reproduction project (Thanks @talentlessguy again!!) to our e2e tests with the failing dependency versions, then update to recent ones to make sure the problems are gone and will never come back (in this one specific combination).

Looks like it's not an issue anymore? I'm using Prisma 2.23 on an old project and it deploys fine


Jesse: Watch britishbookpublishing.co.uk

Jesse requests the watch of the watch_keyword britishbookpublishing\.co\.uk. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

britishbookpublishing\.co\.uk has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Spevacus in Charcoal HQ


Emit Non-Zero Exit Status on Error

Right now, ysc always returns 0, even on error. This makes piloting ysc with other tools much harder, since we must devolve into checking if the stdout is empty or not. (Fwiw, I'm not sure what the politics of stdout or stderr are! The name would suggest stderr is better, but I've never really seen a pattern in other code).

This is probably a good first issue -- I'm on vacation soon but would love to tackle this fella myself

fixed by your own PR, thanks for that


utils: address flake8 errors

Address various issues. Added some ignores so it doesn't conflict with yapf style
Ignored/Exceptions:

E123 closing bracket does not match indentation of opening bracket’s line
E125 continuation line with same indent as next logical line
E251 unexpected spaces around keyword / parameter equals
W503 line break before binary operator
E731 do not assign a lambda expression, use a def
E741 Do not use variables named 'l', 'o', or 'i'

Also changed v2x to add utils to PYTHONPATH

The most important feedback would be on what errors to ignore globally vs. adding one off exceptions. I already have a hand full of E501 for docstrings.

This should wait for #794


Animate the Gapminder data set using gganimate

Inspired by Hans Rosling's TED Talk: The best stats you've ever seen.
The code is taken nearly verbatim from the gganimate example.

Here's the rendered result on RPubs: http://rpubs.com/sjackman/gapminder-gganimate

Awesome! Thanks.


Hello, I want to know what ‘3 training seeds’ mean, can you talk about it in detail?

We report the mean and standard deviation over 9 runs of each method (3 training seeds,
each trained model evaluated 3 times) on Route Completion (RC) and Driving Score (DS) in Town05 Short and Town05
Long settings with scenario and without scenarios. LF: Late Fusion, GF: Geometric Fusion, TF: TransFuser
Hello, I want to know what ‘3 training seeds’ mean, can you talk about it in detail?

We report the mean and standard deviation over 9 runs of each method (3 training seeds,
each trained model evaluated 3 times) on Route Completion (RC) and Driving Score (DS) in Town05 Short and Town05
Long settings with scenario and without scenarios. LF: Late Fusion, GF: Geometric Fusion, TF: TransFuser
Hello, I want to know what ‘3 training seeds’ mean, can you talk about it in detail?

In the experimental design part of the code, I only found the --trafficManagerSeed.
parser.add_argument('--trafficManagerSeed', default='0',
                    help='Seed used by the TrafficManager (default: 0)')

Random seed can affect the training in multiple ways:

The network parameters are initialized by sampling from some distribution. So, every time you start training, the network is initialized differently unless you fix the torch seed.
The dataloader can have multiple sources of randomness: data shuffling, use of other packages like numpy etc. This results in a different data batch every time you rerun training.

It has been noticed in multiple robotics tasks that there can be significant variance in the evaluation performance depending on the random seed used during training. So, we ran the training multiple times with random seeds (by default, it is randomized in PyTorch unless you set it explicitly) to make sure that the results are consistent.
If you want a fully deterministic training protocol such that the network is always initialized the same, the sequence of data batch remains the same then you need to fix all the random seeds in your code (and any other package that you might be using):
def seed(seed=42):
    np.random.seed(seed)
    random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False

Now I understand what '3 training seeds' means, but I'm still wondering about the evaluation seeds.
Did you fix the 'trafficManagerSeed' value as default 0 for 3 times evaluation? or did you change it randomly for every evaluation?
I ask this question because I found that Town05 RouteScenario_23 gives RC score only 2.65 for almost every evaluation and I also found that this is because of the leading vehicle not moving(which is irrelevant to Transfuser's performance since it cannot overtake front vehicles). So I want to modify the movement of this leading vehicle and I want to know if it's possible by changing the random seed.

(This bicycle rider is not moving till game time is over!)
Thank you!

I'm aware of this issue but I don't know how to resolve it.
We did not set the 'trafficManagerSeed' explicitly anywhere so it must be using the default value. The stochasticity in the evaluation comes from the behavior of ego-vehicle and other vehicles in the scene (eg. in CARLA 0.9.10, other vehicles take random turns at intersections which changes the observations of the ego-vehicle resulting in a different behavior). In the newer CARLA versions, the traffic manager has been improved so it should have less stochasticity.

The CARLA leaderboard is based on <IP_ADDRESS> so we use the same.

Such is the case. One more thing, I found that sometimes the "blocked NPC" will disappear. Is it a kind of mechanism of CARLA traffic manager? Thank you in advance for your reply.

That happens only when the agent is part of a spawned scenario. If the agent belongs to the background then it does not disappear.


The Mob Effect Magic Resistance protects only against indirect_magic damage sources

The effect Magic Resistance only protects against the damage sources of type "indirect_magic" and not "magic".
I believe this is due to this line of code:
"(source.isOf(DamageTypes.INDIRECT_MAGIC) || source.isOf(DamageTypes.INDIRECT_MAGIC)"
One of the conditions should probably be: "source.isOf(DamageTypes.MAGIC)"
By the way I love your mod.

Yeah that is definitely a typo, thanks for pointing it out!


sensor sabnzbd not loading 0.27.0-DEV

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version): 27.0 DEV
Python release (python3 --version):
Component/platform: sensor/sabnzbd
Description of problem:
Sensor  not loading
Expected:
Sensor to load
Error message
16-08-25 19:17:07 homeassistant.bootstrap: Invalid config for [sensor.sabnzbd]: required key not provided @ data['host']. Got None value is not allowed @ data['monitored_variables'][0]. Got OrderedDict([('type', 'current_status')]) value is not allowed @ data['monitored_variables'][1]. Got OrderedDict([('type', 'queue_remaining')])
YAML:
 - platform: sabnzbd
    name: SAB
    api_key: 1234566
    base_url: http://<IP_ADDRESS>XX:XXX/
    monitored_variables:
        - type: 'current_status'
        - type: 'queue_remaining'

Ah yes. This is a breaking change in 0.27 that we should write up. Thanks for notifying us.
Config should be:
 - platform: sabnzbd
    name: SAB
    api_key: 1234566
    base_url: http://<IP_ADDRESS>XX:XXX/
    monitored_variables:
        - current_status
        - queue_remaining

CC @fabaff

@balloob
Hmm,
I removed type and got rid of the two last lines
Still got
Invalid config for [sensor.sabnzbd]: required key not provided @ data['host']. Got None
though

Ah, looks like it also transitioned from a base url to specifying a host + port.


Weather API Application

This is a simple JS Weather application that can be easily expanded using other options available in the docs provided in the link in js file. This pull request is being made under Hacktoberfest 2021.

Looks like you are trying to add a html file under Javascript folder, it isnt allowed.
I request to create a separate folder like Weather Application under Javascript folder.

Ok, I'll do it!


how to get download progress when resume

Is it able to set setOnProgressListener when calling PRDownloader.resume(downloadId);?

KotlinCode:
val downloadId = PRDownloader.download(URL, dir, fileName)
.build().setOnStartOrResumeListener(object: OnStartOrResumeListener {
override fun onStartOrResume() {
Log.d(TAG, "onStartOrResume")
}
})
.setOnCancelListener(object: OnCancelListener {
override fun onCancel() {
Log.d(TAG, "onCancel")
}
})
.setOnProgressListener(object: OnProgressListener {
override fun onProgress(progress: Progress?) {
val percent = progress!!.currentBytes * 100 / progress.totalBytes
            }
        })

Kotlin code:
`
val downloadId = PRDownloader.download(URL, dir, fileName)
.build().setOnStartOrResumeListener(object: OnStartOrResumeListener {
override fun onStartOrResume() {
Log.d(TAG, "onStartOrResume")
}
})
.setOnCancelListener(object: OnCancelListener {
override fun onCancel() {
Log.d(TAG, "onCancel")
}
})
.setOnProgressListener(object: OnProgressListener {
override fun onProgress(progress: Progress?) {
val percent = progress!!.currentBytes * 100 / progress.totalBytes
            }
        })

`

val downloadId = PRDownloader.download(URL, dir, fileName)
            .build().setOnStartOrResumeListener(object: OnStartOrResumeListener {
                override fun onStartOrResume() {
                    Log.d(TAG, "onStartOrResume")
                }
            })
            .setOnCancelListener(object: OnCancelListener {
                override fun onCancel() {
                    Log.d(TAG, "onCancel")
                }
            })
            .setOnProgressListener(object: OnProgressListener {
                override fun onProgress(progress: Progress?) {
                    val percent = progress!!.currentBytes * 100 / progress.totalBytes

                }
            })

@xuanzhui
Java code:
downloadIdOne = PRDownloader.download(URL1, dirPath, "facebook.apk")
                        .build()
                        .setOnStartOrResumeListener(new OnStartOrResumeListener() {
                            @Override
                            public void onStartOrResume() {

                            }
                        })
                        .setOnPauseListener(new OnPauseListener() {
                            @Override
                            public void onPause() {

                            }
                        })
                        .setOnCancelListener(new OnCancelListener() {
                            @Override
                            public void onCancel() {

                            }
                        })
                        .setOnProgressListener(new OnProgressListener() {
                            @Override
                            public void onProgress(Progress progress) {

                            }
                        })
                        .start(new OnDownloadListener() {
                            @Override
                            public void onDownloadComplete() {

                            }

@Od1nik I would also like to know this, but I don't think your answer is the solution.
Let's say I start a download, and the app is killed. When the app is restarted, I want to resume the download (assuming I have the database enabled) , which I can do with PRDownloader.resume(id).
But how do I add a listener for that download? If I call PRDownloader.download, where I can add the listener, that will start a new download, will it not?

Ahh I see that the download ID is actually a hash made from the url / dirPath / fileName, so it will resume the download that's in the database.
So the answer is correct, starting a "new download" will resume the old one, if you use the same parameters. I assume, at least.

Not tested, thanks for your replies.

Ahh I see that the download ID is actually a hash made from the url / dirPath / fileName, so it will resume the download that's in the database.
So the answer is correct, starting a "new download" will resume the old one, if you use the same parameters. I assume, at least.

did it work?


Debian error

I get the follow error after I follow https://blog.khophi.co/use-letsencrypt-unifi-controller-ubuntu/
sudo ./unifi_ssl_import.sh
Starting UniFi Controller SSL Import...
Running in Let's Encrypt Mode...
Inspecting current SSL certificate...
Updated SSL certificate available. Proceeding with import...
Missing one or more required files. Check your settings.

celan install last debian version, up to date.
clean install unifi controler 5.5.20

No support here ?

I got the same error. How do I fix it?

Same error here, clean install of everything, domain name matches.
Any update?