Artificial borrowing constraint

If we want to impose a constraint on borrowing, say a >= 0, we use BoroCnstArt in the input dictionary.
Unless I am mistaken, does this not mean that https://github.com/econ-ark/HARK/blob/bf35222d3af40cdbd544f443fe071fc896744cc4/HARK/ConsumptionSaving/ConsIndShockModel.py#L796 should actually read
aNrmNow = np.asarray(self.aXtraGrid) + max(self.BoroCnstNat, self.BoroCnstArt)

?
This took me quite a long time to debug (longer than I'd like to admit, but I really wasn't looking here), because I thought I had set the borrowing constraint to >=0 in my portfolio model, but only quite late did I realize that my problems arose because assets were allowed to become negative, with... "interesting" solutions as a result :)
cc @mnwhite

This isn't actually a bug, but I can guess what might have gone wrong in
your model if you inherited from ConsIndShockSolver(Basic) and overwrote
some of the solution method in a new solver class.
The structure of the solver is to construct the
non-artificially-constrained consumption function everywhere that it
exists-- all the way down to mNrm = BoroCnstNat = aNrmMin.  We also
(trivially) construct the consumption function when the artificial
borrowing constraint binds everywhere that it exists-- all the way down
to mNrm = BoroCnstArt.  The true consumption function is the lower envelope
of these two functions-- literally LowerEnvelope1D(cFuncNowUnc,cFuncCnst).
This isn't strictly necessary; as you say, we could instead only compute
EndOfPrdvP at aNrm values beginning at the greater of the natural and
artificial borrowing constraints-- as long as the lowest aXtra value is
exactly zero.
It was programmed this way for three reasons (IIRC).  First, it's the way
Chris programmed the same model in Mathematica, and it was an iron rule
early on that the Python code had to replicate the Mathematica code
exactly.  Second, there is some in-model reason for doing it, but I can't
remember why now; maybe something to do with the value function
calculation?  I'll sleep on it.  Third, this same structure is used for the
models with two continuous state dimensions in which the natural borrowing
constraint varies with the second state (MNrm in ConsAggShockModel and pLvl
in ConsGenIncProcessModel).  The natural or artificial borrowing constraint
bind at different values of the second state variable, so the exact
location of the constraint kink between MNrm or pLvl gridpoints is more
accurate when you have the entire non-artificially-constrained consumption
function.
My guess is that your code grabs the consumption function "too early",
before it's had the LowerEnvelope operation applied, but I could be wrong.
On Tue, Apr 23, 2019 at 7:17 PM Patrick Kofod Mogensen <
<EMAIL_ADDRESS>wrote:

If we want to impose a constraint on borrowing, say a >= 0, we use
BoroCnstArt in the input dictionary.
Unless I am mistaken, does this not mean that
https://github.com/econ-ark/HARK/blob/bf35222d3af40cdbd544f443fe071fc896744cc4/HARK/ConsumptionSaving/ConsIndShockModel.py#L796
should actually read
aNrmNow = np.asarray(self.aXtraGrid) + max(self.BoroCnstNat, self.BoroCnstArt)
?
This took me quite a long time to debug (longer than I'd like to admit,
but I really wasn't looking here), because I thought I had set the
borrowing constraint to >=0 in my portfolio model, but only quite late did
I realize that my problems arose because assets were allowed to become
negative, with... "interesting" solutions as a result :)
cc @mnwhite https://github.com/mnwhite
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/253, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADKRAFKU5FV3J6D5W66SQLDPR6KJ3ANCNFSM4HH7DWFQ
.

Matt's explanation seems plausible.
If I recall correctly, a main reason for constructing consumption and value
functions all the way down to the natural borrowing constraint is that in a
multidimensional version of the problem you need to make a quick decision
about where the constraint binds and the max operator is very very
efficient (more so than trying to calculate the exact point for every
between-gridpoint interpolated situation).
Not completely sure this is the right story, but it's my best guess.  I've
done a lot of experiments that are not in the Mathematica or python code,
and sometimes multidimensional versions of things make you realize that you
want to change the way you do the two-dimensional version so that it will
generalize nicely to more dimensions.
On Tue, Apr 23, 2019 at 10:12 PM Matthew N. White<EMAIL_ADDRESS>wrote:

This isn't actually a bug, but I can guess what might have gone wrong in
your model if you inherited from ConsIndShockSolver(Basic) and overwrote
some of the solution method in a new solver class.
The structure of the solver is to construct the
non-artificially-constrained consumption function everywhere that it
exists-- all the way down to mNrm = BoroCnstNat = aNrmMin. We also
(trivially) construct the consumption function when the artificial
borrowing constraint binds everywhere that it exists-- all the way down
to mNrm = BoroCnstArt. The true consumption function is the lower envelope
of these two functions-- literally LowerEnvelope1D(cFuncNowUnc,cFuncCnst).
This isn't strictly necessary; as you say, we could instead only compute
EndOfPrdvP at aNrm values beginning at the greater of the natural and
artificial borrowing constraints-- as long as the lowest aXtra value is
exactly zero.
It was programmed this way for three reasons (IIRC). First, it's the way
Chris programmed the same model in Mathematica, and it was an iron rule
early on that the Python code had to replicate the Mathematica code
exactly. Second, there is some in-model reason for doing it, but I can't
remember why now; maybe something to do with the value function
calculation? I'll sleep on it. Third, this same structure is used for the
models with two continuous state dimensions in which the natural borrowing
constraint varies with the second state (MNrm in ConsAggShockModel and pLvl
in ConsGenIncProcessModel). The natural or artificial borrowing constraint
bind at different values of the second state variable, so the exact
location of the constraint kink between MNrm or pLvl gridpoints is more
accurate when you have the entire non-artificially-constrained consumption
function.
My guess is that your code grabs the consumption function "too early",
before it's had the LowerEnvelope operation applied, but I could be wrong.
On Tue, Apr 23, 2019 at 7:17 PM Patrick Kofod Mogensen <
<EMAIL_ADDRESS>wrote:

If we want to impose a constraint on borrowing, say a >= 0, we use
BoroCnstArt in the input dictionary.
Unless I am mistaken, does this not mean that

https://github.com/econ-ark/HARK/blob/bf35222d3af40cdbd544f443fe071fc896744cc4/HARK/ConsumptionSaving/ConsIndShockModel.py#L796

should actually read
aNrmNow = np.asarray(self.aXtraGrid) + max(self.BoroCnstNat,
self.BoroCnstArt)
?
This took me quite a long time to debug (longer than I'd like to admit,
but I really wasn't looking here), because I thought I had set the
borrowing constraint to >=0 in my portfolio model, but only quite late
did
I realize that my problems arose because assets were allowed to become
negative, with... "interesting" solutions as a result :)
cc @mnwhite https://github.com/mnwhite
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/253, or mute the thread
<
https://github.com/notifications/unsubscribe-auth/ADKRAFKU5FV3J6D5W66SQLDPR6KJ3ANCNFSM4HH7DWFQ
.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/253#issuecomment-486040105, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAKCK7ZDPCSGEHQYNHLZ2KTPR662DANCNFSM4HH7DWFQ
.

--

Chris Carroll

Okay, so maybe the problem is with the assumptions in the code I'm inheriting that I'm somehow missing. I do realize the lower-envelope trick, and I appreciated the idea as soon as I saw it, quite elegant compared to padding values of a vector.
My problem is/was that  I thought that I could just grab aNrmNow and solve the portfolio problem on all a in aNrmNow. However, I made the mistake of thinking that aNrmNow respected the artificial borrowing constraint - it does not. It's fine if there's a design reason for this (say, because aNrmNow is used to construct a consumption function from BoroConsNat all the way up ), it's just that some of these things are not clear to the people who did not invent the code :)
I tried to add a comment above the line where aNrmNow is constructed at https://github.com/econ-ark/HARK/pull/254 . Feel free to corrrect it if what I wrote isn't exactly correct.

Thanks.  I looked at the comment and it seems fine.  Oddly, however, Travis
is saying that it has not completed its tests of the code -- 8 hours after
you posted it, and for something that is entirely a comment.
Do you know what's gone wrong with Travis?
On Wed, Apr 24, 2019 at 3:51 AM Patrick Kofod Mogensen <
<EMAIL_ADDRESS>wrote:

Okay, so maybe the problem is with the assumptions in the code I'm
inheriting that I'm somehow missing. I do realize the lower-envelope trick,
and I appreciated the idea as soon as I saw it, quite elegant compared to
padding values of a vector.
My problem is/was that I thought that I could just grab aNrmNow and
solve the portfolio problem on all a in aNrmNow. However, I made the
mistake of thinking that aNrmNow respected the artificial borrowing
constraint - it does not. It's fine if there's a design reason for this
(say, because aNrmNow is used to construct a consumption function from
BoroConsNat all the way up ), it's just that some of these things are not
clear to the people who did not invent the code :)
I tried to add a comment above the line where aNrmNow is constructed at
#254 https://github.com/econ-ark/HARK/pull/254 . Feel free to corrrect
it if what I wrote isn't exactly correct.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/253#issuecomment-486110525, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAKCK777UBD7CJPDDMVWO5LPSAGQPANCNFSM4HH7DWFQ
.

--

Chris Carroll

Do you know what's gone wrong with Travis?

On my computer the status looks fine?

Hm, but to return to the topic at hand, I think I may have to see an example where this is useful. If the user sets the artificial borrowing constraint and the artificial borrowing constraint is "binding" in the sense that it's > the natural borrowing constraint then how can anything below the artificial borrowing constraint ever be relevant?

That's the missing reason #2 from my post.  I'll think on it.  But #1 and
#3 were the dominant reasons for Chris and me respectively.  I was on a big
"as above, so below" kick.
BUT: Travis is definitely malfunctioning.  It's now reporting every
single PR as waiting for its test to be run on the branch, for hours or
days on end... even for PRs that have been outstanding (with passed checks)
for a while.
On Wed, Apr 24, 2019 at 3:27 PM Patrick Kofod Mogensen <
<EMAIL_ADDRESS>wrote:

Hm, but to return to the topic at hand, I think I may have to see an
example where this is useful. If the user sets the artificial borrowing
constraint and the artificial borrowing constraint is "binding" in the
sense that it's > the natural borrowing constraint then how can anything
below the artificial borrowing constraint ever be relevant?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/253#issuecomment-486392283, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADKRAFN53TN26VHBKZXI2BTPSCYAPANCNFSM4HH7DWFQ
.

BUT: Travis is definitely malfunctioning. It's now reporting every single PR as waiting for its test to be run on the branch, for hours or days on end... even for PRs that have been outstanding (with passed checks) for a while.

Can you elaborate on this? This ran fine for exampe https://github.com/econ-ark/HARK/pull/256

Shauna fixed it in the portion of the call before you joined.
On Thu, Apr 25, 2019 at 2:58 PM Patrick Kofod Mogensen <
<EMAIL_ADDRESS>wrote:

BUT: Travis is definitely malfunctioning. It's now reporting every
single PR as waiting for its test to be run on the branch, for hours or
days on end... even for PRs that have been outstanding (with passed checks)
for a while.
Can you elaborate on this? This ran fine for exampe #256
https://github.com/econ-ark/HARK/pull/256
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/253#issuecomment-486797420, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADKRAFL4WGKLYUJLPFYEDHTPSH5N3ANCNFSM4HH7DWFQ
.

Shauna fixed it in the portion of the call before you joined.

Great!


[10.x] value is null by default

I am closing this pull request because it lacks sufficient explanation, tests, or both. It is difficult for us to merge pull requests without these things because the change may introduce breaking changes to the framework.
Feel free to re-submit your change with a thorough explanation of the feature and tests - integration tests are preferred over unit tests. Please include it's benefit to end users; the reasons it does not break any existing features; how it makes building web applications easier, etc.
Thanks!

A default value in the getOrPut method doesn't have much sense (as same for put method).


[1192] Remove hint

Context
https://trello.com/c/L0UlVCKD/1192-remove-pointless-hint
Changes proposed in this pull request
Removes hint and adds a better page title.
Guidance to review
I couldn't just simply remove the hint markup as that would create a smaller gap between the h1 and the date field than on the prototype. Not sure if this is the best solution though.

 Remove pointless Hint

@AbigailMcP that's the edit page for the draft. The edit page for existing records is below. Emma and I removed the hint as it's likely (required for now) that you'll be loading the page with a date already present.

@AbigailMcP that's the edit page for the draft. The edit page for existing records is below. Emma and I removed the hint as it's likely (required for now) that you'll be loading the page with a date already present.

Cool! Understood 👍

 Remove pointless Hint

 Remove pointless Hint


nodejs support

Hello,
Is there any support for node.js in the current release. If not is it in the project roadmap?
Cheers,
Subho.

Dear Subho,
Thanks for your question. I just tried it in node. You are correct, it does not currently work in node. If I try numeric.clone([1,2,3]), it complains about how there's no object numeric.
From my previous (limited) experience with node, I suspect that the problem is as follows. Node uses v8 contexts a lot. Each module is loaded into its own javascript context. See http://stackoverflow.com/questions/5875193/node-javascript-contexts-sharing-built-in-prototypes for some examples of this.
As a result, the dynamically generated function numeric.clone (as well as many other dynamically generated functions) are having a hard time locating the numeric functions.
Not quite sure how to fix this yet. If you have a patch, I'll accept it.

I have confirmed this is a known issue/feature of node.js. On the irc channel for node, user insin related a similar problem in their project. The solution they found is:
 ZedPepsi: I ended up defining everything I needed from the containing scope as an argument and passing it in. Not sure if there are better workarounds
That particular solution sounds like a major rewrite. Not sure where to go from here.


String resource ID #0x0

Code:
import React, { useRef, useEffect } from 'react';
import UnityView from '@azesmway/react-native-unity';
import { View, Button, Text } from 'react-native';

interface IMessage {
    gameObject: string;
    methodName: string;
    message: string;
  }

const Unity = ({ navigation }) => {
  console.log('Open Unity')
  const unityRef = useRef(null);

    useEffect(() => {
      if (unityRef.current) {
        const message: IMessage = {
          gameObject: 'gameObject',
          methodName: 'methodName',
          message: 'message',
        };
        unityRef.current.postMessage(message.gameObject, message.methodName, message.message);
      }
    }, []);

    return (
      <View style={{ flex: 1 }}>
        <Button title='Go back Home' onPress={() => navigation.navigate('Home')} />
        <UnityView
          ref={unityRef}
          style={{ flex: 1 }}
          onUnityMessage = {(result) => { console.log('onUnityMessage', result.nativeEvent.message) }}
        />
      </View>
    );

};

export default Unity;

Seems like my project has problem with unityRef/UnityView.
without
<UnityView           ref={unityRef}           style={{ flex: 1 }}           onUnityMessage = {(result) => { console.log('onUnityMessage', result.nativeEvent.message) }} />
project seems to work fine.

Any idea how to fix it?

@BedkowskiP What did you do to fix it? Getting the same error

This fix can be found in step 5 of the README instructions for setting up android. No sweat for missing it, I missed it for quite some time as we;;


fixed Cantera convenience function outputs

[x] Fixes #72
[x] Tests updated
[x] Added entry into CHANGELOG.md

Changes proposed in this pull request:

Fixes output of Cantera convenience mole/mass fraction functions to match what Cantera actually can read

@pr-omethe-us/chemked

Alright, this is relatively minor and tests pass, so I'm gonna merge


Replace the text Admin UI with Admin Portal

What does this PR do?
Replace text Admin UI with Admin Portal

Need to update here also:
https://github.com/boxyhq/jackson/blob/bb8e4b950b25c5c033555d830f3142dd619190c6/components/Sidebar.tsx#L92


What's the state of this project?

Hi,
after reading about pyjs, i am willing to use it in a production environment. however i noticed it's been a while pyjs has been updated.
what's the current status of the project? is it still usable? is there any recommendation in terms of using it or suggested library?

Please see issue #847.


setInterval() using adjusted duration indefinitely

Description
The timer API implementations want to adjust the initial delay to take into account the time it took for the registration call to arrive from JS to the native side.
It seems that on Android, in the case of a repeating timer (setInterval), the adjusted duration is used for all repeated ticks instead of just the initial one:
https://github.com/facebook/react-native/blob/657c48c21de496734716ed33c9e55d8baae3494f/ReactAndroid/src/main/java/com/facebook/react/modules/core/JavaTimerManager.java#L345
The effect of this is that if during the time when setInterval is invoked, something is happening in the process that's slowing down the calls to the native timer implementation, the required delay adjustment will be significant, and will cause all subsequent interval ticks to fire too quickly.
The change seems to have been introduced in f054928124a2308b2bd450acf8bb754a92ccb16a.
Version
0.65.1
Output of react-native info
System:
OS: Windows 10 10.0.19042
CPU: (8) x64 Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
Memory: 4.83 GB / 15.76 GB
Binaries:
Node: 14.18.0 - C:\Program Files\nodejs\node.EXE
Yarn: Not Found
npm: 6.14.15 - C:\Program Files\nodejs\npm.CMD
Watchman: Not Found
SDKs:
Android SDK: Not Found
Windows SDK: Not Found
IDEs:
Android Studio: Version     2<IP_ADDRESS> AI-203.7717.56.2031.7678000
Visual Studio: Not Found
Languages:
Java: 11.0.12 - C:\Program Files\Amazon Corretto\jdk11.0.12_7\bin\javac.EXE
npmPackages:
@react-native-community/cli: 6.0.0 => 6.0.0
react: 17.0.2 => 17.0.2
react-native: 0.65.1 => 0.65.1
react-native-windows: 0.65.1 => 0.65.1
npmGlobalPackages:
react-native: Not Found
Steps to reproduce
Invoke setInterval() with a delay of 1000 while the process is very busy doing other things.
Expected: The given interval function to be invoked every 1000ms.
Actual: The given interval function is invoked much more quickly.
Snack, code example, screenshot, or link to a repository
No response

This issue is still valid. (Commenting to keep the issue from being closed as stale.)

@cortinico Please see this issue. It is still valid and pretty visible in debug mode. This is causing a major problem in our app when upgrading to newer react native version with newArch turned off.

@cortinico Please see this issue. It is still valid and pretty visible in debug mode. This is causing a major problem in our app when upgrading to newer react native version with newArch turned off.

This is a 3 years old issue. If the issue is still happening, please open a new issue against 0.74 + provide a small repro


fix: eww passes x11 and wayland features down to yuck

Please follow this template, if applicable.
Description

Fixes #305. Allows to cargo check specifically only eww package.
Additional Notes
There was a PR #289 to solve #305, but it was ignored for lack of the issue it fixes.
Checklist
Please make sure you can check all the boxes that apply to this PR.

[x] All widgets I've added are correctly documented.
[x] The documentation in the docs/content/main directory has been adjusted to reflect my changes.
[x] I used cargo fmt to automatically format all code before committing

Looks good, thanks! Sorry for not reviewing the other PR eariler, I've been pretty busy with other stuff lately

The actual compile error seems to have been caused by a mistake I made a few days ago, accidentally referencing an x11 only feature in the use statements of app.rs. this has been fixed by https://github.com/elkowar/eww/commit/5b52cd927acd0b10c69c0b8da5cabfe855941361. Explicitly passing down the feature still seems like a good idea, as this is now supposed to actually work since resolver 2 is out (so pretty much inore my last comment, lol)


Use new args each time in eliminate_contiguous

Eliminate Contiguous tries to replace each argument in a loop.
Therefore need to create copy of new args each time inside the loop as they may change while looping.
I am not able to create a test as all the "Ref" ops allow Non-standard shapes.
But I have a case like following with the CPU backend which doesn't allow non-std shape with Binary Ops.
at = transpose(a)
ac = contiguous(at)
br = reshape_lazy(b)
bc = contiguous(br)
add = binary_add(ac, bc)

Current logic doesn't remove contiguous(br). Reason is it first creates new_args = {ac, bc} then it modifies them to be {at, bc} when it tries to remove contiguous(at). It fails but then it doesn't revert new_args back to {ac, bc} it keeps them at {at, bc} and because of that when it tries to remove contiguous(br) it fails. Therefore contigous(br) is not eliminated.

So it was written this way so we check if removing all contiguouses will work, where each individually dotn work.It can be the case that an operator(like for miopen's convolution) will fail if one is contiguous and the other is nhwc, but if both are nhwc it will work fine. We should probably add a unit test for that.

But I have a case like following with the CPU backend which doesn't allow non-std shape with Binary Ops.

You can add a simple operator that requires standard shape similar to the pass_standard_op that is used in the current unit tests.

You can add a simple operator that requires standard shape similar to the pass_standard_op that is used in the current unit tests.

Added a test.


Revert "[IOTDB-6353] replace cglib to byte-buddy (#14426)"

Performance degradation was observed

Should we try some newer versions before we do this revert?
https://github.com/raphw/byte-buddy/releases

ok, the latest version of byte-buddy still has the problem.


Implement Strikethru and remove underline conflict

In this PR, I implemented strike-through just as how underline is implemented. I also minified the pass or +++SOMEHTML+++ area of HTML-expressed formats like underline and strike-through so it doesn't mess up with others.

Hmm, conflicts..

Rebased.

Apart from a few cosmetic issues, this look good to me.
Thanks for your great work, I will try to publish an updated version this week-end.

All good :+1:


phpspec/prophecy is now explicitely needed for tests

Bug Report
Since PHPUnit dropped phpspec/prophecy from its dependencies (https://github.com/sebastianbergmann/phpunit/issues/5033), the following errors are triggered when running the testsuite with latest PHPUnit.
There were 2 errors:

1) Slim\Tests\Http\RequestTest::testGetRequestTargetWithSlimPsr7Uri
PHPUnit\Framework\Exception: This test uses TestCase::prophesize(), but phpspec/prophecy is not installed. Please run "composer require --dev phpspec/prophecy".

/tmp/autopkgtest-lxc._6y23c80/downtmp/build.jLX/src/tests/Http/RequestTest.php:458
/usr/bin/phpunit:73

2) Slim\Tests\Http\RequestTest::testGetRequestTargetWithNonSlimPsr7Uri
PHPUnit\Framework\Exception: This test uses TestCase::prophesize(), but phpspec/prophecy is not installed. Please run "composer require --dev phpspec/prophecy".

/tmp/autopkgtest-lxc._6y23c80/downtmp/build.jLX/src/tests/Http/RequestTest.php:477
/usr/bin/phpunit:73

Regards
David

Have you tried to install phpspec/prophecy?
composer require phpspec/prophecy --dev

I guess the problem could be solved by adding phpspec/prophecy as a dev dependency to the project.

@odan we will need to add the phpspec/prophecy dev dependency to the Slim 4.x branch. I will PR this soon

The dependency is in our require-dev:
https://github.com/slimphp/Slim/blob/4.x/composer.json#L65
The test suite runs fine.


feat: Add PointCloudLayout for point-cloud components

UI updated~

:tada: This PR is included in version 1.0.0 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:


feat(fw): make NON_RETURNING_SECTION default in Container

🗒 Description
Having non-returning code sections in EOF is much more common (at least in tests) and because we rely on the default values the NON_RETURNING_SECTION is better default.
I only update container.py for visualization and fixed JUMPF failing tests (including one typo). New default can be applied to other tests later.
✅ Checklist

[ ] All: Set appropriate labels for the changes.
[ ] All: Considered squashing commits to improve commit history.
[ ] All: Added an entry to CHANGELOG.md.
[ ] All: Considered updating the online docs in the ./docs/ directory.
[ ] Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
[ ] Tests: Included the type and version of evm t8n tool used to locally execute test cases:  e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
[ ] Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

I'm fixing the tests in src/ethereum_test_tools/tests/ and merge afterwards.

Rebased and removed  the defaults from most containers.
I did a 1-to-1 comparison before and after and the only changes are some invalid containers in tests/prague/eip7692_eof_v1/eip3540_eof_v1/container.py where it makes sense to update the code outputs from zero to the new default 0x80.


Problems with sparse

I've been trying to get this to work with sparse matrices.
The setup:
>>> import pynndescent
>>> import scipy.sparse as sp
>>> x = sp.random(1000, 1000, density=0.01)

Next, I try to construct the index; this raises an error:
>>> nn = pynndescent.NNDescent(x)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-10-63ef2a9d7c43> in <module>
----> 1 pynndescent.NNDescent(x)

~/miniconda3/envs/tsne/lib/python3.7/site-packages/pynndescent/pynndescent_.py in __init__(self, data, metric, metric_kwds, n_neighbors, n_trees, leaf_size, pruning_level, tree_init, random_state, algorithm, max_candidates, n_iters, delta, rho, n_jobs, seed_per_row, verbose)
    565                     )
    566                 metric_nn_descent = sparse.make_sparse_nn_descent(
--> 567                     distance_func, tuple(metric_kwds.values())
    568                 )
    569                 if verbose:

AttributeError: 'NoneType' object has no attribute 'values'

Okay, so metric_kwds must be specified. Very unexpected, but ok, I can fix that.
>>> nn = pynndescent.NNDescent(x, metric_kwds={})  # works!

Great, so I've got the index. Now I want to query it:
>>> nn.query(x)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-11b3373648c0> in <module>
----> 1 nn.query(x)

~/miniconda3/envs/tsne/lib/python3.7/site-packages/pynndescent/pynndescent_.py in query(self, query_data, k, queue_size)
    697         """
    698         # query_data = check_array(query_data, dtype=np.float64, order='C')
--> 699         query_data = np.asarray(query_data).astype(np.float32)
    700         self._init_search_graph()
    701         init = initialise_search(

TypeError: float() argument must be a string or a number, not 'coo_matrix'

Again, weird that I can build an index for a sparse matrix, but not query with it. Ok, I convert it to a dense matrix:
>>> nn.query(x.toarray())
---------------------------------------------------------------------------
TypingError                               Traceback (most recent call last)
<ipython-input-15-c4301d4d66b3> in <module>
----> 1 nn.query(x.toarray())

~/miniconda3/envs/tsne/lib/python3.7/site-packages/pynndescent/pynndescent_.py in query(self, query_data, k, queue_size)
    706             self._random_init,
    707             self._tree_init,
--> 708             self.rng_state,
    709         )
    710         result = self._search(

~/miniconda3/envs/tsne/lib/python3.7/site-packages/pynndescent/pynndescent_.py in initialise_search(forest, data, query_points, n_neighbors, init_from_random, init_from_tree, rng_state)
     73 ):
     74     results = make_heap(query_points.shape[0], n_neighbors)
---> 75     init_from_random(n_neighbors, data, query_points, results, rng_state)
     76     if forest is not None:
     77         for tree in forest:

~/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/dispatcher.py in _compile_for_args(self, *args, **kws)
    348                 e.patch_message(msg)
    349
--> 350             error_rewrite(e, 'typing')
    351         except errors.UnsupportedError as e:
    352             # Something unsupported is present in the user code, add help info

~/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/dispatcher.py in error_rewrite(e, issue_type)
    315                 raise e
    316             else:
--> 317                 reraise(type(e), e, None)
    318
    319         argtypes = []

~/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/six.py in reraise(tp, value, tb)
    656             value = tp()
    657         if value.__traceback__ is not tb:
--> 658             raise value.with_traceback(tb)
    659         raise value
    660

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Internal error at <numba.typeinfer.ArgConstraint object at 0x7f0cf1997c18>:
--%<----------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pavlin/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/errors.py", line 627, in new_error_context
    yield
  File "/home/pavlin/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/typeinfer.py", line 201, in __call__
    assert ty.is_precise()
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pavlin/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/typeinfer.py", line 144, in propagate
    constraint(typeinfer)
  File "/home/pavlin/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/typeinfer.py", line 202, in __call__
    typeinfer.add_type(self.dst, ty, loc=self.loc)
  File "/home/pavlin/miniconda3/envs/tsne/lib/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/pavlin/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/errors.py", line 635, in new_error_context
    six.reraise(type(newerr), newerr, tb)
  File "/home/pavlin/miniconda3/envs/tsne/lib/python3.7/site-packages/numba-0.43.1-py3.7-linux-x86_64.egg/numba/six.py", line 659, in reraise
    raise value
numba.errors.InternalError:
[1] During: typing of argument at /home/pavlin/miniconda3/envs/tsne/lib/python3.7/site-packages/pynndescent/pynndescent_.py (39)
--%<----------------------------------------------------------------------------

File "../../miniconda3/envs/tsne/lib/python3.7/site-packages/pynndescent/pynndescent_.py", line 39:
    def init_from_random(n_neighbors, data, query_points, heap, rng_state):
        for i in range(query_points.shape[0]):
        ^

This error may have been caused by the following argument(s):
- argument 1: cannot determine Numba type of <class 'scipy.sparse.csr.csr_matrix'>

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

I noticed something to do with csr_matrices in that error, so maybe COO format is not supported?
>>> nn.query(x.tocsr())
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-16-77873307e81a> in <module>
----> 1 nn.query(x.tocsr())

~/miniconda3/envs/tsne/lib/python3.7/site-packages/pynndescent/pynndescent_.py in query(self, query_data, k, queue_size)
    697         """
    698         # query_data = check_array(query_data, dtype=np.float64, order='C')
--> 699         query_data = np.asarray(query_data).astype(np.float32)
    700         self._init_search_graph()
    701         init = initialise_search(

ValueError: setting an array element with a sequence.

Same error as with the COO matrix.
A little bit of environment info:
Python 3.7.3
---------------------------
numba==0.43.1
scipy==1.2.0
pynndescent==0.3.0

Hmm, definitely still some issues -- things got brought over from umap, but apparently there are some issues. I don't have time to dig into this right now, but hopefully in a week or two I will have time.

Thanks! Let me know if I've missed out any useful information you might need.

I finally got some time and dug into this. The short answer was that not everything got fully ported over from umap, and there were some issues with sparse support in threaded mode (is as much as that wasn't yet implemented). I think I've got that all fixed now. If you can try cloning the current master and let me know if that resolves the issue for you I would appreciate it.

Thanks so much for looking into this! Both of the problems I mentioned seem to be fixed; I can use it on sparse just like I would expect.
Unfortunately, building and querying the index now raises several numba warnings, even when I run it on dense data e.g.
/home/pavlin/dev/pynndescent/pynndescent/sparse_nndescent.py:121: NumbaPerformanceWarning:
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "pynndescent/utils.py", line 459:
@numba.njit(parallel=True)
def new_build_candidates(
^

  False,
/home/pavlin/dev/pynndescent/pynndescent/sparse_nndescent.py:82: NumbaPerformanceWarning:
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "pynndescent/utils.py", line 79:
@numba.njit(parallel=True)
def rejection_sample(n_samples, pool_size, rng_state):
^

  indices = rejection_sample(n_neighbors, n_vertices, rng_state)

when building the index and
/home/pavlin/dev/pynndescent/pynndescent/sparse_nndescent.py:273: NumbaPerformanceWarning:
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "pynndescent/sparse_nndescent.py", line 175:
@numba.njit(parallel=True)
def sparse_init_from_random(
^

  dist_args,
/home/pavlin/dev/pynndescent/pynndescent/sparse_nndescent.py:288: NumbaPerformanceWarning:
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "pynndescent/sparse_nndescent.py", line 207:
@numba.njit(parallel=True)
def sparse_init_from_tree(
^

  dist_args,
/home/pavlin/miniconda3/envs/tmp/lib/python3.7/site-packages/numba-0.44.0-py3.7-linux-x86_64.egg/numba/ir_utils.py:1958: NumbaPendingDeprecationWarning:
Encountered the use of a type that is scheduled for deprecation: type 'reflected list' found for argument 'forest' of function 'sparse_initialise_search'.

For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-reflection-for-list-and-set-types

File "pynndescent/sparse_nndescent.py", line 248:
@numba.njit()
def sparse_initialise_search(
^

  warnings.warn(NumbaPendingDeprecationWarning(msg, loc=loc))

when querying the index.

It looks like the warnings have started appearing from Numba 0.44. I've opened #68 to remove the ones related to parallel=True. There are two more to do with 'reflective list' being deprecated, but those can't be easily fixed until the replacement appears in Numba 0.45. In the meantime the warnings can be ignored, or Numba 0.43 (or earlier) can be specified explicitly when installing.

Thanks for #68 @tomwhite, that shoudl take care of a lot of them. This is popping up in umap as well, so I will have to take care of it there soon.

There are two more to do with 'reflective list' being deprecated, but those can't be easily fixed until the replacement appears in Numba 0.45.

@tomwhite I haven't seen those warnings myself, but I think Numba 0.45 is out already. Do you know what's the status of those warnings?


issue with walker slayer tower

Type
Incorrect behavior
Operating System
MacOS
OS Version
No response
Bug description
slayer tower doesnt seem  to work with staircases add in the transport
new Transport().addObstacle(new WorldPoint(3435, 3537, 0), new WorldPoint(3433, 3537, 1), "Climb-up").build();
    new Transport().addObstacle(new WorldPoint(3413, 3540, 1), new WorldPoint(3417, 3540, 2), "Climb-up").build();

then use the walker
Microbot.getWalker().walkTo(new WorldPoint(3431,3541,2));
Screenshots or videos
No response
RuneLite version
latest mocroware
Logs
No response

also gnomestronghold cave doesnt to be working
new Transport().addObstacle(new WorldPoint(2430, 9824, 0), new WorldPoint(2430, 3424, 0), "Enter").addReverse("Enter").build(); // gnome stronghold slayer cave
public static boolean walkToBankByLocation(BankLocation bankLocation, boolean useTransport) {
Microbot.status = "Going to bank: " + bankLocation.getName();
Microbot.getWalker().walkTo(bankLocation.getWorldPoint(), useTransport);
if (bankLocation.getWorldPoint().distanceTo(Microbot.getClient().getLocalPlayer().getWorldLocation()) > 4) {
Microbot.getWalker().walkTo(bankLocation.getWorldPoint(), useTransport);
return false;
}
return true;
}
Rs2Bank.walkToBankByLocation(BankLocation.TREE_GNOME_STRONGHOLD_NIEVE, true);

Should be fixed in development branch, can you verifiy this @ikiddoi ?


Linked list find refactor

Codecov Report

Merging #3 into master will increase coverage by 12.33%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           master       #3       +/-   ##
===========================================
+ Coverage   71.25%   83.58%   +12.33%
===========================================
  Files         263      345       +82
  Lines       29327    41953    +12626
===========================================
+ Hits        20897    35068    +14171
+ Misses       8430     6885     -1545

Impacted Files
Coverage Δ

src/core/common/linked_list.hpp
100.00% <100.00%> (+4.54%)
:arrow_up:

src/core/utils/child_supervision.hpp
60.00% <0.00%> (-40.00%)
:arrow_down:

examples/apps/ncp/main.c
48.14% <0.00%> (-33.11%)
:arrow_down:

src/core/coap/coap_secure.hpp
87.50% <0.00%> (-12.50%)
:arrow_down:

src/lib/spinel/spinel_decoder.hpp
52.63% <0.00%> (-9.87%)
:arrow_down:

src/core/crypto/mbedtls.cpp
54.83% <0.00%> (-9.17%)
:arrow_down:

src/core/coap/coap_message.cpp
54.07% <0.00%> (-9.12%)
:arrow_down:

src/core/thread/neighbor_table.cpp
87.93% <0.00%> (-8.26%)
:arrow_down:

src/core/coap/coap.hpp
95.00% <0.00%> (-5.00%)
:arrow_down:

src/posix/platform/system.cpp
96.61% <0.00%> (-3.39%)
:arrow_down:

... and 244 more


Refine the MET Dockerfile logic to make smaller images.

As of met-9.0, scripts/docker/Dockerfile in the MET repo creates images that are quite large... about 1GB/3GB compressed/uncompressed.  This task is to update the Dockerfile to delete unnecessary files to create smaller images.
The Dockerfile downloads and compiles: BUFRLIB, NetCDF-C, NetCDF-CXX, HDF4, HDFEOS, and MET.  There is no reason to retain the tarballs and source code for the dependent libraries.  Recommend that they always be deleted.
Also recommend adding a Dockerfile argument to indicate whether the MET source code should be retained.  By default, it should not.  Then reconfigure the builds on DockerHub to retain the MET source code only for the develop branch.  Could also consider keeping it for the master_v* branches as well, but that probably isn't necessary.

Updated the Dockerfile in the develop branch to always delete the tarballs and source code for the dependent libraries after installing them.  That reduces the resulting image size from 3.0 Gb down to 2.8 Gb.  Also, added logic to automatically remove the MET source code when building released versions, beginning with "met-".  That reduces the image size down to 1.6 Gb.  For branches (e.g. develop and master_v8.1) do not remove the source code since these Docker images could be used for s/w development.


Playwrite with typescript not working

Describe the bug
Basic sveltekit app with typescript and palywrite enabled is not working for me.
Reproduction
Setting up a basic sveltekit app:
npm init svelte@next my-app
cd my-app
npm install
npm run dev -- --open

With all options enabled, I.e. typescript and playwrite enabled causes the tests not to work.
Logs
$ npm run test

><EMAIL_ADDRESS>test
> playwright test

Running 0 test using 0 worker

Error: tests\test.ts: Cannot import a typescript file from an esmodule.

=================
 no tests found.
=================

System Info
I am on windows machine, I am not sure if this issues happens on other platforms but the error does not seem platform dependent

$ npx envinfo --system --binaries --browsers --npmPackages "{svelte,@sveltejs/*,vite}"
Need to install the following packages:
  envinfo
Ok to proceed? (y) y

  System:
    OS: Windows 10 10.0.19041
    CPU: (12) x64 Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz
    Memory: 17.81 GB / 31.71 GB
  Binaries:
    Node: 16.14.2 - C:\Program Files\nodejs\node.EXE
    npm: 8.5.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Spartan (44.19041.1266.0), Chromium (99.0.1150.55)
    Internet Explorer: 11.0.19041.1202
  npmPackages:
    @sveltejs/adapter-auto: next => 1.0.0-next.34
    @sveltejs/kit: next => 1.0.0-next.304
    svelte: ^3.46.0 => 3.46.6

Severity
serious, but I can work around it
Additional Information
To work around this, and probably the way to solve the issue has 2 steps:
playwrite has an alpha release that fixes this issue, see: https://github.com/microsoft/playwright/issues/11755#issuecomment-1082544994
Steps for me were:
npm up @playwright/test@next
npm exec playwright install

And then there is also an issue where the default playwrite file playwright.config.js is made in javascript, and needs to be build as a typescript file if typescript is enabled  playwright.config.ts. So my workaround is just to change the extension of the filename to .ts

I'm having the exact same issue. Switching to 1.20.0 of Playwright also circumvents this.

Closed by #4479, as @Rich-Harris updated PR to use playwright @next version if using typescript.
Tracking the @playwright/test@next removal in #4488


Fix mixing colors

Introduce a new flag to switch between different modes
Implement linear mixing
Change blending to use hue and chroma

The colors are WAY too saturated - I'm not sure what's going on :/
Theory is that it's because it's not converting to linear in between everything??


can't load package: package main

What version of Go are you using (go version)?
$ go version go1.14.6 darwin/amd64

Does this issue reproduce with the latest release?
Code
package main

/*
#include <stdio.h>

int getNumber() {
    return 1;
}
*/
import "C"

import (
	"syscall/js"
)

func main() {
	num := C.getNumber()
	console := js.Global().Get("console")
	console.Get("log").Invoke(1)
}

Build
GOOS=js GOARCH=wasm go build -o static/main.wasm main.go

Error
can't load package: package main: build constraints exclude all Go files in /Users/jesse/workspace/hello-vc/test09

What operating system and processor architecture are you using (go env)?
Switched to branch 'dev/1.4.0'
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jesse/Library/Caches/go-build"
GOENV="/Users/jesse/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jesse/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/dq/05143lls7wl6r50lhh9p52y80000gp/T/go-build545878115=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
I was going to compile the go file to a wasm file, but it showed error.
What did you expect to see?
the go file will be compiled to a wasm file
What did you see instead?
can't load package: package main: build constraints exclude all Go files in /Users/jesse/workspace/hello-vc/test09

js/wasm doesn't support cgo. You cannot build a cgo program for wasm.

Note that files that import "C" have an implicit cog build constraint.
That said, the compiler's error message could be better when compiling with CGO_ENABLED=1 for js/wasm.
example.com$ GOOS=js GOARCH=wasm go build
package example.com: build constraints exclude all Go files in /tmp/tmp.MKI8rv27eJ/example.com

example.com$ GOOS=js GOARCH=wasm go env CGO_ENABLED
0

example.com$ GOOS=js GOARCH=wasm CGO_ENABLED=1 go build
# runtime/cgo
unknown ptrSize for $GOARCH "wasm"

Duplicate of #24068


Time Slider No Longer Displays Cycle or Time

New since VisIt 3.3.3. I have data on the CA I am using to reproduce.

Here is the smoking gun:
https://github.com/visit-dav/visit/commit/0639b8d57f94908a0e3cb2d42355c81f3b5ec34d#diff-21f38099a6d08e89229193bac2facdcabd0f32bef44513b43ca0799952950e51L749
It was the change to the regex that @markcmiller86 made.
These are the names of the files:
SpallImpact_112.00000
SpallImpact_112.00000.1
SpallImpact_112.00000.2
SpallImpact_112.00000.3
SpallImpact_112.00000.4
SpallImpact_112.00000.5
SpallImpact_112.00000.6
SpallImpact_112.00000.7
SpallImpact_112.00088
SpallImpact_112.00088.1
SpallImpact_112.00088.2
SpallImpact_112.00088.3
SpallImpact_112.00088.4
SpallImpact_112.00088.5
SpallImpact_112.00088.6
SpallImpact_112.00088.7
SpallImpact_112.00096
SpallImpact_112.00096.1
SpallImpact_112.00096.2
SpallImpact_112.00096.3
SpallImpact_112.00096.4
SpallImpact_112.00096.5
SpallImpact_112.00096.6
SpallImpact_112.00096.7

The previous regex worked with filenames like this. The new one does not. I don't want to revert the change since it was made to fix another issue. I'm not comfortable enough with regexes to know what the best solution would be. @markcmiller86 @visit-dav/visit-developers please advise.

I have placed the data here on the CZ: /usr/workspace/visit/bug_data/issue_20080/SpallImpact.tar

https://teams.microsoft.com/l/message/19:f2ed7be3682d40d1b8e038744e500a09@thread.skype/1732314766968?tenantId=a722dec9-ae4e-4ae3-9d75-fd66e2680a63&groupId=70162982-9587-4bcc-ad53-20178c76fe11&parentMessageId=1732314766968&teamName=VisIt&channelName=General&createdTime=1732314766968

Will try to fix tomorrow PM. Will be away all day tomorrow.

FWIW, the old logic would correctly obtain cycle ONLY for those filenames without an ending .[1-7]. In other words, for that list of files you have, the old logic correctly obtained cycle for SpallImpact_112.00000, SpallImpact_112.00088 and SpallImpact_112.00096. The all the other cases listed here, it would rerturn the last single digit value.

I will have a few options for fixing monday morning. Handling for ALL possible cases is slightly more involved.
I was thinking of adding an option for all database plugins to specify their format's cycle regex. There would be a default which would be the regex that works with file names used at LLNL but plugin developers would have the option of overriding.  Doing so would involve a new method on either GeneralDatabasePluginInfo or CommonDatabasePluginInfo to get the plugin's default cycle regex. There would be a default implementation in the base class but plugins could override. I would also override MOAB's default.


Syntax highlighting breaks Bracket Colorizer 2

I use https://marketplace.visualstudio.com/items?itemName=CoenraadS.bracket-pair-colorizer-2 to keep track of bracket usage throughout my js code. I love this extension except that it makes all my brackets gray.

@marcusnewton I think you should use that as a feature request to bracket-pair-colorizer-2
This extension can't handle that.

BTW The bracket-pair-colorizer-2 is work well on me  🙂

Any idea why it might be working for you but not for me?


Unable to Implement Guardrails AI on GPT4All

Describe the bug
I'm working on a small research project to effectively implement guardrails on open source LLMs such as GPT4All and other Nomic and Leuther models. While implementing Guardrails AI on GPT4All, I am having the following problem:
To Reproduce
Steps to reproduce the behavior:

Image of Rail:

Runtime arguments:

raw_llm_output, validated_output = guard(
model.generate,
prompt_params={"question": "Give me a brief history of the second world war?"},
temperature=0
)
Expected behavior
Having implemented Guardrails AI on ChatGPT by OpenAI, I want this code to work in a similar manner in that I get a validated output by the end which has been verified by the created Guardrails spec.
Error Message:
The callable fn passed to Guard(fn, ...) failed with the following error: GPT4All.generate() got an unexpected keyword argument 'temperature'. Make sure that fn can be called as a function that takes in a single prompt string and returns a string.
My Understanding of the Problem
From what I've been able to understand, GPT4All has the keyword 'temp' for temperature e.g.
model.generate(prompt='what is the capital of France?', temp=0)
On the other hand, Guardrails AI requires the keyword temperature parameter to be given. I think that is causing a clash.
If my assessment is incorrect, what is the problem and how to solve it? If I am right, how can this be resolved?

Hey, sorry for the late reply. See here for an example of how to use GPT4All with guardrails: https://github.com/guardrails-ai/guardrails/blob/gpt4all-example/docs/examples/gpt4all-guardrails.ipynb


Tabbed Area with React Router not working

Hi All,
I'm trying to get TabbedArea working with react router but each time I hit a different Tab it just refreshes and displays the current Tab again. If I enter the path name in the browser it displays correctly but again if I click to go to another tab i just get a refresh of the current Tab.
code snips below. Any Idea where I'm going wrong?
On a layout page I have the following
  <div className="container">
    <Row>
      <Col md={12}>
        <TabbedArea defaultActiveKey={1}>
          <TabPane eventKey={1} tab='Stock' to="index"><RouteHandler /></TabPane>
          <TabPane eventKey={2} tab='Data' to="data"><RouteHandler /></TabPane>
          <TabPane eventKey={3} tab='Dashboard' to="dashboard"><RouteHandler /></TabPane>
        </TabbedArea>
      </Col>
    </Row>
  </div>

on the data page I have another tab area to display tabs below a header for the data page.
        <Row>
          <Col md={12}>
            <TabbedArea defaultActiveKey={1}>
              <TabPane eventKey={1} tab='Tab1' to="tab1"><RouteHandler /></TabPane>
              <TabPane eventKey={2} tab='Tab2' to="tab2"><RouteHandler /></TabPane>
              <TabPane eventKey={3} tab='Tab3' to="tab3"><RouteHandler /></TabPane>
            </TabbedArea>
          </Col>
        </Row>

Router Code in the app.js
var routes = (

);
Router.run(routes, Router.HashLocation, function (Handler) {
React.render(, container);
});

Ah, you might want to fix the router code in the app.js ;).

It's from react-bootstrap:
http://react-bootstrap.github.io/components.html#tabs
app.js code snippet not appearing for some reason.

I posted a solution for this at react-bootstrap/react-bootstrap#129

thanks, really appreciate it... it worked a treat!!!


Traffic for the  NPL(Node Port local) IP:Port originating with in the same worker node as the pod is not working

Describe the bug
For Traffic originating within the same worker node where the POD is deployed.
The Traffic for the NPL IP & PORT not working.The same works fine if the traffic originate outside the worker node.
To Reproduce
step1 :deploy the POD
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 4
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: <IP_ADDRESS>:5001/service-chain-apps
        ports:
        - containerPort: 8080
        - containerPort: 8081

step2 : create SVC object with NPL enabled
apiVersion: v1
kind: Service
metadata:
  annotations:
    nodeportlocal.antrea.io/enabled: "true"
  labels:
    app: nginx1
  name: svc-nginx-2
spec:
  ports:
  - name: web
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: web-sec
    port: 443
    protocol: TCP
    targetPort: 8081
  selector:
    app: nginx

root@tvk-k8s119-ant-master-1:~# kubectl get svc
NAME          TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubernetes    ClusterIP   <IP_ADDRESS>      <none>        443/TCP          6d18h
svc-nginx-2   ClusterIP   <IP_ADDRESS>   <none>        80/TCP,443/TCP   23h
root@tvk-k8s119-ant-master-1:~#

spte3: verify the NPL annotations & IP tables rules
NAME                              READY   STATUS    RESTARTS   AGE   IP             NODE                                     NOMINATED NODE   READINESS GATES
nginx-deployment-f9798f67-4vn9z   1/1     Running   0          19m   <IP_ADDRESS>   tvk-k8s119-ant-worker-1.vcd.vmware.com   <none>           <none>
nginx-deployment-f9798f67-vs6xd   1/1     Running   0          19m   <IP_ADDRESS>   tvk-k8s119-ant-worker-4.vcd.vmware.com   <none>           <none>
nginx-deployment-f9798f67-vwpzx   1/1     Running   0          19m   <IP_ADDRESS>   tvk-k8s119-ant-worker-3.vcd.vmware.com   <none>           <none>
nginx-deployment-f9798f67-z78hc   1/1     Running   0          19m   <IP_ADDRESS>   tvk-k8s119-ant-worker-2.vcd.vmware.com   <none>           <none>

root@tvk-k8s119-ant-master-1:~# kubectl get pods -o=jsonpath="{range .items[*]}{'\n'}{.metadata.name}{':\t'}{.metadata.annotations}{end}"

nginx-deployment-f9798f67-4vn9z:	{"nodeportlocal.antrea.io":"[{\"podPort\":8080,\"nodeIP\":\"<IP_ADDRESS>\",\"nodePort\":50000},{\"podPort\":8081,\"nodeIP\":\"<IP_ADDRESS>\",\"nodePort\":50001}]"}
nginx-deployment-f9798f67-vs6xd:	{"nodeportlocal.antrea.io":"[{\"podPort\":8080,\"nodeIP\":\"<IP_ADDRESS>\",\"nodePort\":50000},{\"podPort\":8081,\"nodeIP\":\"<IP_ADDRESS>\",\"nodePort\":50001}]"}
nginx-deployment-f9798f67-vwpzx:	{"nodeportlocal.antrea.io":"[{\"podPort\":8080,\"nodeIP\":\"<IP_ADDRESS>\",\"nodePort\":50000},{\"podPort\":8081,\"nodeIP\":\"<IP_ADDRESS>\",\"nodePort\":50001}]"}
nginx-deployment-f9798f67-z78hc:	{"nodeportlocal.antrea.io":"[{\"podPort\":8080,\"nodeIP\":\"<IP_ADDRESS>\",\"nodePort\":50000},{\"podPort\":8081,\"nodeIP\":\"<IP_ADDRESS>\",\"nodePort\":50001}]"}root@tvk-k8s119-ant-master-1:~#

Lets take pod deployed in  worker node 1
IP Tables in worker node 1
root@tvk-k8s119-ant-worker-1:~# iptables -L ANTREA-NODE-PORT-LOCAL -t nat
Chain ANTREA-NODE-PORT-LOCAL (1 references)
target     prot opt source               destination
DNAT       tcp  --  anywhere             anywhere             tcp dpt:50000 to:<IP_ADDRESS>:8080
DNAT       tcp  --  anywhere             anywhere             tcp dpt:50001 to:<IP_ADDRESS>:8081

step3::  send traffic to  svc-ip  from  worker-1  success
root@tvk-k8s119-ant-worker-1:~# curl <IP_ADDRESS>:80

  <title>Service Chaining App</title>

East-West Service Chaining Application

See get-service-info

root@tvk-k8s119-ant-worker-1:~#

step4: send traffic to pod IP from the worker-1 success
root@tvk-k8s119-ant-worker-1:~# curl <IP_ADDRESS>:8080

  <title>Service Chaining App</title>

East-West Service Chaining Application

See get-service-info

step5: send traffic to the NPL  IP from worker-1 - Fail
root@tvk-k8s119-ant-worker-1:~# curl <IP_ADDRESS>:50000
curl: (7) Failed to connect to <IP_ADDRESS> port 50000: Connection refused
root@tvk-k8s119-ant-worker-1:~#

The same works fine if the traffic originate from other than worker (node 4) where the POD is deployed

Thanks @tamilhce for pointing this out. Currently we add a rule in PREROUTING chain to handle traffic for NodePortLocal, to handle traffic originating in the same node we need to do the same in OUTPUT chain.


[Accessibility] Make terminal a11y buffer even more accessible

CC @meganrogge @Tyriar @isidorn
I have tested the terminal a11y buffer by building the latest vscode main branch on my end.
Good news is that the buffer is now readable  Thank you very much! I am sharing some suggestions along with bug reports.

[ ] No clue on whether users have reached the end of the terminal buffer.

Say, you are navigating the a11y terminal buffer via UpArrow/DownArrow. When there are repetitive prompt line, such as "PS C:\Users\john>" screen reader users cannot easily understand whether they have reached the end of the buffer or that's just the repetitive prompt message.
I suggest VSCode team add audio cues to indicate the buffer's top/bottom ending points. Instead of making a new sound, I personally think a gentle chime audio cue like "Terminal Bell (Disabled)" could be used for this purpose.

[ ] Terminal a11y buffer is not read-only.

Since this is contenteditable textbox, unintended delete key can mess up the output content. I confirmed that delete key works inside the buffer.
It would be even nicer if this buffer could be marked up with readonly to prevent users from accidentally deleting their output content.

[ ] Terminal live output is not announced.

As far as I tested on Windows with JAWS and NVDA, terminal aria-live event is not auto-announced. I typed "echo hello" in terminal, but there was no read-aloud feedback.

[ ] Ctrl+UpArrow/Ctrl+DownArrow still does not trigger line-by-line announcement.

This issue does not seem to be fixed on Windows.

I have confirmed that The report I shared above is also replicable with today's insiders:
Version: 1.75.0-insider (system setup)
Commit: f1b09237cd400d92987cf7a9681995e8011ef54c
Date: 2023-01-23T11:53:59.088Z
Electron: 19.1.9
Chromium: 102.0.5005.194
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22621
Sandboxed: Yes

I had filed a very similar report in #171833 and it was closed claiming to be fixed as part of #169853, which was not true at all.

The terminal buffer accessibility has improved by using the textbox view. However, what makes uses feel it worse are:

Ctrl+UpArrow and Ctrl+DownArrow (line-by-line) navigation does not work. yes, this is a critical bug and must be fixed.However, I would like to highlight that this inadvertent regression was already introduced before @meganrogge implemented this contenteditable buffer. This means there are other factors that had broken this feature.

Terminal real-time aria-live event does not work. This is also a critical bug that needs to be fixed asap. I did not experience this issue before this new a11y buffer so I assume that there was something breaking this aria-live.

Ctrl+UpArrow and Ctrl+DownArrow (line-by-line) navigation does not work. yes, this is a critical bug and must be fixed.However, I would like to highlight that this inadvertent regression was already introduced before @meganrogge implemented this contenteditable buffer. This means there are other factors that had broken this feature.

This is what we call "navigation mode" and we closed this issue and https://github.com/microsoft/vscode/issues/171833 because we believe this new buffer experience will be better than and replace that. Navigation mode has been broken for sometime and we aren't sure why - here's the upstream issue investigating that https://github.com/xtermjs/xterm.js/issues/3247. Once this new mode is polished and bugs are ironed out, we can reassess to see if navigation mode is still needed.

Terminal real-time aria-live event does not work. This is also a critical bug that needs to be fixed asap. I did not experience this issue before this new a11y buffer so I assume that there was something breaking this aria-live.

I'll use this issue to track fixing this and create separate ones for the other issues mentioned in the description.

I'm going to make separate issues from this one

@meganrogge

This is what we call "navigation mode" and we closed #166238 and #171833 because we believe this new buffer experience will be better than and replace that.

Yes, I understand. If VSCode team would end up replacing the conventional navigation mode with the new a11y buffer, please don't forget to take out the misleading message as highlighted by @pawin35 via #169853 comment:

Since we do not use ctrl+up and down arrow anymore, the hint to use ctrl+up and down arrow to navigate should be removed.

thanks, tracking that now in #172011 and #170988

Tracking everything in separate issues

@meganrogge Apologies if this is the wrong place to ask, and perhaps I'm confused, but if navigation mode is not fixed, does that mean one can not scroll through the history of previously entered commands and rerun one simply by pressing enter on it? That would be very unfortunate from an efficiency point of view.

@rmwilliams2023 are you asking from a screen reader perspective? If so, it depends. Navigation mode is flaky atm, but for non-windows users, shell integration's Run Recent Command command can be used to navigate history. It does not work on windows atm for screen readers bc pwsh disables PSReadline, which we rely upon


Unclear Instruction in Readme: "And set playwright_cli_executable_path: './node_modules/.bin/playwright'"

In the readme the instruction after installing the npm package for playwright says:
And set playwright_cli_executable_path: './node_modules/.bin/playwright'
What does this mean? set where? A system Env var? in some document? What is this referring to?
Thanks!

Please refer this guide. https://playwright-ruby-client.vercel.app/docs/article/getting_started


Changelog generator discussion

From #25:

What about adding a changelog generator from commits 🤔

From #28:

We do limit the changelog to ten lines, though. I think I recall mention in one of the MPP issues of having MPP generate changelogs from GH. Ferry does this, if you're interested. However, Ferry requires the version numbers for the previous published version and the new published version as properties. Even so, it's a messy process.
https://github.com/TerraformersMC/GradleScripts/blob/2.6/ferry.gradle#L35-L96
[...]
As for the changelogs I think this is worth a new issue to discuss. For Fabric and my own projects ive been using: https://github.com/FabricMC/fabric-action-scripts/blob/main/src/changelog.ts the problem is this is specific to github actions.

So, here's a new issue!
Personally, I'm fine with the current situation, since Ferry already has changelog generation we like.  If changelog generation gets added to MPP, ideally it would be optional.  A couple of differences I think I see:

Ferry only picks changelog lines starting with -  which allows using *  to generate programmer-facing log lines that don't go into the user-facing publications.
Instead of the last action run, Ferry requires the user tell it the last tag, which means to run the workflow we have to go look that up first.  However, it makes the behavior a bit more flexible and is not dependent on GH actions.
There are some configurables for Ferry's changelog generation:

# Changelog Options
# changelog_hide_unimportant_commits tries to hide Merge commits
changelog_hide_unimportant_commits=true
changelog_max_commit_search=200
discord_webhook_changelog_line_limit=10

ideally it would be optional

100%, it would be opt-in and you could always use your own soltuion, in the same way you can now.

Ferry requires the user tell it the last tag, which means to run the workflow we have to go look that up first. However, it makes the behavior a bit more flexible and is not dependent on GH actions.

I wonder if the solution would be to have multiple ways to supply the last commit, either manually, from the last succesfull GHA run (same as the fabric scripts), or the last tag on the branch?
The API could be something along the following lines?
publishMods {
    changelog = githubActions {
        workflow = "release.yml"
    }

    changelog = git {
        from = "v1.0.0" // Tag name
    }
}

The API could be something along the following lines?

I like this.


Fix pt_BR translations

As stated in the Upgrade Notes of version 0.26.0:

Translations - Since many interfaces and lines of text have been updated, It may take a little while for some translations to catch-up. Expect to see more English text than usual if you're using a non-English language option.

This pull request fixes the pt_BR translation strings.

Great work, Thank you very much for offering these translations @DeehSlash. Will merge now so they're in the next feature release.


The SimpleMessageListenerContainer does not shutdown properly

We can see that from logs when we close an application normally:
2024-01-09T10:56:32.795-05:00 DEBUG 24368 --- [    container-1] o.s.a.r.listener.BlockingQueueConsumer   : Received message: (Body:'[B@756a9bb3(byte[5])' MessageProperties [headers={}, contentType=application/octet-stream, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=, receivedRoutingKey=test-supplier-queue, deliveryTag=2, consumerTag=amq.ctag-sbd2qMhyoUNhBz69Y9isVQ, consumerQueue=test-supplier-queue])
2024-01-09T10:56:32.797-05:00 DEBUG 24368 --- [    container-1] o.s.a.r.l.SimpleMessageListenerContainer : Shutting down Rabbit listener container
2024-01-09T10:56:32.806-05:00  INFO 24368 --- [    container-1] o.s.a.r.l.SimpleMessageListenerContainer : Waiting for workers to finish.
2024-01-09T10:56:32.807-05:00 DEBUG 24368 --- [pool-3-thread-5] o.s.a.r.listener.BlockingQueueConsumer   : Received cancelOk for tag amq.ctag-sbd2qMhyoUNhBz69Y9isVQ (test-supplier-queue); Consumer@35a9e665: tags=[[amq.ctag-sbd2qMhyoUNhBz69Y9isVQ]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest@<IP_ADDRESS>:53456/,1), conn: Proxy@2cdb5974 Shared Rabbit Connection: SimpleConnection@6d4502ca [delegate=amqp://guest@<IP_ADDRESS>:53456/, localPort=53462], acknowledgeMode=AUTO local queue size=0
2024-01-09T10:56:37.819-05:00  INFO 24368 --- [    container-1] o.s.a.r.l.SimpleMessageListenerContainer : Workers not finished.
2024-01-09T10:56:37.821-05:00  WARN 24368 --- [    container-1] o.s.a.r.l.SimpleMessageListenerContainer : Closing channel for unresponsive consumer: Consumer@35a9e665: tags=[[amq.ctag-sbd2qMhyoUNhBz69Y9isVQ]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest@<IP_ADDRESS>:53456/,1), conn: Proxy@2cdb5974 Shared Rabbit Connection: SimpleConnection@6d4502ca [delegate=amqp://guest@<IP_ADDRESS>:53456/, localPort=53462], acknowledgeMode=AUTO local queue size=0

Pay attention to 5 sec delay between Waiting for workers to finish. and Workers not finished..
This is reported by the code in the SimpleMessageListenerContainer:
				boolean finished = this.cancellationLock.await(getShutdownTimeout(), TimeUnit.MILLISECONDS);
				if (finished) {
					logger.info("Successfully waited for workers to finish.");
				}
				else {
					logger.info("Workers not finished.");

So, the cancellationLock is not released from a consumer which has just reported Received cancelOk for tag.
The BlockingQueueConsumer must be fixed in its handleCancelOk() to add:
BlockingQueueConsumer.this.activeObjectCounter.release(BlockingQueueConsumer.this);

NOTE: the DirectMessageListenerContainer already has such a logic:
		private void closeChannel() {
			RabbitUtils.setPhysicalCloseRequired(getChannel(), true);
			RabbitUtils.closeChannel(getChannel());
			RabbitUtils.closeConnection(this.connection);
			DirectMessageListenerContainer.this.cancellationLock.release(this);
		}

which is called from the mentioned handleCancelOk().

@artembilan
This issue seems to be still unresolved, by using spring-rabbit 3.15, I try to shutdown SimpleMessageListenerContainer, but also get "Workers not finished" warnning
2024-06-17T15:26:32.411Z INFO [] [SpringApplicationShutdownHook] [][][][][] [o.s.a.r.l.SimpleMessageListenerContainer.lambda$shutdownAndWaitOrCallback$2(693)] - Workers not finished.
2024-06-17T15:26:32.413Z WARN [] [SpringApplicationShutdownHook] [][][][][] [o.s.a.r.l.SimpleMessageListenerContainer.lambda$shutdownAndWaitOrCallback$1(697)] - Closing channel for unresponsive consumer: Consumer@6e18d3f9: tags=[[amq.ctag-zUuqxb9iGhy0RJvcL5ghxQ, amq.ctag-hKRvkNR0o2FzFCqw-Q06IQ]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest@<IP_ADDRESS>:5672/,1), conn: Proxy@4063af6 Shared Rabbit Connection: null, acknowledgeMode=AUTO local queue size=0
I'm trying to understand the commit https://github.com/spring-projects/spring-amqp/commit/37d96412a1fc92d349f05976eb179cb7d8da0bd6#diff-537549efc15fbf10a5941ac7f1a8648a24ffc56cccc7c46f190ccff45351b44b and but I'm worried that he might not work because if a consumer is cancelled, the mainLoop() will not be executed and no chance to release itself?
try { 	initialize(); 				while (isActive(this.consumer) || this.consumer.hasDelivery() || !this.consumer.cancelled()) { 					mainLoop(); 				} 			}

I think you mean like this:
	finally {
				SimpleMessageListenerContainer.this.cancellationLock.release(this.consumer);

after that while with mainLoop().
Yes, if we don't enter to that loop, we don't release this cancelled consumer.
Please, confirm that we are on the same page, and I'll provide the fix shortly.

@artembilan
Thank you very much for your quick reply, I think your fix should work. Please help to fix it at your convenience.

BTW, @haloxinyu , we have just released version 3.1.6. Give it a shot with your project!

@artembilan
It works good! Amazing release speed, thanks a lot!


[🐞] onClick$ not always fired

Which component is affected?
Qwik Runtime
Describe the bug
When dealing with the onClick$ event I am expecting that the code is always executed. Looks like there's some racing codition between the default behaviour of anchor's click and the onClick$ event itself.
Reproduction
https://github.com/nelsonprsousa/qwik-app-on-click-cookies
Steps to reproduce

Start the app with yarn start
Open developer tools
Click the custom anchor I added to the page
The onClick$ event isn't fired
Change the onClick$ event for the other simple one. There's also a race condition even with this simples use case. Sometimes it is fired, sometimes it isn't.

System Info
System:
    OS: Windows 11 10.0.22621
    CPU: (16) x64 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
    Memory: 37.51 GB / 63.72 GB
  Binaries:
    Node: 20.10.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - C:\Program Files\nodejs\yarn.CMD
    npm: 10.2.3 - C:\Program Files\nodejs\npm.CMD
    pnpm: 8.15.4 - C:\Program Files\nodejs\pnpm.CMD
  Browsers:
    Edge: Chromium (122.0.2365.66)
    Internet Explorer: 11.0.22621.1
  npmPackages:
    @builder.io/qwik: ^1.5.1 => 1.5.1
    @builder.io/qwik-city: ^1.5.1 => 1.5.1
    undici: * => 6.7.1
    vite: ^5.1.4 => 5.1.5

Additional Information
No response

@nelsonprsousa have you tried preventing the default behavior of onClick$ on your a tag?

@nelsonprsousa you're triggering a race condition.
Your onclick handler gets called, but since it sometimes has to be downloaded, it can't fire before the browser follows the link.
Instead, you can make your handler a sync$ function (just wrap it with sync$, or use a button that looks like a link, or set the cookie at an earlier time.
Closing as everything is working as intended.


Getting all tweets / replies

Hey!
Thanks for creating such a great library!
I'm trying to retrieve all of my tweets and replies (I have thousands), but I couldn't find any mention of pagination to fetch beyond the maximum limit. Does the library support this feature?
Also, I don't see an option to get my own username or user ID after authentication. Could you clarify how to achieve that?

hi! saw you contributed UserTweetsAndReplies thank you so much! i’ll add tests and merge it tomorow.
i suppose you already found out how to paginate.
this library haven’t yet implemented getting the current user from cookie. you can get screen_name / user_id by making get request to https://api.twitter.com/1.1/account/multi/list.json with empty body.
though if your cookie has multiple accounts logged in (have auth_multi cookie) this will return data for all accounts without flaging which is currently active. to get currently active screen_name you can make get request to https://api.twitter.com/1.1/account/settings.json

Thanks, now we have account endpoints for getting screen_name :)
I tried to use FetchTweetsAndRepliesByUserID by iterate it and sleep 10 seconds between each iteration
But got this error:
panic: response status 429 Too Many Requests: Rate limit exceeded

func run() {
	creds, err := auth.GetCredentials()
	if err != nil {
		panic(err)
	}
	scraper := twitterscraper.New()
	authToken := twitterscraper.AuthToken{Token: creds.AuthToken, CSRFToken: creds.Ct0}
	scraper.SetAuthToken(authToken)

	if !scraper.IsLoggedIn() {
		panic("Invalid AuthToken")
	}
	settings, err := scraper.GetAccountSettings()
	if err != nil {
		panic(err)
	}
	log.Println("Logged in as: ", settings.ScreenName)

	userId, err := scraper.GetUserIDByScreenName(settings.ScreenName)
	if err != nil {
		panic(err)
	}

	// Load cursors for posts and replies
	cursorPosts, err := storage.LoadCursor(".cursor_posts")
	log.Println("Current Cursor:", cursorPosts)
	if err != nil {
		log.Println("No cursor file found for posts, starting from the beginning.")
		cursorPosts = ""
	}

	// Counter for the number of fetched tweets
	fetchedCount := 0

	// First loop to fetch and save posts
	for {
		// Fetch tweets using the cursor
		tweets, newCursorPosts, err := scraper.FetchTweetsAndRepliesByUserID(userId, 20, cursorPosts)
		if err != nil {
			panic(err)
		}

		// If no new tweets are fetched, exit the loop
		if len(tweets) == 0 {
			log.Println("No new posts found. Exiting...")
			break
		}

		// Increment the fetched count by the number of newly fetched tweets
		fetchedCount += len(tweets)
		log.Printf("Fetched %d new tweets. Total fetched: %d\n", len(tweets), fetchedCount)

		// Save the new cursor state for posts
		if err := storage.SaveCursor(".cursor_posts", newCursorPosts); err != nil {
			panic(err)
		}

		// Save each tweet in JSONL format
		if err := storage.SaveTweetJSONL("posts.jsonl", tweets); err != nil {
			panic(err)
		}

		// Update cursor for the next iteration
		cursorPosts = newCursorPosts

		// Optional: Delay to avoid hitting rate limits
		time.Sleep(sleepBetweenRequests)
	}

	log.Printf("Total tweets fetched: %d\n", fetchedCount)
}

The default sleepBetweenRequest is 10*time.Second`
Did I make the requests too quickly?
How many tweets it takes by default? I noticed that in twitter UI it takes 20 at each scroll.

@thewh1teagle i was doing the same task recently and 15 seconds delay was enough. each request usually return 20 tweets, but sometimes can do 15-90. this lib has implemented method scraper.WithDelay(15) which you can use instead your sleepBetweenReques

@imperatrona
Thanks! Good to hear that you did it recently, though note that I'm using the new endpoint from https://github.com/imperatrona/twitter-scraper/pull/20. It's almost the same like the getTweets except that it returns basically everything that the user posted - tweets / replies / reposts / quotes etc.
I changed it to use withDelay instead of sleeping and increased the timeout. I'll check later. hope it will works without this error.

	creds, err := auth.GetCredentials()
	if err != nil {
		panic(err)
	}

Could you please give me an example of how to implement the func auth.GetCredentials() ?

Could you please give me an example of how to implement the func auth.GetCredentials() ?

I used playwright in go.
Instead of messing with complicated auth I log in manually and then extract the cookie with playwright and store / load them from file.

thank you.
I will try the repository https://github.com/playwright-community/playwright-go.

Instead of messing with complicated auth I log in manually and then extract the cookie with playwright and store / load them from file.
@thewh1teagle
Could you please provide an example of steps or document?
I have read the playwright document, it is so difficult to implement a twitter login.

I have read the playwright document, it is so difficult to implement a twitter login.

I didn't implemented automatic login.
I open the browser from the code, and wait for the user to login and press enter in the terminal

I found out an example https://github.com/go-numb/x-post-to-blue/blob/master/mod.go#L193
but I am not sure this is your way.
and I try the code https://github.com/playwright-community/playwright-go?tab=readme-ov-file#example, replace https://news.ycombinator.com with "https://x.com/i/flow/login", but does not open the browser from the code.
Could you please share me an example of how to open the browser from the code?
and how to get the auth_token and CSRFToken.

@wade-liwei
See their examples
https://github.com/playwright-community/playwright-go/blob/main/examples/scraping/main.go

hi @thewh1teagle
I use the following code to get the cookies file.
finish the task: "I log in manually and then extract the cookie with playwright and store / load them from file."
Could you please help me implement how to get the auth_token and CSRFToken?
package main

import (
	"log"
	"time"

	"github.com/playwright-community/playwright-go"
)

func main() {
    // initialize a Playwright instance to
    // perform browser automation
    pw, err := playwright.Run()
    if err != nil {
        log.Fatalf("Could not start Playwright: %v", err)
    }

    // initialize a Chromium instance
    browser, err := pw.Chromium.Launch(
      playwright.BrowserTypeLaunchOptions{
          Headless: playwright.Bool(false), // set to false in development
      },
  )
    if err != nil {
        log.Fatalf("Could not launch the browser: %v", err)
    }

    // open a new page within the current browser context
    page, err := browser.NewPage()
    if err != nil {
        log.Fatalf("Could not open a new page: %v", err)
    }

	//https://x.com/i/flow/login
	if _, err = page.Goto("https://x.com/i/flow/login"); err != nil {
        log.Fatalf("Could not visit the desired page: %v", err)
    }

	time.Sleep(time.Second*100)

	page.Context().StorageState("./cookies.json")

	time.Sleep(time.Second*100)

}

Something like this should work too: https://gist.github.com/cmj/17fa133a948eedd0167bdcbff1dfff19

thank you.


It's stuck after inputing my notes path

$ node main
Notion Export Path:
/Users/bevis/Downloads/Export-3c2c8378-eb74-4a84-b82b-1a8ca3706768

No other logs, Ctrl + C or D doesn't work.

If it's working correctly, only needs more time to run. Could you please print more logs so that the users can get to know it's working well?

Hi @beviz! There might be a slight problem with the starting directory. This is purely a guess, but it could be worth a shot.
Try renaming Export-3c2c8378-eb74-4a84-b82b-1a8ca3706768 to just Export or something else that's a bit shorter. Let me know if that worked out for you. :)
I'll create an issue for verbose logging.
Thank you!

@beviz I updated the script so grab a new copy and give it a shot. Logging can be enabled by using the -v arg when running. You can add verbosity by using -vv or -vvv.

@connertennery thanks for your prompt response. Let me have a try.


Update Contribution guides with TS

Part of #219

Codecov Report

Merging #221 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #221   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          29     29
  Lines         302    302
  Branches       71     71
=====================================
  Hits          302    302

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 571610f...827ce72. Read the comment docs.

Thanks a lot for contributing!
Based on our community guidelines, every person who has a PR of any kind merged is offered an invitation to the ReaKit organization.
Should you accept, you'll get write access to the main repository and a fancy ReaKit logo on your GitHub profile. You'll be able to label and close issues, create new branches etc. Make sure to read our contribution and community guidelines, which explains all of this in more detail.
If you have any questions, let me know!


ERROR: tinycudann-1.7-cp39-cp39-linux_x86_64.whl is not a supported wheel on this platform.

This version of tinycudann can no longer run on colab. What is the reason for this? Is there any solution?
this is image

same

me too， and local install code is invalid too

Just build from source
# 23 min
!pip install torch==1.13.1 torchvision functorch --extra-index-url https://download.pytorch.org/whl/cu117
!pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

Per one of my friends' solution, we verify that this works. You can use the Tools in the menu, choose "Command Palette" and search for "Use Fallback runtime version". Then colab will turn back to python 3.9 and everything will get back to normal.
Hope nerfstudio team come up with 3.10 colab fix in the near future. @KoryakovDmitry

Per one of my friends' solution, we verify that this works. You can use the Tools in the menu, choose "Command Palette" and search for "Use Fallback runtime version". Then colab will turn back to python 3.9 and everything will get back to normal.
Hope nerfstudio team come up with 3.10 colab fix in the near future. @KoryakovDmitry

只需从源代码构建
# 23 min
!pip install torch==1.13.1 torchvision functorch --extra-index-url https://download.pytorch.org/whl/cu117
!pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

只需从源代码构建
# 23 min
!pip install torch==1.13.1 torchvision functorch --extra-index-url https://download.pytorch.org/whl/cu117
!pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

sorry,it is invalid

根据我朋友的解决方案，我们验证这是否有效。您可以使用菜单中的工具，选择“命令面板”并搜索“使用回退运行时版本”。然后 colab 将回到 python 3.9，一切都会恢复正常。
希望nerfstudio团队在不久的将来提出3.10 colab修复程序。

thanks


XamlCloudFontIntegration, where do the fonts come from?

I have been trying to find the documentation about where the fonts come from, because in the app only appears 'Neue Haas Grotesk Text Pro'. I thought it has to come from a source like: https://www.microsoft.com/typography/fonts/  but for instance it does not exist there so:
1.- Where is the source ?
2.- Is it possible to use an Uri to set the source?
Thank you

The fonts are coming from a Microsoft-owned service for serving fonts included in the Windows license accessible by Windows 10 devices. This sample just uses a few fonts; the intent of this sample wasn't to show all of the fonts that are available, but only to show that the mechanisms for downloading fonts from this service that were added into the DirectWrite platform are fully integrated into the XAML framework.
If you want a more-detailed look at the DirectWrite mechanisms and related APIs, you can take a look at the DWriteTextLayoutCloudFont sample. Related to this are extensions to the APIs that we have added in the Windows 10 Creators Update that allow a DirectWrite app to use these mechanisms to access fonts from other font services. For a closer look at that, you can take a look at the DirectWriteCustomFontSets sample that got published on GitHub last week.
But if what you're interested in is more information about the additional fonts, such as Neue Haas Grotesk Pro, that you may not see in your Windows 10 system by default, you can find some details in this topic in the Windows 10 community forum site.

Thank you, now at least I have a list and a 'source' of the fonts.
About other font services, yes, I read https://msdn.microsoft.com/en-us/library/windows/desktop/hh802480(v=vs.85).aspx about  'Some font services, such as Adobe Typekit'. I am a novice in that services I will try to test that one.
Is there any other one service to try? If I understand well, if I have that font service installed and I set the name, the font just works in the UWP app?. Would be possible create one by myself?
The reason of my questions is that I'm creating a great update in 'Universal Character Map' with a lot of new features, and in next releases I would like to master about all downloadable fonts and fonts services.

There are various online font services, most of them intended for use in Web sites using the CSS @font-face rule. (See CSS Fonts Level 3 for more on that.) These may or may not allow for native applications to access fonts.
Some font services, such as Adobe Typekit and Google Fonts, have APIs that apps (Web or native) can interact with. But note that those two examples have very different target audiences and business models: Google Fonts is geared mainly toward use on the Web, and most fonts are open-source fonts provided without any licensing fees. Typekit is geared to a large extent toward Adobe Creative Cloud users, and less than 10% of the fonts are available for free. The pricing model is significant particularly because it generally affects how the font data is handled --- whether there are mechanisms in place to secure it. In the case of Google fonts, the raw font files can be accessed directly with a URL, and it's not hard to discover those URLs. But with TypeKit, on the other hand, the data is pretty tightly controlled. As a result, it's fairly straightforward to use a font from the Google Font service (as well as the Windows service) with the DirectWrite APIs in Windows 10 Creators Update, but it would be harder, if even possible, to access the fonts from Typekit directly with your own code.
Adobe Typekit and Monotype SkyFonts are services that have native sync client utilities that can be used to sync fonts from those services onto a Windows PC and use them like any normally-installed font. These sync clients don't actually install the fonts in the usual sense. Rather, they load the fonts into GDI after you have logged onto the system and the clients are started up. So, you can't use the fonts if the Typekit or SkyFonts client isn't running. The MSDN page you referred to mentions these client utilities; because of the way they interact with GDI, fonts that they load previously weren't visible to DirectWrite apps. But in the Anniversary Update, we did work so that DWrite is aware of fonts that utilities such as these load into GDI.
If you want to access fonts from the Windows service in your app, and the app calls DirectWrite APIs directly, then you can use IDWriteFactory3::GetSystemFontSet or use IDWriteFactory3::GetSystemFontCollection passing includeDownloadableFonts = TRUE, and the results will include fonts that are in the Windows service as well as those that are installed. You can determine the current locality state using IDWriteFontFaceReference::GetLocality, IDWriteFont3::GetLocality, or IDWriteFontFamily1::GetFontLocality.
If your app uses XAML, then you can just reference any of the fonts in the Windows service, the same way as is done in this sample. If the fonts are not cached locally, text will initially display using fallback fonts, but then the XAML text control will automatically refresh when DirectWrite has finished downloading the font.
If you want to use fonts from other online sources in your app, and you call DirectWrite APIs directly, then the new APIs in the Windows 10 Creators Update will be helpful, and you should check out the Custom Font Sets sample I mentioned earlier. But if your app uses XAML or HTML, then there currently isn't a way to use them directly as is done in this sample. Instead, if you have a Web view, you can write @font-face rules to reference font files available on the Web. (If you'd like to be able to use fonts from other services directly in intrinsic XAML text controls, we're interested in hearing about that.)
I hope this helps.

Really appreciated for the detailed information, I will do my best with that information.
If I understand well, In case I want to show in a native XAML app text from a font face from some web oriented font services, an option is using a webview where I should use that css rule (hopefully I was able to create a transparent webview).
I will try also to use 'src: url(http://another.example.com/fonts/simple.woff);' and check how it behaviour.
Thank you for all the details, I will try all the possibilities.

Yes, you understand that correctly. The webview should support URLs pointing to .ttf, .woff or .woff2 files.


Heating Threshold Temperature

Please can anyone help with a way to bypass this
Homebridge Terminal for Synology DSM
Node.js Version: v16.16.0
homebridge-config-ui-x v4.50.2
homebridge-google-nest-sdm v1.1.9
[1/31/2023, 9:23:26 PM] [homebridge-google-nest-sdm] This plugin generated a warning from the characteristic 'Heating Threshold Temperature': characteristic was supplied illegal value: number 4.4 exceeded minimum of 4.5. See https://homebridge.io/w/JtMGR for more info.

That's interesting how did you manage to do this?


PHP 8 compatibility

PHP 8+ - It doesn't like to have functions with required arguments after optional ones - More info: https://php.watch/versions/8.0/deprecate-required-param-after-optional.
Use at least dbal v2.13 to be compatible with PHP 8 as there were an issue with a types being incompatible with the method signature
Remove unnecessary mb_http_input(UTF-8) which had an invalid value. mb_http_input() is used to get the input encoding, but not to set one. Passing an encoding to it doesn't make sense.
PHP-8 Fix incorrect argument given for session_set_cookie_param. lifetime's first argument must be an integer. String was given.

Hope it helps @quartzcms! 😊
Good luck 🎉

Great start, however latest version of DBAL is not compatible with PHP 7.x. Would have to reconsider going strictly with PHP 8 and moving forward. I am going to accept this merge.

Absolutely, this is the reason why I kept DBAL used the latest version of v2.* branch, instead of jumping to DBAL v3.*
https://packagist.org/packages/doctrine/dbal#2.13.9

You are correct. I had an old version of PHP on my server which resulted in errors. I set it to 7.4 to make it work. I will accept those edits and publish some extra after.