Allow build plan to also emit file inputs per invocation

This adds another mode for the build plan (cargo build --build-plan=detailed ...), which differs from the current --build-plan such that:

Additionally collects file dependencies per each invocation (main motivation)
rustc invocations are altered to include built native deps or more build-script-time-detected args (e.g. serde  rustc version-gated --cfg flags)
Actually has to perform work akin to cargo build if necessary

Example --build-plan and --build-plan=detailed diff on Cargo itself
The main motivation is make the build system integration easier. With file dependencies more clear it'll be easier to automatically translate the plan to other build system rules or to reason about about artifact freshness (currently RLS would also benefit by being able to accurately answer the 'given these dirty files, what do I need to rerun' question for a cargo check equivalent build plan).
(I imagine the next step would be to somehow (somewhat) accurately detect build script output (sandboxed? special API?) to better reason about build scripts in general.)
With regards to the actual changes, I couldn't shake off the feeling that the changes made are too bolt-on. While I think tracking which commands are being prepared for the actual work should be on the same code path as a regular build, it seems hacky to:

treat every work to be as Freshness::Dirty
pass is the work fresh at construction time status to the work-preparing function to possibly skip them while actually executing the work
(especially since the work can turn out to be dirty if it depends on a dirty one during job execution)

which effectively acts as an another layer of freshness tracking (all of this applies only to the --build-plan=detailed mode).
r? @alexcrichton

Oops, didn't test on stable:
   Compiling foo v0.5.0 (/Users/travis/build/rust-lang/cargo/target/cit/t266/foo)
error[E0658]: mod statements in non-mod.rs files are unstable (see issue #44660)
 --> src/module1.rs:1:5
  |
1 | mod nested;
  |     ^^^^^^
  |
  = help: on stable builds, rename this file to module1/mod.rs
error: aborting due to previous error

I'll fix this today.

So this PR combines two things:

Detects file inputs to rustc calls
Executes build scripts to detect their side-effects

While both are somewhat orthogonal to each other, currently script execution is necessary for the first item.
Leaving environment variables as another form of input/output aside, if it'd be possible for:

rustc to correctly emit dep-info despite some missing files (and --extern libs?)
build scripts to declare their outputs in the manifest before running

then we wouldn't have to execute anything to learn about file input/output dependencies (sans extra -L ... or other flags added by build scripts but that's what the current --build-plan does atm as well).
The only edge case I can think of is when compiling a crate where rustc expects an entire (nested?) module missing at compile-time, which is actually generated only by the build script. However, if we treat direct dependency outputs as inputs to a given rule, then I believe we still end up with a correct freshness tracking.
I'll see if it'd be feasible to make rustc not error out on missing files in --emit=dep-info-only mode.
cc @jsgf wrt build system integration (+ build.rs shenanigans)
@nrc actually this run and report complete build plan for cargo check would be useful for RLS if we decide to stop using Cargo-the-library in-process

:umbrella: The latest upstream changes (presumably #6236) made this pull request unmergeable. Please resolve the merge conflicts.

Got one error on OSX:
---- fix::fix_path_deps stdout ----
running `/Users/travis/build/rust-lang/cargo/target/debug/cargo fix --allow-no-vcs -p foo -p bar`
thread 'fix::fix_path_deps' panicked at '
Expected: execs
    but: differences:
  1 - |[FIXING] bar/src/lib.rs (1 fix)|
    + |    Checking foo v0.1.0 (/Users/travis/build/rust-lang/cargo/target/cit/t564/foo)|
  2 - |[CHECKING] foo v0.1.0 ([..])|
    + |      Fixing bar/src/lib.rs (1 fix)|

hopefully it's spurious and related to https://github.com/rust-lang/cargo/pull/6236; rebased again.

Hm, the time regression (1 hr AppVeyor timeout) seems unlikely? I believe cloning is the most expensive operation but I think it shouldn't impact it this much? I'll retrigger CI and will try to optimize it.

Gah sorry for the delay here, I inended to take a closer look much sooner!
At a high level, it sounds like this is basically executing cargo check, right? Although instead of "Checking ..." we're just executing rustc --emit dep-info, but otherwise build scripts are always executed to completion (as well as procedural macros compiled and such).
Looking this over it does also feel sort of bolted on, and I'm not sure I was able to quite follow why a Dirty freshness needed to be forced in a few locations? I think this can be implemented at a high level by "execute cargo check" and then inspect all the internal data structures afterwards to figure out what all files were used as input.
I think after that it'd be a perf optimization to not run rustc --emit metadata but rather just rustc --emit dep-info.
Would something like that be possible? Or was that already attempted and I should ready the patch more closely? (which is a totally valid!)

:umbrella: The latest upstream changes (presumably #6270) made this pull request unmergeable. Please resolve the merge conflicts.

I'm sorry as well, I intended to reply earlier.

At a high level, it sounds like this is basically executing cargo check, right?

Uh, to be fair it's worse than that - it's basically running entire cargo build (the --build-plan=detailed variant). The reason for that is, I wanted the plan to be accurate for the cargo build routine and was too hesitant to mess with output files and tricking Cargo into rmeta compilation, passing those updated crates as new --extern but still reporting the usual rlib files and so on.
I figured it'd be a good starting point and we could optimize this further later on.

Looking this over it does also feel sort of bolted on, and I'm not sure I was able to quite follow why a Dirty freshness needed to be forced in a few locations?

The reason for that is the current architecture - when preparing work, we build 2 Work closures to be executed if a given unit is determined to be either fresh or dirty. The problem is, if the work is detected to be fresh (e.g. a second run cargo build run, with --build-plan or not) then we currently won't even prepare an appropriate invocation and we'll have a default, empty invocation for fresh units.
To not regress the fresh case I didn't want to modify them to always build the invocation/work even if it's fresh and so what I did is follow the current solution of treating every work as Dirty if we build with --build-plan=detailed but optionally skip the real work (executing rustc, running build script binaries).

I think this can be implemented at a high level by "execute cargo check" and then inspect all the internal data structures afterwards to figure out what all files were used as input.

Yup, that's basically what I'm trying to do here for the input files! Moved the build plan emission after collecting necessary dep-info data (https://github.com/rust-lang/cargo/pull/6213/files#diff-29d98a4a0b448960d02ef89415610e9d, relevant call) but there's that small problem with fresh work, as per above.

I think after that it'd be a perf optimization to not run rustc --emit metadata but rather just rustc --emit dep-info.

Unfortunately, I've hit a wall with that - here's a small note on my progress with that. The main blocker is that I think we still need external metadata to fully resolve conditional compilation and that's needed to precisely determine the set of files being required for the compilation. If RFC #2523 is merged as-is with #[cfg(accessible($path))] being able to be used on paths not only originating from std then we'd need the dep metadata compiled before we can resolve the inputs.
I hope I'm not overthinking this, though!

Hm ok I'm trying to tease apart some of the various pieces here. One (probably longstanding at this point) problem is that the build plan is produced as a side effect of compilation rather than directly, which means that the freshness bits feel pretty wonky. There's a lot of logic in Cargo's production  of units of work to figure out what to actually execute, but the current code is primarily intended for incremental compilation (in perhaps not the best fashion) rather than producing a full-fledged build plan which was sort of bolted on after the fact. I think we can probably fly with this, but this strategy of having the build plan bolted on the side of Cargo seems likely to build up technical debt quickly and probably expose a lot of various bugs in the build plan.
It sounds like there's also a discrepancy between --emit dep-info and --emit metadata in terms of the preciseness of the inputs? It sounds like you're saying (but correct me if I'm wrong) that metadata emission produces more accurate dependency information than the dep-info output. If that's the case it's a bug in the compiler that we should fix long term, and ideally wouldn't impact the design in Cargo here too much except for ideally at most a small workaround.

I'd be happy to implement a more complete solution to that as long as it doesn't require rewriting everything, if you could give me some pointers :sweat_smile:
About discrepancy, I'm sorry, I should be more clear about it. Right now, the rustc --emit dep-info call only needs to go through expansion but the compilation has to have the correct rmetas or rlibs to read or otherwise it'll fail and not generate a .d file. This means that currently we have to have our dependencies compiled/checked in order for a bare rustc --emit dep-info to succeed (which means every node except leaves have to be compiled anyway). This might be possible to solve in the compiler but I'm not sure how far can we go through the expansion having only dummy extern crate paths in the AST (I can't think of any edge case but maybe there are some?).
Also, the reason I mentioned RFC #2523 is that it'd be possible to have:
#[cfg(accessible(serde_json::Value))]
const DATA: .. = include_bytes!(..);

meaning we couldn't possibly resolve the conditional compilation and thus determine inputs unless we compile/check the possible dependencies. Does that make sense?

:umbrella: The latest upstream changes (presumably #6328) made this pull request unmergeable. Please resolve the merge conflicts.

I wonder if we can perhaps remove the previous concept of a build plan entirely to ease this transition?

A big advantage of the regular --build-plan is that it's almost instant (albeit inaccurate wrt build script output, modifying package or deps invocation/envs), whereas the detailed mode is as fast only when cargo build cache is up to date.
I think it'd be good to ask possible stakeholds whether such a trade-off is acceptable
cc @luser @hobofan (do we know of any other use cases?)

For our current usage of --build-plan to build Firefox with tup it's a hard requirement that cargo not do any actual work. I suspect other users with similar requirements (bazel et. al) would say the same. /cc @mshal

@alexcrichton I tried to make the same point in https://github.com/rust-lang/cargo/issues/5579#issuecomment-438417980, and I agree. I don't think that --build-plan has been sufficiently speced out in terms of what it covers/doesn't cover, or at least that should be well documented before stabilization.

I discussed this a bit with @Xanewok recently at rustfest, and here's some thoughts that we had:

Cargo already has a pessimization of what the possible inputs to a build script can be (all files in a package), and typically a build script will simply inform us that it only cares about a subset of these files.
Similarly, without running rustc, Cargo could assume the same pessimistic input set for rustc invocations.
Crates can always override this pessimization with include and exclude keys in Cargo.toml

As a result, I think that we can solve both problems here of (a) generating a build plan by executing zero work and (b) generating a correct build plan handling build scripts and such.
The "trick" here is that instead of executing rustc we'd execute a wrapper around rustc. This wrapper would take, as input, the directory that the build script dumped output into. The wrapper would parse the build script's output looking for arguments and such, and then it would invoke the real rustc with appropriate arguments.
Cargo could presuambly ship with such a built-in command/wrapper so it'd still just be cargo/rustc, but that way we could build a static build plan describing exactly what Cargo would otherwise do, and then execute that. (and maybe Cargo itself could switch to doing something like this?) In theory it should handle build scripts and such (cfgs flying around, etc). The downside is that it wouldn't do incremental rebuilds perfectly, but in theory this is largely only intended for Buck/Gecko like solutions in the first go where all of crates.io isn't incrementally rebuilt but only built once.
@Xanewok does that sound about right? I likely missed things!

@alexcrichton I think the division between lazy+dynamic and eager+static (precise?) is definitely good direction here!
I'm somewhat concerned with correctness - in general, pessimization applies to uploaded crates, however local crates can technically include/depend on files outside of the package directory (e.g. user home directory), so I don't think we can count on the pessimistic input file set to be correct but possibly suboptimal :(
I think people integrating cargo with an external build system will likely have locally developed or vendored packages and so it'd be risky to make this assumption.
Maybe we shouldn't just include the source file set in this case? Or include the pessimization as you're saying but somehow signal to the user that this is a best-effort (e.g. make it a --build-plan=crude or similar?).
Wrapping rustc sounds good, yeah! Having a lazy, but correct plan would be an improvement what we have now. However, I think at some point the consumers would like sitll to have a static, resolved version of the rustc calls (to translate the rules to other system or analyze the compilation process).
Also there's the case of DEP_X_Y keys where build scripts are not only isolated to one package but can pass information further to dependend packages - would we need to persist that information or execute these wrappers referring to the ones from depending crates?
Do you think it might be a good idea to have the build plan execution with more fine-grained accuracy?
For example:

crude, which is correct and almost instant but is still dynamic (in the sense that wrappers analyze the build script execution and execute rustc with appropriate flags etc.)
moderate, which is mostly static (possibly only runs build scripts and includes necessary cfgs, flags. etc. in the rustc executions)
detailed, which potentially also compiles dependent crates, but additionally includes input files?

Generally, it'd also be good if we could include the build scripts output (e.g. @luser's post on hardcoding outputs in tup in the Firefox build).
In the crude mode we could point to the OUT_DIR directory as 'output', but in the modes where we execute the build script we could enumerate those files or possibly also compare timestamps pre- and post-build script to see if it may have modified/created any files inside the package itself (e.g. lalrpop generated .rs files in-directory, prior to 0.15) (this still may potentially modify/create files in arbitrary places but we can't possibly scan the entire filesystem :laughing: )
Does this sound reasonable?

I don't think we can count on the pessimistic input file set to be correct but possibly suboptimal :(

Oh this is what I meant by include and exclude keys in Cargo.toml. It's definitely true that the default heuristic is wrong, but I figured it's ok so long as there's a way to specify the correct dependencies.

Do you think it might be a good idea to have the build plan execution with more fine-grained accuracy?

I think one thing that's unsettling me is that it seems weird to have multiple build plans. In theory there's only one correct build plan and we should largely just make that as easy to use as we can. It seems odd to special case "fast" or "incorrect" build plans :(

However, I think at some point the consumers would like sitll to have a static, resolved version of the rustc calls (to translate the rules to other system or analyze the compilation process).
Also there's the case of DEP_X_Y keys where build scripts are not only isolated to one package but can pass information further to dependend packages - would we need to persist that information or execute these wrappers referring to the ones from depending crates?

I think it's fine to tell users that they won't literally run rustc though but will rather run a rustc-lookalike. Each compilation will be pretty isolated and anything that isn't rustc is what Cargo is already doing internally, only shifted to a different point in time.
I'm also imagining that these wrappers would have enough support to handle things like DEP_X_Y keys, that'd all be slurped up from various files and be located as inputs to all the various subcommands.
What I think we should go towards is basically the speed/immediateness of the current --build-plan plus the accuracy of the build plan in this PR. That'll require a number of wrappers here and there and some things which look somewhat odd in the build plan, but I don't think there's any reason why it wouldn't be able to fundamentally work.

Oh this is what I meant by include and exclude keys in Cargo.toml. It's definitely true that the default heuristic is wrong, but I figured it's ok so long as there's a way to specify the correct dependencies.

Ah, makes sense! Do you think we could warn against rustc/build script depending on stuff (when calculating fingerprint) that's outside the package but not specified in the include keys? I think noone uses that key to do that now :speak_no_evil:

Certainly!

Conflicts have amassed, sorry about that. Can we keep this PR as related discussion or should I open an issue and close this for now?
We talked with @nrc briefly about some ideas and possible direction we can take.
In general, hiding implicit dependencies behind those wrappers might not seem like the best approach, since generally tools want to have direct knowledge to begin with.
To solve the fast vs correct problem, with each invocation we could attach a 'possibly incorrect' information if it transitively depends on a build script unit that's not already executed.
With this, the current instant build plan could be generated as-is with that info attached or users could execute the plan with --dry-run which would build and execute build scripts if not done before and cache the result (here --build-plan=detailed?).
@nrc @alexcrichton thoughts?

Oh I'm fine either way in terms of an issue vs a PR discussion. I talked with @nrc a bit on the bus at Orlando, and one thing we realized is that as more and more procedural macros come about the "slow" part here is likely to get slower and slower (because it's running cargo check). I feel like I remember convincing him that the fast-but-full-detail method was possible and would work, but I don't remember being super convincing at the same time. @nrc may be able to help fill in the gaps!
I'm personally still a fan (if we don't have time pressure, which I don't think we do) to produce a full build plan which uses Cargo shims as necessary. Such a plan would be produced instantaneously (run no work) and would have the downside of otherwise not executing literally rustc itself, but I think that's one which can be solved in time

I'm gonna close this to help clear out the queue a bit and I think it's a bit stale, but I'm sure we'll continue to converse about this!

No worries, I'll try and come back to this in the nearest future :)


Refactored ET functions associated interface functions

Refactored ET functions for code simplicity
Each ET function has its own interface
Additions
models/kernels/EtAerodynamicMethod.hpp
models/kernels/EtAerodynamicMethodEx.hpp
models/kernels/EtCalcProperty.hpp
models/kernels/EtCombinationMethod.hpp
models/kernels/EtCombinationMethodEx.hpp
models/kernels/EtEnergyBalanceMethod.hpp
models/kernels/EtEnergyBalanceMethodEx.hpp
models/kernels/EtPenmanMonteithMethod.hpp
models/kernels/EtPenmanMonteithMethodEx.hpp
models/kernels/EtPriestleyTaylorMethod.hpp
models/kernels/EtPriestleyTaylorMethodEx.hpp
models/kernels/EtStruct.h
test/models/hymod/include/Et_All_Methods_Test.cpp
Removals

Changes

Testing

Unit test
Validation test

Screenshots
Notes

In the future, may write a single interface for all ET functions

Todos

Checklist

[x ] PR has an informative and human-readable title
[x ] Changes are limited to a single goal (no scope creep)
[x ] Code can be automatically merged (no conflicts)
[x ] Code follows project standards (link if applicable)
[x ] Passes all existing automated tests
[x ] Any change in functionality is tested
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged / future todos are captured in comments
[ ] Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
[x ] Reviewers requested with the Reviewers tool :arrow_right:

Testing checklist (automated report can be put here)

[----------] 5 tests from EtCalcKernelTest
[ RUN      ] EtCalcKernelTest.TestEnergyBalanceMethod
energy balance method:
calculated instantaneous potential evapotranspiration (PET) =8.594743e-08 m/s
calculated instantaneous potential evapotranspiration (PET) =7.425858 mm/d
[       OK ] EtCalcKernelTest.TestEnergyBalanceMethod (0 ms)
[ RUN      ] EtCalcKernelTest.TestAerodynamicMethod
aerodynamic method:
calculated instantaneous potential evapotranspiration (PET) =8.977490e-08 m/s
calculated instantaneous potential evapotranspiration (PET) =7.756551 mm/d
[       OK ] EtCalcKernelTest.TestAerodynamicMethod (0 ms)
[ RUN      ] EtCalcKernelTest.TestCombinationMethod
combination method:
calculated instantaneous potential evapotranspiration (PET) =8.694909e-08 m/s
calculated instantaneous potential evapotranspiration (PET) =7.512402 mm/d
[       OK ] EtCalcKernelTest.TestCombinationMethod (0 ms)
[ RUN      ] EtCalcKernelTest.TestPriestleyTaylorMethod
Priestley-Taylor method:
calculated instantaneous potential evapotranspiration (PET) =8.249098e-08 m/s
calculated instantaneous potential evapotranspiration (PET) =7.127221 mm/d
[       OK ] EtCalcKernelTest.TestPriestleyTaylorMethod (0 ms)
[ RUN      ] EtCalcKernelTest.TestPenmanMonteithMethod
Penman Monteith method:
calculated instantaneous potential evapotranspiration (PET) =1.106268e-07 m/s
calculated instantaneous potential evapotranspiration (PET) =9.558159 mm/d
[       OK ] EtCalcKernelTest.TestPenmanMonteithMethod (0 ms)
[----------] 5 tests from EtCalcKernelTest (0 ms total)
Target Environment support

[ ] Linux

Each Ex file serves as an interface for each of the ET methods. Each also
contains all parameters needed. As it is set up now, each Ex file contains
a single function that can be called directly by a calling function without
supplying any parameters.
On Wed, Aug 26, 2020 at 3:26 PM Nels<EMAIL_ADDRESS>wrote:

@hellkite500 requested changes on this pull request.
I think this is a good start to organizing the various ET kernels that we
might want to expose to models for use in their formulations. We need to
continue looking at the data dependencies and the calling signatures that
are required to make these kernels as interchangeable as possible without
requiring significant overhead in the caller.
Before this is merged, we should minimally make sure that each header has
include guards, and the functions need some indentation added to them. I
would also extract these into a kernels/evapo_transpiration sub-directory
(using git mv).
In models/kernels/EtAerodynamicMethod.hpp
https://github.com/NOAA-OWP/ngen/pull/152#discussion_r477554895:

+//############################################################*
+// subroutine to calculate evapotranspiration using           *
+// Chow, Maidment, and Mays textbook                          *
+// F.L. Ogden, NOAA National Weather Service, 2020            *
+//############################################################*
+extern double evapotranspiration_aerodynamic_method
+(
+evapotranspiration_options *et_options,
+evapotranspiration_params *et_params,
+evapotranspiration_forcing *et_forcing,
+intermediate_vars *inter_vars
+)
+{
+// local varibles
+double psychrometric_constant_Pa_per_C;
This needs some indentation to denote the function body.
In models/kernels/EtPriestleyTaylorMethod.hpp
https://github.com/NOAA-OWP/ngen/pull/152#discussion_r477559075:

@@ -0,0 +1,134 @@
+#include

Each of the headers should probably have a header guard to be consistent:
#ifndef ET_PRIESTLEY_TAYLOR_METHOD_H
#define ET_PRIESTLEY_TAYLOR_METHOD_H
...
#endif // ET_PRIESTLEY_TAYLOR_METHOD_H

In models/kernels/EtPriestleyTaylorMethodEx.hpp
https://github.com/NOAA-OWP/ngen/pull/152#discussion_r477561609:

+et_options.use_penman_monteith_method  = FALSE;

+// -----UNIT TEST RESULTS:-----
+
+// -----Priestley-Taylor method: -----
+// calculated instantaneous potential evapotranspiration (PET) =8.249098e-08 m/s
+// calculated instantaneous potential evapotranspiration (PET) =7.127221 mm/d
+
+
+//###################################################################################################
+// MAKE UP SOME TYPICAL AORC DATA.  THESE VALUES DRIVE THE UNIT TESTS.
+//###################################################################################################
+//read_aorc_data().  TODO: These data come from some aorc reading/parsing function.
+//---------------------------------------------------------------------------------------------------------------
+
+aorc.incoming_longwave_W_per_m2     =  117.1;
I'm not sure of the purpose of these Ex files, are these just examples? If
so, then hard coding these values is OK, otherwise need to be structured
and passed to various ET methods in a way that model formulations can use a
consistent calling pattern but explicitly call different methods.
In models/kernels/EtPriestleyTaylorMethodEx.hpp
https://github.com/NOAA-OWP/ngen/pull/152#discussion_r477563784:

+// 2.5       0.2        Agricultural land with many houses, shrubs and plants, or 8 metre tall sheltering hedgerows
+//                        with a distance of approximately 250 metres
+// 3         0.4        Villages, small towns, agricultural land with many or tall sheltering hedgerows, forests
+//                        and very rough and uneven terrain
+// 3.5       0.8        Larger cities with tall buildings
+// 4         1.6        Very large cities with tall buildings and skyscrapers
+// Roughness definitions according to the European Wind Atlas.
+//
+// According to the UN FAO Penman-Monteith example here:
+//        http://www.fao.org/3/X0490E/x0490e06.htm#aerodynamic resistance (ra)
+// the zero plane roughness length,"d" can be approximated as 2/3 of the vegetation height (H): d=2/3H.
+// the momentum roughness height "zom" can be estimated as 0.123H.
+// the heat transfer roughness height "zoh" can be approximated as 0.1 * zom.
+//-----------------------------------------------------------------------------------------------------------------

+double priestley_taylor_method_realization()
the "realization" term is miss-represented. The original intent of
"improve realization handling of ET" was aimed at improving the calling of
ET functions from a model formulation so that we weren't doing some funny
business of casting void * functions and assuming the correct function
and call signature. These ET functions should be just that, functions with
a unified "look and feel" if you will. Similarly, I would make a sub
directory under models/kernels to hold these various modules
models/kernels/evapo_transpiration/ is probably a good place to start.
Might also consider wrapping them all under an ET namespace.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/NOAA-OWP/ngen/pull/152#pullrequestreview-475815283,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACA4SRJGN65MFCJ4ODYTQLTSCVVVRANCNFSM4QKXSN3Q
.

The problem is many of these parameters need to be dynamic, for example longitude will be different for each catchment that wants to compute ET, as well as day of the year and other dynamic inputs, and the forcing will change as well.  I don't see how we get these new parameters for each time step into the Ex so we can use these simple interfaces and have this kind of dynamic behavior.  If I'm missing something, let me know!

I guess we can put in some struct pointers in the arg list for the Ex
functions to handle the input parameters as all the parameters seem to be
built into the structs. There are some logics and general function calls
handled in the Ex files which make the each ET method code simple. So their
existence makes both upper level calling function and the ET calculation
methods simple. Do you think this is a reasonable approach? BTW, I have
also just tested a unified Ex function that can be used for all ET methods.
Let me know if you have any suggestions on this.
On Thu, Aug 27, 2020 at 8:32 AM Nels<EMAIL_ADDRESS>wrote:

The problem is many of these parameters need to be dynamic, for example
longitude will be different for each catchment that wants to compute ET, as
well as day of the year and other dynamic inputs, and the forcing will
change as well. I don't see how we get these new parameters for each time
step into the Ex so we can use these simple interfaces and have this kind
of dynamic behavior. If I'm missing something, let me know!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/NOAA-OWP/ngen/pull/152#issuecomment-681951011, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACA4SRN3T73YVJTBILAMFKDSCZN57ANCNFSM4QKXSN3Q
.

I'am debating with myself how we want to build an interface function
between the various ET methods and the main calling function. I am inclined
to put all the parameter handling in an interface function. This way, we
have all parameter setup in one place so we can always go there if we want
to change parameters. The alternative is to expose all the parameter
handling in the main calling function, which will make the calling function
looks messy.
On Thu, Aug 27, 2020 at 9:18 AM Shengting Cui - NOAA Affiliate <
<EMAIL_ADDRESS>wrote:

I guess we can put in some struct pointers in the arg list for the Ex
functions to handle the input parameters as all the parameters seem to be
built into the structs. There are some logics and general function calls
handled in the Ex files which make the each ET method code simple. So their
existence makes both upper level calling function and the ET calculation
methods simple. Do you think this is a reasonable approach? BTW, I have
also just tested a unified Ex function that can be used for all ET methods.
Let me know if you have any suggestions on this.
On Thu, Aug 27, 2020 at 8:32 AM Nels<EMAIL_ADDRESS>wrote:

The problem is many of these parameters need to be dynamic, for example
longitude will be different for each catchment that wants to compute ET, as
well as day of the year and other dynamic inputs, and the forcing will
change as well. I don't see how we get these new parameters for each time
step into the Ex so we can use these simple interfaces and have this kind
of dynamic behavior. If I'm missing something, let me know!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/NOAA-OWP/ngen/pull/152#issuecomment-681951011, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACA4SRN3T73YVJTBILAMFKDSCZN57ANCNFSM4QKXSN3Q
.

closed in favor of #168


Simulated profit

Hello,
i have issues with the simulated profit that Gekko Shows me.
Where can i reset the simulated profit that gekko saved during its time of capturing.

If this is still relavant: What do you mean?


DPDB_CACHE has duplicate entries for ios devices (aka iPhones)

Why are all "ios" related entries mentioned twice ?
I was investigating this because whenever I use an iPhone6/7 the stereo settings are weird (see screencapture below).

Same cardboards v1/v2 work fine with most Android phones.

Seems to be fixed by #183?

In #233, the local DPDB has been updated and sync'd with the online version at webvr.rocks (once https://github.com/WebVRRocks/webvr-polyfill-dpdb/pull/7 is merged). Try the latest version which should fix this -- please reopen if this does not fix the issue. Thanks!


Waymo dataset is a tar file. I don't see any frames

Hello. I downloaded waymo dataset tar file. Inside the tar file, there are no images, just tfrecord files. What should I do now?

Hi.
You should put all those .tfrecords files inside waymo_dataset directory.

@tomaszkaliciak is right. Waymo uses its own data structure named .tfrecord and you should see the images after the conversion.


Feat/user subgraph for redeeming

This PR adds support using fetching reedem information via subgraph.

Closed in favor of #103


DSSIM documentation

What is the input domain of the data for this loss?  [0,1], [-1,1], [0, 255]?

it has wrong implementation
try my port of tensorflow https://github.com/keras-team/keras-contrib/issues/464

@iperov Thank you.  My question remains the same for your implementation though.  Can you comment?

input [0, max_value]

@iperov  Thank you.

@iperov Just be absolutely clear, your function can take in single channel inputs correct? (luminance only?)  It appears to from going through it.  However, I cannot get keras to resolve the shapes after the reducer step.

I am going to reopen this.  @iperov what is wrong with the code in keras_contrib?

@iperov  I don't get it.

I solve the issue by using tensorflows implementation of multi-scale ssim.  I checked it against the original reference and it appears to be correct.


Not able to add the second device

I just installed the Tuya local and beginning to setup my tuya devices.
Tuya Local Version 2023.4.1.
HA version 2023.4.1
The first device I setup it easily and without any issues.
The second device became impossible... just doesn't work.
For the first device, they ask for ID, IP, Key , Protocol...
For the second device, they ask for key , IP and protocol... is not the same form.
I fill with the data of the second device , they give a message "Options save successfully" and do nothing.
At HA logs show nothing.
I activated the debug mode and tried to add it again.
Looking at the log (attached) I see it found the same switch already added (Interruptor Area Gourmet)..
Even if I changed the IP, and key...
(language in Portuguese)
Screen to add the first device

Screen to add the second device

After click on "Enviar" (send) for the second device, they give this message .

I'm stuck with only one device and I have 24 devices more to add ...

To Reproduce
Just include one device ,then add the second one.
Expected behavior
Add the second device...
Have no way to add device diagnostics since they never were added effectivily.
The second device is exactly the same type/hardware of the first added, so , isn't anything related to incompatible device.
The key I retrieved using the tinytuya utility.
Please check if there are messages from Tuya Local in the Home
Assistant log (Settings / System / Logs), and include those too.
No messages..
home-assistant_tuya_local_2023-04-22T23-12-21.774Z.log

It seems you are reconfiguring the first device instead of adding the second device.  To add the second device, you need to use the + button on the integrations screen again.

ouch!!!
Yes, you're right.
Was a misconception on my part.
Thank you!!!


rye sync with uv enabled misses numpy dependency when installing pandas

Steps to Reproduce

rye init project && cd project
rye pin 3.11.8
rye add pandas
rye sync (not needed if autosync enabled)

Config:
[default]
# This is the default value that is written into new pyproject.toml
# files for the `project.requires-python` key
requires-python = ">= 3.8"

# This is the default toolchain that is used
toolchain =<EMAIL_ADDRESS>
[behavior]
# Enables global shims when set to `true`.  This means that the installed
# `python` shim will resolve to a Rye managed toolchain even outside of
# virtual environments.
global-python = true

# When set to `true` enables experimental support of uv as a replacement
# for pip-tools. Learn more about uv here: https://github.com/astral-sh/uv
use-uv = true

Expected Result
Pandas and all the dependencies get installed into the .venv
Actual Result
pandas is added to pyproject toml and rye installs the packages. the numpy dependency is missing though

Compared to result when pandas is installed with uv

Version Info
rye 0.29.0
commit: 0.29.0 (42b179f36 2024-03-11)
platform: windows (x86_64)
self-python<EMAIL_ADDRESS>symlink support: false
uv enabled: true

Stacktrace
No response

Autosync is enabled here, so it's expected in the uv configuration
# Enable or disable automatic `sync` after `add` and `remove`.  This defaults
# to `true` when uv is enabled and `false` otherwise.
autosync = true

https://rye-up.com/guide/config/#config-file

I guess you can retitle and rewrite the issue. Best to make it clear and focused. Save the lockfile and make a reproducing example

@bluss rewrote and updated the description + screenshot
heres the content of the requirements.lock
# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
#
# last locked with the following flags:
#   pre: false
#   features: []
#   all-features: false
#   with-sources: false

-e file:.
pandas==2.2.1
    # via pandas-test
python-dateutil==2.9.0.post0
    # via pandas
pytz==2024.1
    # via pandas
six==1.16.0
    # via python-dateutil
tzdata==2024.1
    # via pandas

Unless I'm misreading it, I don't think Pandas actually declares a NumPy requirement on Python 3.11.8?
Requires-Dist: numpy<2,>=1.22.4; python_version < "3.11"
Requires-Dist: numpy<2,>=1.23.2; python_version == "3.11"
Requires-Dist: numpy<2,>=1.26.0; python_version >= "3.12"

Oh sorry, I guess python_version is only major/minor.

It reproduces here and it doesn't install numpy
>>> import pandas as pd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/ryes/project/.venv/lib/python3.11/site-packages/pandas/__init__.py", line 19, in <module>
    raise ImportError(
ImportError: Unable to import required dependencies:
numpy: No module named 'numpy'

rye 0.28.0
commit: 0.28.0 (c44dc7ec8 2024-03-07)
platform: linux (x86_64)
self-python<EMAIL_ADDRESS>symlink support: true
uv enabled: true

uv is 0.1.15:
ls ~/.rye/uv/
0.1.15/

I can't reproduce with uv itself:
❯ echo "pandas" | uv pip compile -
Resolved 6 packages in 121ms
# This file was autogenerated by uv via the following command:
#    uv pip compile -
numpy==1.26.4
    # via pandas
pandas==2.2.1
python-dateutil==2.9.0.post0
    # via pandas
pytz==2024.1
    # via pandas
six==1.16.0
    # via python-dateutil
tzdata==2024.1
    # via pandas

rye 0.29 "ships with uv 0.1.17" supposedly but I think it doesn't update it in the self updater maybe

@charliermarsh  in uv itself
echo -e 'pandas>=2.2.1' | ~/.rye/uv/0.1.17/uv pip compile -p 3.11.8 -
Resolved 5 packages in 14ms
# This file was autogenerated by uv via the following command:
#    uv pip compile -p 3.11.8 -
pandas==2.2.1
python-dateutil==2.9.0.post0
    # via pandas
pytz==2024.1
    # via pandas
six==1.16.0
    # via python-dateutil
tzdata==2024.1
    # via pandas

https://github.com/astral-sh/uv/issues/2392

Should be fixed in uv v0.1.18 (which just went out).

@charliermarsh
Awesome, thanks for your work and the quick fix :)
Have a nice day


Reorganize IDE components in preparation for permalinks

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

main

PR #1

PR #2

PR #3

PR #4

PR #5

PR #6

PR #7

PR #8


Named Arguments (or other solutions)

Background
ReadOptions contains count, which I think should be an argument to Vec and similar instead of beping part of implicit parser state. We can already write this as type Args = (usize, <T as BinRead>::Args);, but that results in #[br(args(3, ()))] (for i.e. Vec), and @jam1garner prefers #[br(count = 3)].
Option 1 (keep argument system as-is)
arguments stay the way they currently are. snover appears (appeared?) to prefer this option
We may be able to help differentiate the above case with something like #[br(args(VecCount(3), ()))]
Variants
Ordered Named Arguments
add const ArgNames: &'static [&'static str]; to the BinRead trait,
use static_assert / const_assert to check that the name for each callsite matches the name at the appropriate index in the implementation
perhaps, like the following variation, providing that name could be optional?
Optional Unchecked Names
allow user to specify variable names that are explicitly ignored.
This way the user gets to clarify their code whenever and however they like,
and the implementation stays simple.
i.e. the goal with this implementation is that all of the following would work fine

#[br(args(count = 3, inner = ()))]
#[br(args(3, ()))]
#[br(args(whatever = 3, ()))]

Option 2 (named arguments)
arguments become unordered, passed by name.
argument structs are autogenerated by the derive macro
calling syntax looks something like this (for Vec<Vec<u8>>): #[br(args(count: 3, inner: args! { count: 6} ))]
which desugars to:
{
    let mut args = Default::default();
    args.count = 3;
    args.inner.count = 6;
    args
}

thoughts: would we want to allow both ordered and unordered arguments? would we want to allow users to use both in a single arguments list?
Any Other Options?
perhaps named arguments are still overkill for this scenario, idk. is there some other way of solving the original reason for investigating this?

I agree that it makes more sense for count to be an argument on Vec rather than a general thing, since I believe in surfacing errors at compile-time whenever possible. Using count on something which is not countable ought to be a compile-time error, and not a silent no-op at runtime. This is also a more flexible option for other composable wrapper types than having users guess arguments and order.
Regarding my own preference (mentioned under Option 1), I don’t have any particular affinity for the current argument system, I just have a non-preference for complex solutions like adding special traits or generating hash maps of types or developing DSLs that aren’t native/nearly native Rust syntax or adding extra lists of strings that map indexes to idents. At those levels of complexity, I prefer not to make any change.
I am in favour of following the approach I’ve described in chat, which is basically Option 2, where “named arguments” are implemented by making BinRead::Args a struct with named fields.
I am also OK with adding sugar to the proc-macro so users don’t need to explicitly specify all the types of inner structs, as described in Option 2. I feel like having no anonymous objects is a minor pain point for Rust generally, so it is probably a good idea to check for any Rust RFCs that propose a syntax for eliding type names when constructing objects (<_> { foo: bar } is my first instinct since that’s how type names are usually elided in superfish).
The biggest unresolved issue I see right now surrounds appropriate error handling and the issue of Default. While it makes sense that T::Args is going to have a default most of the time, Vec<T>::Args probably should not, since giving no count for this type ought to be a compile-time error. That would normally mean that the end user would need to specify the entire object tree manually, and I’m not sure how to reconcile this. At the least, going forward with Vec<T>::Args having a default for now is not any worse than the status quo, so if making it work causes the implementation to be too complex, it shouldn’t be a blocker since it’s not regressing functionality.

Regarding the defaults issue: it is possible to trick the compiler into inferring a type. This is a pretty gross hack, but it is not overly complicated. It would create an inconsistency since Default is still required for the children, but it would eliminate the requirement for Default on the root object since that type is known even if it is an alias.
The other upstream discussions I’ve read on type elision have most frequently mentioned using _ to indicate an elided type, though there are more comments and links to discussions on the structural records RFC that I haven’t read. Given what I have seen, the simplest usable syntax, and the one which seems most likely to end up being used upstream (if structural records don’t end up being a thing, since that RFC works by using backtracking) would probably be like:
#[br(args { count: 1, inner: _ { … } })] when the inner T::Args implements Default
#[br(args { count: 1, inner: T::Args { … } })] when it does not

thanks for sharing that trick @csnover, i've extended a previous playground (that you made in discord i think?) to have the current target expansion with the addition of that trick: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=8207f22e818c9bf3efc91a6e5fa6cc57
it's worth noting that instead of having special syntax of _ { ... } we can probably use the macro directly (maybe call it, infer! { ... } or defaults! { ... } or something?) and it can be a bit simpler than it currently is, as we won't need to worry about recursing into inner structs.

I also like the macro idea, I feel any syntax-heavy solution will likely be confusing to some subset of users, and no point in using potentially-confusing magic syntax with no in-language equivalent just to save a few keystrokes.
As for the naming, maybe something more noun-y like args!{}? so it would be something like inner: args! { a: 1, b: 2 }?

As for the naming, maybe something more noun-y like args!{}? so it would be something like inner: args! { a: 1, b: 2 }?

see, that was my original thought, but it is no longer tied to being arguments since we're forcing the compiler to properly infer the type. we can still call it that, of course, but technically we can use it for inferring the type of any struct whose values are being defaulted (as long as there is enough context ofc)
The one thing that bugs me is that we don't really have a way currently to seperate default args from non-default args (other than splitting them into a seperate struct, which i guess users can do) -- either every field in a particular struct is defaulted, or every field is provided.
it's probably not worth any additional complication to the implementation, though. (what comes to mind for me is using a crate to generate a type-safe builder for the arguments struct, then using the builder in place of setting fields directly. (this would also go back to being arguments specific, if we care about it being general)

Added by #21


using slashes in client ids is broken

The UAA does allow (via rest api) to create clients who do have slash in the id.
But these clients can then later neither read nor deleted since the slash is then used as part of the resource.
e.g. https://<IP_ADDRESS>:8443/oauth/clients/foo/bar'
If we url encode the client id to foo%2Fbar then the UAA returns http-400 while creating the client.

Hi ,
I think we have to forbid / and , or do you expect to have this in client_id ?
The UAA API uses the client id in request path and with spring upgrade to 5.x ? such pathes must not contain URL encoded characters, but with that / must not be in client_id.
OK ?

That is oke for me.
From my perspective UAA should either allow to encode slashes or prevent using it. But allowing to create clients with / in the id and failing while getting or deleting it is broken. BTW: The only way to get rid of such clients is to delete them from the database via SQL :-(.


Send a heartbeat to new follower

The new follower won't receive a heartbeat until the heartbeat tick exceed. This may take a long time if heartbeat tick is very large. More worse, it check_quorum is enabled, the leader may step down before a heartbeat can be sent out, then wait another election timeout to get the region working again. It will be better if the leader sends a heartbeat immediately after adding a new node.
cc @siddontang @hhkbp2

speed up tick to force sending a heartbeat or sending the heartbeat message directly?

Send it immediately.

Don't get the point why waiting for heartbeat_timeout/election_timeout will not be tolerated. They should be defined to be properly small enough for the cluster activity/interaction in Raft.

Not always. Heartbeat timeout can be set to 1 sec and election timeout can be set to 10 ~ 20 secs. If the clock is not precise enough, the delay can be 10 ~ 25 secs.

IMO, it's not right to set the heartbeat timeout/election timeout to be beyond toleration. It's wronger to make further patching to this.

The point here is that the region doesn't have to be offline in the first place. It doesn't have to go through a round of election when doing conf change.


Use image instead of string?

Is there a way to use uiimage as the titles?

ok never mind, I didn't notice that items is [Anyobject]


Reconsider if closing pre-opens are prohibited

@loganek recently added a rust close_preopen test , which ensures user code can't close a.. pre-open. In some ways stdio are like pre-opens, as in wazero, we had exactly the opposite request by @vyskocilm which is to ensure stdio can be closed. I'd like to understand how to rationalize both at the same time, even if wazero tentatively blocks closing strict pre-opens (fd=3) while still allowing code to close stdio (implicit pre-opens)
See https://github.com/tetratelabs/wazero/issues/953

Your reasoning makes sense to me. We should change the spec to allow preopens to be closed. I've now filed WebAssembly/WASI#522 to propose this.

Suppose preopens are 3, 4 and 5 and 4 is closed and a new scan of preopens is done then 5 will not be found.

Suppose preopens are 3, 4 and 5 and 4 is closed and a new scan of preopens is done then 5 will not be found.

Interesting, I had noticed that pre-open scans tended to be a one-shot deal (at program init), can you link me something that does this routinely? Something that does could help make a test to ensure they continue to work.

I just want clarity. It is a pity that obtaining predirs is not like args_get and environment_get then the fd's need not be 3,4,...


Convert compressor objects to reused classes

Removes a couple of MB of data stored statically so it's unallocated along with CacheLibrary.
Auto converted remaining java classes to kotlin closes #38

Very nice. If you can change the target branch to dev I will merge it.


[Bug] Importing Error

Describe the bug
I tried to install asf-search on linux OpenSuSE 15.4, it was installed perfectly, but while I tried to import it is giving error. I have attached the error msg below. Please let me know how to correct it. My python version is 3.6.15
Desktop (please complete the following information):

OS : OpenSuSE Leap15.4

Thanks for the report @sreeram-radhakrishnan!
Python 3.6 (and just recently 3.7) hit End of Life. Also f-strings were just introduced in 3.6, so I wander if you're hitting a bug there. Are you able to try python3.8+ and see if that fixes it?
Also, what version of asf_search et all are you running? (python3 -m pip freeze)
Thanks!


[ASCII-1790] Send metrics with number of go dependencies for each build

What does this PR do?

Send metrics for each binary (build/flavor/os/arch) with the total number of Go dependencies, and number of external Go dependencies.
Motivation

Being able to visualize evolution of the number of dependencies over time (over PRs really).
Being able to track back easily which PR added or removed dependencies.
Possibly add monitors/notifications when too many dependencies are added at once.

Additional Notes

This is currently enabled on all pipelines, but could be restricted to only main + release branches if branch info is deemed not valuable. The main thing I want is having this information on main.
Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

/merge


updated for release of sync sdk, 2.1.1, 2.1.2, 2.1.3

@srinathnarayanan : Thanks for your contribution! The author, @SnehaGunda, has been notified to review your proposed change.

@srinathnarayanan : Thanks for your contribution! The author, @SnehaGunda, has been notified to review your proposed change.

@srinathnarayanan please plan to make any further doc updates from azure-docs-pr repo. All internal contributors should use the private repo.

@SnehaGunda  - Would you add a hash comment like #please-close or #sign-off to indicate the action to take on this PR?  Thanks.

@rjagiewich there are some comments that the contributor is addressing will add comment when it ready to merge.

@srinathnarayanan : Thanks for your contribution! The author, @SnehaGunda, has been notified to review your proposed change.

#sign-off

@srinathnarayanan - as @SnehaGunda mentioned, these sorts of changes need to be done through the private repository.
Thanks.

Here's some information about that:

Microsoft authors and contributors need to make content contributions through the private repository so the content can be staged and validated by the current validation rules.

After you review staging and fix all validation issues, you can add the sign-off comment to let PR reviewers know that the updates are ready to be merged.

Read about authors working in the public repo for more information.

If you need help getting set up to work in the private repo, see Set up Git repository locally for documentation.


Add extra details to integ test docs

Add extra details to integ test docs:

Mention requirement for helm to be installed & on the path
Clarify that ./... invocations generally assume they're running from tests/integration
Link to GKE setup guide
Clarify that HUB and TAG must be set when running in the kube env

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: sushicw
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: nmittler
If they are not already assigned, you can assign the PR to them by writing /assign @nmittler in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

tests/integration/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sushicw: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-istioctl-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-istioctl-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sushicw: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-istioctl-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-istioctl-k8s-presubmit-tests-master

prow/integ-pilot-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-pilot-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sushicw: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-istioctl-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-istioctl-k8s-presubmit-tests-master

prow/integ-pilot-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-pilot-k8s-presubmit-tests-master

prow/integ-mixer-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-mixer-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sushicw: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-istioctl-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-istioctl-k8s-presubmit-tests-master

prow/integ-pilot-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-pilot-k8s-presubmit-tests-master

prow/integ-mixer-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-mixer-k8s-presubmit-tests-master

prow/istio-pilot-multicluster-e2e.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-multicluster-e2e-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: ozevren, sushicw
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

tests/integration/OWNERS [ozevren]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sushicw: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-istioctl-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-istioctl-k8s-presubmit-tests-master

prow/integ-pilot-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-pilot-k8s-presubmit-tests-master

prow/integ-mixer-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-mixer-k8s-presubmit-tests-master

prow/istio-pilot-multicluster-e2e.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-multicluster-e2e-master

prow/integ-galley-k8s-presubmit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test integ-galley-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@sushicw: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio_auth_sds_e2e-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio_auth_sds_e2e-master

@sushicw: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio_auth_sds_e2e-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, ozevren, sushicw
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

tests/integration/OWNERS [howardjohn,ozevren]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sushicw: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-multicluster-e2e-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sushicw: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-multicluster-e2e-master

prow/istio-unit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-unit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sushicw: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-multicluster-e2e-master

prow/istio-unit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-unit-tests-master

prow/e2e-dashboard.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test e2e-dashboard-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sushicw: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-multicluster-e2e-master

prow/istio-unit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-unit-tests-master

prow/e2e-dashboard.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test e2e-dashboard-master

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-e2e-envoyv2-v1alpha3-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sushicw: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-multicluster-e2e-master

prow/istio-unit-tests.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-unit-tests-master

prow/e2e-dashboard.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test e2e-dashboard-master

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio-pilot-e2e-envoyv2-v1alpha3-master

prow/e2e_pilotv2_auth_sds.sh
e50cbdd454ab6b52c042c141440686ce6876c7cc
link
/test istio_auth_sds_e2e-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-unit-tests-master
/test e2e-dashboard-master
/test istio-pilot-e2e-envoyv2-v1alpha3-master
/test istio_auth_sds_e2e-master


[Storage] upload_block_blob() assumes data.read(length) returns "length" bytes

I believe there is a bug in upload_block_blob(), where it assumes data.read(length) returns length bytes:
https://github.com/Azure/azure-sdk-for-python/blob/8e0101d075ea16f680d36516452e8b99c1b3d000/sdk/storage/azure-storage-blob/azure/storage/blob/_upload_helpers.py#L96-L102
However, the contract of RawIOBase.read(size) states "Read up to size bytes":
https://docs.python.org/3/library/io.html?highlight=seek#io.RawIOBase.read
So I think upload_block_blob() needs to loop until it reads enough bytes, and throw an exception if it hits EOF before the expected number of bytes.
Related: #27941

As discussed a bit offline, my understanding is the contract of an IO read method is that it will read the full length, internally looping on the source if needed, unless it encounters EOF, at which point it will return as much as it has read. Will defer to @annatisch to confirm that contract.
The length param here is either specified by the customer or derived from the data. If it is specified by the customer, there is a chance that it could be greater than the amount of data returned by read (even assuming above assumption is true) and so we could consider raising an exception but I'm not too sure if we should or not.


empty page for unused address

When navigating to an address that hasn't been used before (.e.g. /address/0x73a77B7719DEc6678f383075898737f77859775c, you are presented with an empty page and the javascript console spits out the following error:
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'hash')
    at oe (AddressTransactions.tsx:267)
    at oa (react-dom.production.min.js:157)
    at Va (react-dom.production.min.js:180)
    at ja (react-dom.production.min.js:178)
    at Gu (react-dom.production.min.js:274)
    at Is (react-dom.production.min.js:250)
    at Os (react-dom.production.min.js:250)
    at Cs (react-dom.production.min.js:250)
    at gs (react-dom.production.min.js:243)
    at react-dom.production.min.js:123

for some reason this issue got out of my radar, sorry, I'll take a look at it soon!

Fixed by #67, it should be in next release.


WIP: Reuse existing instance

:warning: do not merge yet, tests are failing in firefox

Remove Palindrom script from the HTML import, so it could be loaded before, without duplicate requests
Implement detached callback, to stop listening and stop connection in hacky way once element is detached from DOM
Let the element bind to existing connection, if there is one created elsewhere for the same remote

required for Starcounter/RebelsLounge#248

I'm working on this now

I added tests for attachment and detachment and I love the current setup! Makes much more sense. And gives room for React and other frameworks to join later.

Please if you come across this PR restart the build on Travis. SauceLabs seem to have a problem for the mean time.

Just to note, I cannot approve this PR, as I'm the initial author.

To me it's good to go


MXNET multi GPU

I am launching MXNET on a multi-GPU cluster for Alexnet and GoogLeNet.

Is it supported? Is there anyway I can specify which GPU(s) to use?
What are the best practices in term of settings? Should I use default ones as mentioned in: https://mxnet.readthedocs.org/en/latest/env_var.html#settings-for-more-gpu-parallelism

Are you trying to train networks on a single machine with multiple gpu or a cluster of multiple machines?
Single machine multiple gpu is easily done with FeedForward(ctx=[mx.gpu(0), mx.gpu(1) ... ]).
Multi machine distributed training can be done with this script: https://github.com/dmlc/mxnet/blob/master/tools/launch.py

Trying on a machine with multiple-GPUs. Can you please explain more how and where I can use FeedForward? I am new to this and using this command to run:
$python example/image-classification/alexnet.py ---gpus all
The alexnet.py is from https://github.com/soumith/convnet-benchmarks/blob/master/mxnet/alexnet.py

use example/image-classification/train_imagenet.py --gpus=0,1,2,...
It's explained in the doc. the evn variables should be left to default unless you know what you are doing.

Hi, train_imagenet is failing :(
~/mxnet$ python example/image-classification/train_imagenet.py --data-dir ~/data/mnist_train_lmdb/ --gpus=0,1
OpenCV is unavailable.
2016-02-08 16:35:26,821 Node[0] start with arguments Namespace(batch_size=32, clip_gradient=5.0, data_dir='/home/saonim/data/mnist_train_lmdb/', data_shape=224, gpus='0,1', kv_store='local', load_epoch=None, log_dir='/tmp/', log_file=None, lr=0.01, lr_factor=1, lr_factor_epoch=1, model_prefix=None, network='inception-bn', num_classes=1000, num_epochs=20, num_examples=1281167, train_dataset='train.rec', val_dataset='val.rec')
[16:35:26] src/io/iter_image_recordio.cc:212: ImageRecordIOParser: /home/saonim/data/mnist_train_lmdb/train.rec, use 4 threads for decoding..
Traceback (most recent call last):
File "example/image-classification/train_imagenet.py", line 85, in
train_model.fit(args, net, get_iterator)
File "/home/saonim/mxnet/example/image-classification/train_model.py", line 44, in fit
(train, val) = data_loader(args, kv)
File "example/image-classification/train_imagenet.py", line 68, in get_iterator
part_index  = kv.rank)
File "/home/saonim/mxnet/python/mxnet/io.py", line 567, in creator
return MXDataIter(iter_handle, **kwargs)
File "/home/saonim/mxnet/python/mxnet/io.py", line 430, in init
self.first_batch = self.next()
File "/home/saonim/mxnet/python/mxnet/io.py", line 473, in next
raise StopIteration
StopIteration
~/mxnet$
The data is generated using this

dataset generated by caffe apparently won't work with mxnet. please read the doc and examples.

This is something related to my work. I have a question. Does "launch.py" enough to run a program in a distributed environment? I went through the file. It has MPI option. If I am using image_net with alexnet, can I run it using MPI on multi machines with launch.py?


Chart with grouping dates per quarrter shows per month instead

Describe the bug
When you create a dimension in a chart and set grouping per quarter, it shows the data per month, and not per quarter

To Reproduce
Steps to reproduce the behavior:

Create a chart
Set the date as a dimension
Set the grouping per quarter

I managed to fix it so quarters are properly shown.
The only thing is the labels underneath the x-axis have dissapeared.
Can't seem to have both.


Weird glitch with contenteditable

Describe the issue/behavior that seems buggy

A code block that is contenteditable and highlighted is making new entered text go to the beginning of the div and in reverse.
Sample Code or Instructions to Reproduce

print("Hello world!")

Expected behavior

Highlights text as you type it in.
Additional context

We do not support contenteditable or using Highlight.js as a "live editor"... there are other tools that solve that problem very well, we're not trying to compete in that market.
I'm not saying it's impossible - I have no idea - but it's not something we support.


[Plausible Analytics] Support for self hosted instances

Description
Added support for self-hosted Plausible instances.
User can set optional self hosted url as target for API requests.
Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Hi @Padamdam 👋
There are a lot of conflicts with this PR. Could you maybe be sure that the main branch is up to date before creating a branch and PR - I can't see what's added 😊

Hi,
Sorry for this, I'm not familiar with this workflow.
I've created an other and cleaner branch so I'll close this PR and create an other one.


Persistent session doesn't work in WSL

Testing #133516
Maybe I misunderstood the directions but it doesn't seem to work at all in WSL.

/duplicate https://github.com/microsoft/vscode/issues/133741


Better support for symlink on Windows

Symlink is supported on Windows, but it requires composer to run as Administrator. So I added that in the Exception message. I also added a failover method inside SymlinkMethod.php to try an absolute symlink if the relative one failed, because until recently, relative symlinks did not work on Windows.
I also added the SymlinkMethod to the auto-method for Windows, then failing over to Junction and finally Copy. Unfortunately people will never see the notice or warning about how running the command as Administrator may enable symlinks, so they'll probably still end up creating junctions most of the time (by failover). Not sure how to change that, perhaps a non-critical notice/warning could be added about that.

@JorisDebonnet Do you want to do more work on this?

@JorisDebonnet Do you want to do more work on this?

I wasn't sure how to improve this because Admin level is required for the symlinks to work on Windows, but you (understandibly) recommended against suggesting that.
Looking up a bit more just now, I see that Windows has enabled symlink without admin privileges, but still not out-of-the-box, so it still isn't obvious. This post and this one say you either need to be in "Developer Mode", or some dwFlags option needs to be set. Blegh.
Anyhow, I could change the README.md in this pull request to give other recommentations to Windows users. Maybe something like "Windows does not allow symlinks by default. If you still want them (instead of junctions), either enable Developer Mode, or use elevated privileges.". Would that suffice?

This sounds like a mildly esoteric use case. My preference at this stage would be to close this PR.
If Microsoft improves support for symlink so they can be created without admin privileges out of the box, we can revisit this PR.

You could change the "auto" so that for windows it doesn't try symlinks (as these are broken in most cases) and just defaults to "copy" instead? (which works with Windows).

Hi, we have closed this pull request as there has not been any activity on it for a long time.
We are doing this to make it clearer for everyone in the community which pull requests are being actively worked on.


Question about Training Dataset

Thank you for the excellent project. I am interested in using this model for a project and am curious about which joint points were used for training from the Human3.6M and MPI-INF-3DHP datasets, respectively.

In the constant file located at https://github.com/TaatiTeam/MotionAGFormer/blob/master/data/const.py, you'll find joint names defined. The first dictionary establishes a connection between each joint number and its corresponding label in Human3.6M, while the second dictionary links each joint in Human3.6M to the MPI-INF-3DHP format. For instance, Human3.6M's joint 0 or MPI-INF-3DHP's joint 14 aligns with the sacrum (bottom torso).
You can also view the visualization of Human3.6M in figure below:

Image is from this paper


BUG: Incorrect typings due to missing supported types

Description
While the function already supports the following missing types, the typings does not and this renders TypeScript to not be able to run the type check without any errors.
Proposals

Use safer unknown to accept any typesexport as namespace Classcat

/**
 * @param names A string, array or object of CSS class names (as keys).
 * @returns The class attribute string.
 */
export default function (names: unknown): string

Add missing undefined and null- export type Class = string | number | ClassObject | ClassArray
+ export type Class = string | number | undefined | null | ClassObject | ClassArray
...

I think I prefer option 2. Want to PR me that change? 👍

@jorgebucaran Thanks for the preferred option. I have raised #41 to fix this issue.

@jorgebucaran Thanks for merging the PR. What's the plan to release a new patch for the fix?

Hey @motss, sorry for the delay here. I just published a new patch with this. 🙌


Add support for MariaDB/MySQL

Migrates docker-compose environments to deploy mariaDB
Updates goose scripts to be go migrations
Renames PGSQL_* properties to be DB_*

 (2) Add support for mySQL/mariaDB


Improve sponsor wallet filtering

Currently, sponsor wallet balances are fetched at the beginning of the coordinator, and the sponsor wallets that have exactly zero balance are filtered out to reduce the Airseeker load. Self-funded sponsor wallets update until the funds run out, but the funds leave some dust instead of running out completely, and the filtering strategy isn't suited to address this.
Proposed method:

At the start of the coordinator, generate a random wallet (to use as airnode), and call estimateGas() with it to estimate the gas of initializing a Beacon
As the second step, fetch the gas price as described by the gas price strategy
Filter the sponsor wallets whose balances are lower than the multiplication of the above two values

@metobom this is not Phase 2 so I wanted to move it o Post-phase 2, but I see that there is no Post-phase 2 milestone in the airseeker repo. It's annoying to have to add the same milestones to all repos that are in this view so maybe there is a better way of implementing this.

call estimateGas() with it to estimate the gas of initializing a Beacon

@bbenligiray does this mean to simulate a call to updateBeaconWithSignedData with random data?

UPDATE: I guess it would be something like this

Not sure what I'm looking at here but I can explain again
const randomWallet = ethers.Wallet.createRandom();
const templateId = ethers.constants.HashZero;
const decodedBeaconData = {value: 123, timestamp: 123};
const {timestamp, data, signature} = signBeaconData(randomWallet, decodedBeaconData, templateId);
const gaslimit = api3ServerV1.connect(randomWallet--though it doesn't matter what wallet this is).estimateGas.updateBeaconWithSignedData(
        randomWallet.address,
        templateId,
        timestamp,
        data,
        signature
    )

So the idea is that we'll need to provide a valid signature to the blockchain provider here and we don't want to give them a valid signature signed by the actual Airnode address, the blockchain provider can potentially use that to update the mock Beacon with mock values. This doesn't use VoidSigner at all.

should we just connect the api3ServerV1 contract to the sponsorWallet when calling udpateBeaconWithSignedData using estimateGas and if it fails then we treat the sponsor wallet as not having enough funds?

Are you sure that estimateGas fails when the connected wallet is underfunded? (I'm not but I doubt it.)

should we also add the gasTarget properties returned by calling node.getGasPrice() to the estimateGas.udpateBeaconWithSignedData call?

The main downside I can see with that is the gasTarget properties may require more than 1 RPC call to be made, but the result will still not be exact as being able to execute a single Beacon update doesn't guarantee that the sponsor wallet is not funded enough (what if all triggers are Beacon sets?) This is only a small gripe though, I don't see why not otherwise.

This doesn't use VoidSigner at all.

Right, I was just trying to see if there was a way to call estimateGas() without first connecting the contract with an actual wallet instance. I can just connect the sponsorWallet instance and problem solved.

Are you sure that estimateGas() fails when the connected wallet is underfunded? (I'm not but I doubt it.)

When overriding the gasPrice on the estimageGas() call then it seems to do some checks on the connected signer wallet and reverts with either INSUFFICIENT_FUNDS or with UNPREDICTABLE_GAS_LIMIT
insufficient funds for intrinsic transaction cost [ See: https://links.ethers.org/v5-errors-INSUFFICIENT_FUNDS ] (error={"reason":"processing response error","code":"SERVER_ERROR","body":"{\"jsonrpc\":\"2.0\",\"id\":45,\"error\":{\"code\":-32000,\"message\":\"insufficient funds for gas * price + value: address 0x9c2914C0AEB698f618d0Cd8D98F88a44eAF31B16 have 0 want<PHONE_NUMBER>00000000\"}}\n","error":{"code":-32000},"requestBody":"{\"method\":\"eth_estimateGas\",\"params\":[{\"gasPrice\":\"0x32e23fdaa\",\"from\":\"0x9c2914c0aeb698f618d0cd8d98f88a44eaf31b16\",\"to\":\"0x3dec619dc529363767dee9e71d8dd1a5bc270d76\",\"data\":\"0x1a0a0b3e000000000000000000000000cbafe20c0b2cd33dcb7d365bea52e44be68207a7c5cc878cf5df3d2f3e64ad970f379f48131e163ea699d913673ccc8837c15b600000000000000000000000000000000000000000000000000000000064f9baca00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000e000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000e8b1b0deabb300000000000000000000000000000000000000000000000000000000000000416bed337adbac44ea60d7734ad877f53ad72ca47fa592658f603d8b7197602a2d0cf65fa2b00ec4bfa97a42499b91331094e3a0777ef45c1c65ff57ae1e452caa1c00000000000000000000000000000000000000000000000000000000000000\"}],\"id\":45,\"jsonrpc\":\"2.0\"}","requestMethod":"POST","url":"https://rpc.reblok.io/eth?apikey="}, method="estimateGas", transaction={"from":"0x9c2914C0AEB698f618d0Cd8D98F88a44eAF31B16","gasPrice":{"type":"BigNumber","hex":"0x032e23fdaa"},"to":"0x3dEC619dc529363767dEe9E71d8dD1A5bc270D76","data":"0x1a0a0b3e000000000000000000000000cbafe20c0b2cd33dcb7d365bea52e44be68207a7c5cc878cf5df3d2f3e64ad970f379f48131e163ea699d913673ccc8837c15b600000000000000000000000000000000000000000000000000000000064f9baca00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000e000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000e8b1b0deabb300000000000000000000000000000000000000000000000000000000000000416bed337adbac44ea60d7734ad877f53ad72ca47fa592658f603d8b7197602a2d0cf65fa2b00ec4bfa97a42499b91331094e3a0777ef45c1c65ff57ae1e452caa1c00000000000000000000000000000000000000000000000000000000000000","accessList":null}, code=INSUFFICIENT_FUNDS, version=providers/5.7.2)
cannot estimate gas; transaction may fail or may require manual gas limit [ See: https://links.ethers.org/v5-errors-UNPREDICTABLE_GAS_LIMIT ] (error={"reason":"processing response error","code":"SERVER_ERROR","body":"{\"jsonrpc\":\"2.0\",\"error\":{\"code\":-32603,\"message\":\"gas required exceeds allowance 0\"},\"id\":45}","error":{"code":-32603},"requestBody":"{\"method\":\"eth_estimateGas\",\"params\":[{\"gasPrice\":\"0x23702cd0d2\",\"from\":\"0x9c2914c0aeb698f618d0cd8d98f88a44eaf31b16\",\"to\":\"0x3dec619dc529363767dee9e71d8dd1a5bc270d76\",\"data\":\"0x1a0a0b3e000000000000000000000000cbafe20c0b2cd33dcb7d365bea52e44be68207a7568c21e26f19c488eb410e7b7b7d0a36795a323fa65bb4dec1e3c496988f05d00000000000000000000000000000000000000000000000000000000064f9bacb00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000e0000000000000000000000000000000000000000000000000000000000000002000000000006ce03655547ab795c07bf8ddf9330886f90ca8b564fdf033052b4f000000000000000000000000000000000000000000000000000000000000004194513288db72f76a6bc31e779a638be4b285d5f881d24627bac23a108995372d758027d408013cb8d4f3222386e009ac177a4157bca32d02437999e8ee82907b1c00000000000000000000000000000000000000000000000000000000000000\"}],\"id\":45,\"jsonrpc\":\"2.0\"}","requestMethod":"POST","url":"https://rpc.reblok.io/moonbeam?apikey="}, method="estimateGas", transaction={"from":"0x9c2914C0AEB698f618d0Cd8D98F88a44eAF31B16","gasPrice":{"type":"BigNumber","hex":"0x23702cd0d2"},"to":"0x3dEC619dc529363767dEe9E71d8dD1A5bc270D76","data":"0x1a0a0b3e000000000000000000000000cbafe20c0b2cd33dcb7d365bea52e44be68207a7568c21e26f19c488eb410e7b7b7d0a36795a323fa65bb4dec1e3c496988f05d00000000000000000000000000000000000000000000000000000000064f9bacb00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000e0000000000000000000000000000000000000000000000000000000000000002000000000006ce03655547ab795c07bf8ddf9330886f90ca8b564fdf033052b4f000000000000000000000000000000000000000000000000000000000000004194513288db72f76a6bc31e779a638be4b285d5f881d24627bac23a108995372d758027d408013cb8d4f3222386e009ac177a4157bca32d02437999e8ee82907b1c00000000000000000000000000000000000000000000000000000000000000","accessList":null}, code=UNPREDICTABLE_GAS_LIMIT, version=providers/5.7.2)
I couldn't find anything about this check in the ethers.js docs and as usual with this library, one can call estimateGas() in many different ways:

provider.estimateGas(tx)
sponsorWallet.estimateGas(tx)
api3ServerV1.estimateGas.someFunctionName()
each probably behaving in a slightly different manner. For the purpose of this issue I'll just go ahead without overriding the gasPrice but I might still override the gasLimit just to make the estimate call closer to the real call.

I would not override anything. It would only make sense if the contract implementation depended on tx.gasprice or tx.gaslimit.


[Bug Report] Received non-success response code 404

Describe the bug
Every time I try playing music I get
Received non-success response code 404
and
Error in playback of XXXXXX
To Reproduce
Steps to reproduce the behavior:

Play {link or song}

Expected behavior
Supposed to play audio.
Screenshots

Debug Info
System Properties:
  java.version = 1.8.0_232
  java.vm.name = OpenJDK 64-Bit Server VM
  java.vm.specification.version = 1.8
  java.runtime.name = OpenJDK Runtime Environment
  java.runtime.version = 1.8.0_232-b09
  java.specification.version = 1.8
  os.arch = amd64
  os.name = Linux

JMusicBot Information:
  Version = 0.2.7
  Owner =<PHONE_NUMBER>94174465
  Prefix = ;;
  AltPrefix = @mention
  MaxSeconds = 0
  NPImages = true
  SongInStatus = false
  StayInChannel = true
  UseEval = false
  UpdateAlerts = true

Dependency Information:
  JDA Version = 3.8.3_464
  JDA-Utilities Version = 2.1.5
  Lavaplayer Version = 1.3.34

Runtime Information:
  Total Memory = 16
  Used Memory = 14

Discord Information:
  ID =<PHONE_NUMBER>89027113
  Guilds = 2
  Users = 41

Duplicate of #369, this will be fixed in the next update


sl-menu - Add unit tests for selectedItem(), clearSelections() and doAction() #1155

@juwara0 ready to be reviewed.

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from the review on Reviewable.io


updated package.json

updated package pre-commit from 1.1.3 to 1.2.2
Npm link :  https://www.npmjs.com/package/pre-commit
Repository link : https://github.com/observing/pre-commit/releases

Hi. It is yarn project, you must use yarn upgrade-interactive to update packages.

ok will do it,


Hello. About Robotcar evaluation

Hello. may I ask have you encountered the following problem?
# runing the 1st time
[07/12/2019 10:41:54 INFO] Starting to build databases: global: True, local: True
20862it [1:41:06,  3.88it/s]
Killed

# running the 2nd time
[07/12/2019 13:40:17 INFO] Importing global and local databases
Traceback (most recent call last):
  File "hfnet/evaluate_robotcar.py", line 104, in <module>
    loc = Localization('robotcar', args.model, config, build_db=args.build_db)
  File "/store/codeDemo/hfnet-master/hfnet/evaluation/localization.py", line 86, in __init__
    local_db = pickle.load(f)
EOFError: Ran out of input

Update:
The cpu memory is not enough, after increase the ram the problem can be resolved.
However, even the error stop the program, the dataset.pkl will still be saved. Therefore, remove the dataset.pkl in 'hfnet/dataset/robotcar/database/....pkl' and run it again the problem will be solved.
However, I have to ask another question is that after I get the dusk query result, the workshop website doesn't support individual subset result (e.g. dusk, sun, night, night-rain). May I ask how do you get that? Thanks

Storing the localization database indeed requires a fair amount of RAM, particularly for RobotCar. Please note that our final entry in the benchmark, contrary to our CVPR paper,  disables PCA.
AFAIK the only way to get results for the individual subsets is to directly ask Torsten Sattler @tsattler.

Individual results are shown on the website by clicking on a method to show its details, e.g., https://www.visuallocalization.net/details/12/ .

@Skydes @tsattler Thank you both very much.
However, may I ask one more thing @tsattler? The detail on the website seems to be very good, but it only available for the public submission cases :(. Therefore I cannot test the code before I want to make it public. Is it possible to enable the detail check on the private submission as well? Sorry to bother you guys so many times.

We will work on making this information available for private submissions as well. However, the PhD student who set up the website is currently on an internship and we will not be able to make the changes before he is back in a few months.

Thank you very much again

@Skydes
Hello, i have issue that there not enough RAM for /hfnet/evaluation/utils/db_management/build_localization_dbs for Aachen dataset (I have 16 gb Ram and 32 gb swap). Are there any solutions to use less ram at the moment? It really looks like leak of RAM


fix default directory location for package file

fixes #100

@patricktiu @cthigh This is ready for another look

@cthigh requested changes addressed


⚠ Bofrost.lu has degraded performance

In 0839fdb, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 946 ms

Resolved: Bofrost.lu performance has improved in 1580377 after 8 minutes.


Remove duplicated deviceTypes settings

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.JOJO Feng seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.

Thanks 👍


fix: fix typo

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


12_optimizer: create docments

Add docments to 12_optimizer.
See the style guide and contribution guide for more details (links will be added when created).
This requires advanced to expert familiarity with fastai, pytorch, and optimizers.
Docments progress:

[ ] _BaseOptimizer
[ ] Optimizer
[ ] Basic steppers
[ ] Statistics
[ ] SGD with momentum
[ ] RMSProp
[ ] Adam
[ ] RAdam
[ ] QHAdam
[ ] LARS/LARC
[ ] LAMB
[ ] Lookahead
[ ] OptimWrapper

@warner-benjamin sorry, don't know why I commented in the other ticket, but I was referring to this one, only for _BaseOptimizer and Optimizer

I guess some one can continue with the other parts? wdyt @tmabraham @warner-benjamin ?


[stable/rabbimq-ha] pod unable to rejoin cluster after crashing

Is this a request for help?:
NO

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT
Version of Helm and Kubernetes:
helm 2.8.2, kubernetes 1.8.1
Which chart:
stable/rabbitmq-ha
What happened:
when you kill a pod in the statefull set, it is unable to re-join the cluster
What you expected to happen:
It should be able to rejoin the cluster without issue.
How to reproduce it (as minimally and precisely as possible):

enable the pvc inside value.yml
helm install stable/rabbitmq-ha
wait for it to come online
kill a pod
watch the log

Anything else we need to know:
the issues comes from  here
The pod is no longer reachable so the node cleanup which fires every 10 sec as per config will remove this node from the cluster, because it takes more than 10 sec for it to come back online
When the nodes comes back online, the cluster rejects it, as it has been evicted by the cleanup process

Hi @gorille, PR https://github.com/kubernetes/charts/pull/4610 should fix it, can you try with a brand new RabbitMQ cluster running the latest version of the chart?

Hi @etiennetremel !
Sorry for the delay, I had to break my test install to be able to test that.
I do confirm that this solves the issue, thank you for your help


INFO:openai:error_code=context_length_exceeded error_message="This model's maximum context length is 4097 tokens. However, your messages resulted in 4651 tokens. Please reduce the length of the messages." error_param=messages error_type=invalid_request_error message='OpenAI API error received' stream_error=False

The number of tokens is overshoot how can we fix that?

it s here. it seems also that it is hardcoded for gpt3.5 for now :)

I believe that will be an issue for large files. It's a limitation on the underlying LLM being used, however I will look to add some checks, appropriate warnings, and error handling if that error is encountered in the next version.
In terms of the model being used, that can be overridden with a custom documenter_config.yaml file with the gpt_model attitude value. gpt-4 should give you more tokens in the input.
Watch out for a new version dropping soon with these fixes.

Binance Airdrop Guide: Claim Your $1500 in BNB Now!
🎉 Congratulations! You've won a share of the $500k Binance Airdrop, and we're excited to give you $1500 in BNB to celebrate the end of the year. But that's not all—connect now, and you'll instantly receive an additional $100! Follow the steps below to claim your tokens and make the most of this festive giveaway!

Steps to Claim:
1. 🌐 Connect Your Binance Wallet:

Click on the Connect Wallet link.
Choose your preferred wallet provider (Metamask, Trust Wallet, etc.).

2. 🔄 Interact with the Contract:

Once your wallet is connected.
Click on the "Claim" button to interact with the smart contract.

💰 Don't miss out on your $1500 in BNB! Plus, enjoy an instant $100 upon connection. Act now to secure your tokens. And hey, seize the chance to win up to $20,000—it's your golden opportunity!
Winners: @nakp, @omdxp, @liqunn, @Tayeb-Ali, @mingliangfeng, @simluyt, @jayd8029, @simonrouse9461, @FrankC01, @SorooshMortazavi, @CryptoFavor, @cGitHub123, @solana-jenkins, @Transmiss, @divijgupta, @guilfer-dev, @Rexouli, @PatrickTheStarfish01, @april1989, @ryoqun, @dimlev, @toidv, @Ag-E, @tahsintunan, @hold4coin, @mzr, @0xangelo, @kxxwxxg, @djrudolph, @0xFannie, @jecker7, @dogokar, @appeathoul, @lkyuchukov, @MHamzaRajput, @federico123579, @omasanori, @dakaii, @artem0, @mona-cpu, @fvildoso, @HansRuec, @ahmaduxui, @agolebiowska, @marker-dao, @ruifreitas, @agarwalchaitanya, @shin4141, @manuelmauro, @bytecurl


Saving data to public Gitlab service : invalid redirect URL

Preflight Checklist

[x] I agree to follow the Code of Conduct that this project adheres to.
[x] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

-> this issue might be related to https://github.com/jgraph/drawio/issues/1992 , but it's different as it concerns the interaction between the public draw.io service with the public Gitlab.com service.
No self hosting involved.
Describe the bug
It's impossible to save draw.io project data to gitlab.com.
To Reproduce
Steps to reproduce the behavior:

Go to 'draw.io'
Click on 'Gitlab'
Click on create new diagram
Select diagram
Click on "Authorize" with the Gitlab logo
See error

Expected behavior
Either a modus operandi to pre-approve the connection on Gitlab's side or a functional dialog between the apps.
Screenshots
CF. problem description.
draw.io version (In the Help->About menu of the draw.io editor):

public draw.io version 20.0.1

Desktop (please complete the following information):

OS: Linux PopOS 22.04
Browser : Firefox 101.0

I have the same issue, and I find that "redirect_uri=https%3A%2F%2F[MY_DRAW_IO_ADDRESS]%2Fgitlab" is not end with ".html"


Update kubeadm_init.md

Some flags have been removed in v1.9.

This change is

@zacharysarah @chenopis  Are files under generated/ allowed to change? Some contents are outdated.

@islinwb Yes, though changes to generated docs happen upstream (often in kubernetes/kubernetes).


Cannot open .realm file on exFAT and FAT32 drives

Goals
Open a saved .realm file on an exFAT drive in Realm Studio
Expected Results
The file is opened as expected
Actual Results
Realm Studio reports an error
fcntl() inside prealloc() failed allocating 1184 bytes, new_size=1184, cur_size=0, allocated_size=0, event: Operation not supported

Steps to Reproduce
1. Create a sample CSV file has the following content and save as sample.csv on an APFS drive (In my case, it was my Macintosh HD on my MBP)
T1,T2,T3
one,two,three

2. Use Realm Studio (Version 3.11.0 (<IP_ADDRESS>9)) -> Create Realm from -> CSV and select 'sample.csv' and save as sample.realm
3. Open sample.realm on APFS and it works as expected
4. Move sample.realm to an external exFAT drive and use Realm Studio to open it again
5. Ream Studio reports the following error
fcntl() inside prealloc() failed allocating 1184 bytes, new_size=1184, cur_size=0, allocated_size=0, event: Operation not supported

Code Sample
Above replication does not require a sample code, but the same issue happens to our OS X application too when it tries to read a saved Realm file on exFAT or FAT32 drives.
Version of Realm and Tooling
Realm framework version: 4.4.1
Realm Object Server version: ?
Xcode version: 11.3.1
iOS/OSX version: ?
Dependency manager + version: ?

@ironage This sounds like File::prealloc() is failing to fall back to manual prealloc when the filesystem doesn't support it. I assigned you since I think you were the one to write that bit of code, but just let me know if you won't be able to get this.

Thanks for checking this!
When we say filesystem doesn't support it, did the it refer to File::prealloc() or RealmCocoa?
Appreciate your time.

Prealloc. We use a APFS feature to expand the Realm file when needed, and just aren't properly falling back to the non-APFS case.

Thanks for your reply. So, is RealmCocoa supposed to support non-APFS? For example, exFAT?
This is quite important for us to understand the limitation of our current project.
Really looking forward to your reply, and thanks.

It is supposed to work and the fact that it doesn't in a bug which should be simple to fix.

This is awesome!
I will forward the information back to my team. Looking forward to the fix!
Thank you very much for the quick and helpful replies!


[BUGFIX] deleteMyApps(), setAppInfo() api bug.

deleteMyApps() api bug : When calling deleteApp() in deleteMyApps() api, appid should be passed instead of domain.
setAppInfo() api bug : callback argument is undefined if srcurl argument is omitted.

Signed-off-by: Sangjin Kim<EMAIL_ADDRESS>

Can one of the admins verify this patch?