Not using --recursive causing problems?

Hey there,
Thanks for making such a fantastic project! Just a quick question -- I'm setting up a existing large project that formerly compiled code in order to get coverage so that it works with coffee-coverage. However, I'm finding that the report generated shows little to no actual coverage at all, while the compiled report was quite high and the project is tested thoroughly. I have followed instructions exactly, and it's not erroring out, just giving an incorrect report. The one thing I am not including is the --recursive flag, as it's not needed to run my test suite. I tried including this, but then mocha tried to run some of the fixtures included in the tests folder and errored.
Is the --recursive flag necessary to generate correct coverage? If not, what further information could I provide to make this incorrect coverage issue easier to shed light on? And if so, how do people typically deal with fixtures being run as tests with the recursive flag?
Thanks again!

--recursive shouldn't be needed for mocha (unless you actually have tests that need it, obviously.)  I wrote a really, really high level overview of how coffee-coverage works over here if that's at all helpful.
The obvious thing to check off the top of my head is; Do your unit tests run straight from your .coffee files, or are they loading your compiled .js files?  If you're running from compiled code, and trying to use run-time instrumentation, the coffee-coverage never has a chance to get in there and do it's thing.

The tests run straight from coffee files. The entire project is coffee actually, there's no js and no compiling (except for a task I use to convert before publishing, which was formerly used for js instrumentation before I switched to coffee-coverage).\
This is the project, if that helps at all. package.json points to the lib folder and loads up index.coffee.

Just gave this a quick try, and I'm seeing statement coverage of 40.2% (361 / 898), which I'm guessing is too low?  (I'm also seeing 100 tests passing and 9 pending.)

I'm wondering if this:
--require test/support/all
--require test/support/helpers

is the problem.  I'm wondering if mocha is somehow loading these files without coverage...

If I do:
--require coffee-coverage/register-istanbul
--require test/support/all
--require test/support/helpers

I get 40% coverage.  If I do:
--require test/support/all
--require test/support/helpers
--require coffee-coverage/register-istanbul

I get 18% coverage...  So that's maybe not it.  :/

If I hack up coffee-coverage to show what it's instrumenting, then I see:
10:54:42 [jwalton@nimue ~/Development/roots (master)] $ npm run coverage

><EMAIL_ADDRESS>coverage /Users/jwalton/Development/roots
> mocha && istanbul report

Instrumenting /Users/jwalton/Development/roots/lib/index.coffee
Instrumenting /Users/jwalton/Development/roots/lib/analytics.coffee
Instrumenting /Users/jwalton/Development/roots/lib/global_config.coffee
Instrumenting /Users/jwalton/Development/roots/lib/config.coffee
Instrumenting /Users/jwalton/Development/roots/lib/extensions.coffee
Instrumenting /Users/jwalton/Development/roots/lib/extensions/compiled.coffee
Instrumenting /Users/jwalton/Development/roots/lib/extensions/static.coffee
Instrumenting /Users/jwalton/Development/roots/lib/api/bail.coffee
Excluding /Users/jwalton/Development/roots/test/analytics.coffee
Excluding /Users/jwalton/Development/roots/test/cli.coffee
Instrumenting /Users/jwalton/Development/roots/lib/cli/index.coffee
Excluding /Users/jwalton/Development/roots/test/compile.coffee
Excluding /Users/jwalton/Development/roots/test/deploy.coffee
Excluding /Users/jwalton/Development/roots/test/extensions.coffee
Excluding /Users/jwalton/Development/roots/test/index.coffee
Excluding /Users/jwalton/Development/roots/test/new.coffee
Excluding /Users/jwalton/Development/roots/test/template.coffee
Excluding /Users/jwalton/Development/roots/test/watcher.coffee

Instrumenting /Users/jwalton/Development/roots/lib/api/template.coffee
Instrumenting /Users/jwalton/Development/roots/lib/sprout.coffee
  analytics
    ✓ should track events when the __track function is called (309ms)
    ✓ should be able to disable analytics through the api
    ✓ should be able to enable analytics through the api (266ms)
...

But then I never see any more coffee files get instrumented.  It's like the .coffee extension handler is somehow getting un-registered.  Hrm.

Huh, very strange. Thanks for investigating this, and glad it's not just me doing something stupid! If there's anything else I can help with here happy to do so, just let me know :grinning:

Did you ever get this working, BTW?

Nope. I've got it working with other projects, but not with this one :cry:


Speed up the dashboard

Currently, the dashboard starts to slow down with datasets at about 1.6MB, this becomes more pronounced by 3.3MB (CSV with about 12,000 rows by 15 columns).
The slow down is most pronounced in

Initial processing of the dataset (hence we have a loading indicator).
Selecting different color-by options for the map distribution; the change in the map has a pronounced delay.

Diagnosing the specific source of the slow down (if it's one or more reasons) is still required.
The Dash documentation provides various suggestions under their performance section, and the dash-extensions package also has some options:

Flask caching. Since the processing is only done once, this would only have potential to speed up the graphing components, though initial testing didn't seem to show improvement.
a. A similar solution is background caching.
b. Dash-extensions has a package for server-side caching: ServersideOutput Transform. This removes the need for JSON serialization between callbacks and should speed things up, though I'm not sure how effective it was (I tried an implementation on the server-store branch). This would also require a regular clean-up of the cached files (as with the previous two).
Dash Patch Class for updating the map (and potentially other graphs as well). Plotly express is fast, but starts to slow down around 15K points (per dash performance docs). I attempted to implement this with the map, but my initial attempt was unsuccessful.
Using orjson. I'm unclear on precisely how this potential solution works, it seems (and this) that having the package installed allows dash to serialize json strings with orjson instead. There are errors thrown if you try to use it for the serialization (the speedup comes from the fact that it's bytes).
Clientside Callbacks. To implement this, the graphing portion would have to be translated into Javascript.

Note: It runs slower on the AWS server than when run locally. Could this have to do with nginx settings?

Note: It runs slower on the AWS server than when run locally. Could this have to do with nginx settings?

Determined this was a result of the AWS hardware, not nginx through running on a Hugging Face Space, which sped up the processes to match running locally.

Image sampling can be quite slow on larger datasets; perhaps switching to polars could offer a speed-up for some of these features? It does integrate with plotly-dash and boast many of the same features of pandas at quite the enhanced speed for DataFrame manipulations.


Issue: Extra spacing between top of the page and the header on variou…

…s Components pages
Fix: Replacing \r\b with \r\n in ComponentSamples.js causes space between top of the page and the header to go away.

Icons page is mostly updating the CSS to be consistent with Components pages, spacing changes, and code style updates

I like what you got! Good job. Approved.

Looks good! Let's merge this in.


Add an overloaded setState(State => State)

Fixed by https://github.com/shadaj/slinky/commit/2b78534fa086cfeca6126abe09b5ce522467f8fd


Update participantes.json

Updating participants with linkedin and adding Zachary Mainen

Can you please make the participants' avatar clickable?

@pfortuna done. Also added a small animation to make it clearer


Concurrent connections

This properly implements a test case that stalls due to a bug under high concurrency and load.

what do you think the reason for this is? my first guess is gc pressure. does it ever finish?

@juliangruber it never finishes because a callback is never called for a readStream or a delete (but fairly sure its a readStream). Its something I ran into with this massive migration I'm running thats multi-process so I know its not GC pressure. I've found that tuning the concurrency in the each-limit when reading the data makes it more deterministic. In my live test case changing it from Number.MAX_VALUE to 15 allowed it to complete. The other factor here is record size. When i had it has utf8 values, it completed without issue but when making the records bigger and changing it to json, it causes it to stall.

interesting it passed!

hmmm

@juliangruber i originally reproduced the issue with https://github.com/jcrugzz/multileveldown-hang.

@juliangruber so i figured out it was due mostly to the size of the json record stored so I think it might be a bug in https://github.com/mafintosh/pbs. The concurrency was just a contributor as well but the size of the data was a larger piece of the cause. I worked around this limitation by ensuring i didn't store any json greater than 10000 bytes.

@jcrugzz  great job! i have time to look into / fix this tomorrow


Nodejs version needs to be updated

nodejs version in the template.yaml file needs to be updated from nodejs10.x to nodejs14.x

I believe this is fixed by commit https://github.com/aws-samples/lambdaedge-openidconnect-samples/commit/ebc38a1b267c0e33390ad1988c0db6bcd256880c

This has been updated to the latest. Thank you for opening this issue


messages being processed multiple times by same worker name

According to the readme, "subscribers that share the same application_name will act as multiple workers on the same queue. Only one will get to process each message", but this isn't true.
I have an app where processing each message is taking ~30 seconds of processor time.
Because of the code here:
https://github.com/iHiD/propono/blob/7b0dfe84569914704bc45d8542c9c34de4a67036/lib/propono/services/queue_listener.rb#L66
while the first worker is processing the message, a second worker has the opportunity to grab it.
Reversing those two lines kinds solves this issue, but creates a new one... a message can get lost if the worker fails during processing.
I don't think the semantics of sqs and sns alone can fix this issue one way or the other; sqs would need to be able to mark a message as 'processing'.
Perhaps deleting the message, calling the message handler, rescuing an exception, republishing the message, then re-raising the exception would be a potential strategy for handling this, but certainly raises the complexity.

Diving into how the visibility_timeout parameter works with SQS
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
it seems like there is an answer, but that there might be a bug in propono.
I believe this line:
https://github.com/iHiD/propono/blob/7b0dfe84569914704bc45d8542c9c34de4a67036/lib/propono/services/queue_listener.rb#L42
should reference "@visibility_timeout" rather than 'visibility_timeout'.
I'll investigate more during daylight.

What have you set visibility_timeout to in your config? https://github.com/ihid/propono#visiblity-timeout

should reference "@visibility_timeout" rather than 'visibility_timeout'.

That's just using the attr_reader not the instance variable directly. Shouldn't make a difference.

Once I grokked the visibility timeout and resolved which client instance I needed to set it to, all is well.
I put it on my todo list to write some documentation for the project about this.


All sections viewer

How can i show all the sections while scrolling and highlight the section which is being scrolled over. For example much like google photos date scroller on the photos screen.

Hi @debjitk 👋🏾
Did you ever find a solution to this?


DOCSP-31491: update synonym file

@casthewiz -
I added synonyms for searches with zero results and a few others I've been collecting.  If this process works, let me know who else could approve PRs for the synonym file going forward.
Thanks!
Sarah

@sarahemlin i deleted an old remnant NULL entry. i think this may have been a remnant of previous commits. will drop in a ticket to refresh the table to prevent this error. we're good to merge!


Set link to use semi bold font weight

Description
Set link to use semi bold font weight to add more visual distinction from the text around it
Issues Resolved
Fixes #933
Check List

[ ] New functionality includes testing.
[ ] New functionality has been documented.
[x] All tests pass

[x] yarn lint
[x] yarn test-unit

[ ] Update CHANGELOG.md
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Can you add some example screenshots of how this changes the appearance of links?


✨ Add option to include props to component

How to do this?
I mean, what format would be accepted?
A comma separated string?
An array?
Which formats are possible/accepted by commander?
Also, we must take into account the prop types when generating TS components.

I think something like "rc c example --str prop1,prop2 --int prop3,prop4 --bool prop5,prop6" looks good, then you can create the array with division.
Putting the PropTypes in js and creating an interface or type aliases in TS would be nice too.
A cool extra would be to be able to pass a useState function something like "rc c example --use-state fn01=number,fn02=string,fn03=bool" then you could split with comma and then with equals operator, and it would output this:
interface IExample {
fn01: Dispatch<SetStateAction>;
fn02: Dispatch<SetStateAction>;
fn03: Dispatch<SetStateAction>;
}
obviously this would only be a option for TS.

I've just read about variadic option on commander docs, and it seems the best option, but I propose a solution a little bit different of yours.
Example 1: Specifying simple/primitive props (with destructuring)
rc c example --props prop1:string prop2:number prop3:boolean prop4 --destructureProps

would generate something like:
type ExampleProps = {
  prop1: string;
  prop2: number;
  prop3: boolean;
  prop4: unknown;
}

export function Example({ prop1, prop2, prop3, prop4 }: ExampleProps) {
  ...
}

Example 2: Specifying simple/primitive props and setState props (without destructuring)
rc c example --props prop1:string func1:setState(Record<string, number>) func2:setState(boolean) func3:setState

would generate something like:
type ExampleProps = {
  prop1: string;
  func1: Dispatch<SetStateAction<Record<string, number>>>;
  func2: Dispatch<SetStateAction<boolean>>;
  func3: Dispatch<SetStateAction<unknown>>;
}

export function Example(props: ExampleProps) {
  const { prop1, func1, func2, func3 } = props;
  ...
}

Example 3: Specifying props and states
rc c example --props prop1:string func1:setState(boolean) --state state1:boolean[false] state2:string state3:Record<string, string> state4 --destructureProps

would generate something like:
type ExampleProps = {
  prop1: string;
  func1: Dispatch<SetStateAction<boolean>>;
}

export function Example({ prop1, func1 }: ExampleProps) {
  const [state1, setState1] = useState<boolean>(false);
  const [state2, setState2] = useState<string>();
  const [state3, setState3] = useState<Record<string, string>>();
  const [state4, setState4] = useState<unknown>();
  ...
}

What do you think?

I didn't like it for primitives values, because it will be too verbose and repetitive, so, I don't see why someone would use it. I think the main objective of the CLI is to be fast and productive, in this way, it's faster to just type in the file. But I think it's good for the useState and other functions.
Taking this into consideration, for primitive types, I think the best would be:
"rc c example -s prop1,prop2 -i prop3,prop4 -b prop5,prop6" // with alias -string, -integer, -boolean
If you want, I can do a vote on Linkedin, to see what people prefer.
I liked the destructuring flag.

I like the ideia of aliases. But how do we do with optional values?
Option 1 (<> indicates optional):
rc c example -s prop1 <prop2>

Option 2 (? indicates optional):
rc c example -s prop1 prop2?

Other alternatives???
I think it is a good ideia to gather people opinion with a voting :+1:
That said, I don't think we should support all primitive types... i mean bigint, null, symbol and undefined seems too much effort for too little benefit, like you pointed out, it's often faster to just type in file.

I prefer option 2, sems easyer to understand and more intuitive.

I'm starting to work on this :handshake:

We may have a problem here. I use zsh as my default shell, and when testing the optional props, I get a zsh: no matches found: error, because I'm using ? as the optional marker, and zsh interprets it as a wildcard.
With bash this isn't a problem.
As far as I see we can:

Change the optional marker to something else, that doesn't get interpreted as a wildcard
Instruct the users to create an alias for rc when using zsh (alias rc="noglob rc")

What do you think @HenrikSantos?

As zsh is widely used, I think it's better to use the option you suggested previously, , or allow both syntaxes and explain this problem with zsh in the documentation.


Starship is initialized wrong when loaded by direnv in zsh

Describe the bug
Starship outputs wrong control characters when initialized from zsh through direnv.
A clear and concise description of what the bug is.
To reproduce
This can be reproduced by creating a .envrc (by running devenv init) in the starship example in git from zsh.

The integration adds eval "$(starship init $(echo $0))" to the enterShell.
$0 will be the bash that's started by devenv, not the parent shell. This causes starship to output bash control characters instead of zsh ones.
Version
devenv: 0.6.3

A workaround is to set the STARSHIP_SHELL variable to zsh. However, this will cause starships shell module to display zsh instead of bash.

how did youll resolve this? I also have the same problem but the setting of STARSHIP_SHELL did not work
{ pkgs, lib, config, inputs, ... }:

{
  env.STARSHIP_SHELL = "zsh";

  enterShell = ''
    if [ -d ".venv" ]; then
      source .venv/bin/activate
    else
      uv venv && source .venv/bin/activate
    fi
  '';

  languages = {
    python = {
      enable = true;
      version = "3.11.6";
    };
  };

  dotenv.enable = true;

  starship.enable = true;
}

I ended up not using starship from devenv, but initializing it from my .bashrc / .zshrc
In devenv I now only have
enterShell = ''
export STARSHIP_CONFIG=${config.env.DEVENV_ROOT}/starship.toml
.......
''

This affects me with Fish as well:

I've also noticed that the prompt gets mangled if I just call bash in there, but not if I invoke the systemwide bash provided by Nix. I think we've ended up with pkgs.bash rather than pkgs.bashInteractive. When I add pkgs.bashInteractive to my packages declaration in devenv.nix, my prompt looks fine when I enter a new shell with bash.
I think we can maybe fix this if we get mkShell or something similar to use bashInteractive instead of plain bash. The WIP devShellTools.buildShell might solve this issue.

When I add pkgs.bashInteractive to my packages declaration in devenv.nix, my prompt looks fine when I enter a new shell with bash.

So this is kinda crazy and heavy-handed, but for me setting:
stdenv = pkgs.stdenv.override (old: { shell  = "${pkgs.bashInteractive}"; });

fixes this for flake.nix-based projects for me with nix (Nix) 2.23.3. For devenv 1.0.8 (aarch64-darwin), though, devenv's Nix (forked from 2.21) spits out an error:
error:
       error: 'nix develop' only works on derivations that use 'bash' as their builder

✖ Command produced the following output:

Error:   × Command `/nix/store/fqb0biq56868v5hvnp47fb3aabsk6as2-nix-2.21-devenv/bin/nix --show-
  │ trace --extra-experimental-features nix-command --extra-experimental-features
  │ flakes --option warn-dirty false --keep-going --max-jobs 5 --option eval-cache false
  │ print-dev-env --profile /Users/pxc/Documents/Code/Learning/Command-Line Rust/01/
  │ hello/.devenv/gc/shell --option extra-substituters https://devenv.cachix.org --option
  │ extra-trusted-public-keys devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/
  │ Hm9XgU50cw=` failed with with exit code 1

direnv: failed to build the devenv environment. devenv.nix may contain errors. see above.


Why do we have two SigHashTypes?

First one:
https://github.com/rust-bitcoin/rust-bitcoin/blob/master/src/blockdata/transaction.rs#L676-L695
Second one:
https://github.com/rust-bitcoin/rust-bitcoin/blob/master/src/util/sighash.rs#L102-L126

The second one has the Reserved variant, here are some comments about it:
https://github.com/rust-bitcoin/rust-bitcoin/pull/628#discussion_r670280342
Nobody was super-sure about it, we can decide to remove it before release

Sorry, my previous comment was pertinent but not the real reason, the real reason it's the Default variant added in Bip341

Can we somehow differentiate them by name, do a deprecated stuff on the old one or add necessary doc explanations on the choice of the structure?

Lol, let's just remove the one in transaction.rs

Lol, let's just remove the one in transaction.rs

If we do so, we would have to add checks to existing methods to not use the new variants. At which I think that having two separate types is the correct thing.

If we have to maintain both structures we need to differentiate them by name and clearly describe the purpose of both in the docs

Also, I think we are re-exporting the legacy one https://github.com/rust-bitcoin/rust-bitcoin/blob/a961ab452657a2e94925632747a1794b768c17a6/src/lib.rs#L125


Allow more than 4 registers for 74HC165 and 74HC595

Hello,
Is your feature request related to a problem? Please describe.
Input and ouput registers are limited to a maximum of 4 chips in MobiFlight.
But technically, 74HC165 and 74HC595 registers allow a maximum of 8 chips in a chain.
Could it be possible to address 8 chips also in MobiFlight ?
Describe the solution you'd like
Just increase the dropdown selector up to a maximum of 8 items instead of 4.
Describe alternatives you've considered
There is no way to use more than 4 shift registers in MobiFlight.
Thank you
Regards
Stephane

@elral - with no limit on the FW side, this now is totally limited by the boards.json only

@DocMoebiuz Yes, but the user is requesting 8 shifters daisy chained. No changes would be required in the firmware for this and in the board.json file.

Is it only limited by the dropdown selector of the user interface in win10 application ?
Easy to change ?

I tried to create a branch and push code to make a PR, but I've got 403 error.
So I give you my changes, just 2 lines of code:
for both
https://github.com/MobiFlight/MobiFlight-Connector/blob/main/UI/Panels/Device/MFShiftRegisterPanel.cs
https://github.com/MobiFlight/MobiFlight-Connector/blob/main/UI/Panels/Device/MFInputShiftRegisterPanel.cs
set
private int MAX_MODULES = 8;
instead of
private int MAX_MODULES = 4;
Here is a patch:
Fix_ShiftRegister_and_InputShiftRegister_chips_up_to_MAX_MODULES=8.patch

We have removed the device limitations in the firmware. These come now from the settings in the boards.json file. Please experiment with it.
This was the PR on the firmware side: https://github.com/MobiFlight/MobiFlight-FirmwareSource/pull/253
It is already included in the beta. And it will be included in the next release.
I will close this issue.

Sorry, my bad - you were looking at the number of modules in the chain.

Hello @DocMoebiuz,
Is there any chance this ticket will be fixed?
Thank you
Regards
Stephane


feat: Add geocoding provider here

Description
Adds new provider for address/geocoding profile

Motivation and Context

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I haven't repeated the code. (DRY)
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

We need to update parser in Brain before merging.

We need to update parser in Superface services before merging.

@martinalbert I just realised that parser and ast have been updated already to version 1.1.0 in Superface services (Brain). I am not sure about the CLI, but Brain is ready.

Looks good 👍


Create Modbus Server

I am trying to simulate an inverted so I can test different SunSpec Modbus client devices.  Can the Modbus Map Class be used to create a Modbus server just using pysunspec or will I also have to use pymodbus?  Thanks.

Same question here.


Added the floating navbar to go up and down on scroll

Related Issue #414
[Cite any related issue(s) this pull request addresses. If none, simply state “None”]
Description
https://github.com/user-attachments/assets/12c9e4aa-8644-4ecf-ad89-45f3c281c4e1
[Please include a brief description of the changes or features added]
Type of PR

[ ] Bug fix
[x] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
[Attach any relevant screenshots or videos demonstrating the changes]
Checklist:

[x] I have performed a self-review of my code
[x] I have read and followed the Contribution Guidelines.
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[x] I have commented my code, particularly in hard-to-understand areas.

Additional context:
[Include any additional information or context that might be helpful for reviewers.]

Thank you for submitting your pull request! 🙌 We'll review it as soon as possible. In the meantime, please ensure that your changes align with our CONTRIBUTING.md. If there are any specific instructions or feedback regarding your PR, we'll provide them here. Thanks again for your contribution! 😊

@ashutosh-rath02
error is related to useSearchParams right ??
we need to wrap the Home in suspence with fallback loading to fix it

@ashutosh-rath02
Kindly reivew this I resolved the conflicts


400 error for admin panel over SSH tunnel

I have not forwarded port 81 on VPS for internal security reasons.
I have logged in via SSH with tunnel setting as L81--><IP_ADDRESS>:81
But the browser is giving Bad Request - Invalid Hostname error when I open http://<IP_ADDRESS>:81 in browser.
How to access Admin web panel using ssh tunnel?

It seems I have to enable SYN packets on port 81 from iptables to access it from browser.
It worked after that.

@itsKV What is your connection string to the VPS?
I just did this on mine it worked find.
I would recommend port forwarding a higher port than 81.  your host operating system might have an issue with :81
ssh -L8181::81 user@
this assumes that :81 is the management port that docker is listening to for management of the NPM container.

Sorry, my bad.
The port I was trying to use on local machine was hogged by local application which was giving 400 error. Changed the port no 'free' one. That solved the issue.


Update cheatsheets/weekly-realignment with Fix conflicts link to cheatsheets/fix-conflicts.md (to create)

Update the existing cheatsheet: cheatsheets/weekly-realignment.md with explanation on how to solve Git conflicts

Create a cheatsheet about how to fix conflicts in Git command lines and update the sections about How to fix conflicts in the file: cheatsheets/weekly-realignment.md (2 sections to update -> fast-forward and non-fast-forward merging sections) with a relative link to the newly created file.


Optimize unnecessary columnar->row->columnar transitions with AQE

Is your feature request related to a problem? Please describe.
Sometimes AQE processing can lead to a query plan with what appears to be a degenerate columnar->row->columnar transition like this:

It would be great if this wasteful transition between formats could be avoided, keeping the data on the GPU throughout.
Describe the solution you'd like
Plugin transition overrides recognizes this situation and removes unnecessary transitions when possible.

Here is one minimum reproduce which can trigger similar issue when AQE is on:
val df = Seq(1).toDF()
df.write.format("parquet").mode("overwrite").save("/tmp/testaqe")
spark.read.parquet("/tmp/testaqe").createOrReplaceTempView("df")
# No GpuColumnarToRow or GpuRowToColumnar
val out=spark.sql("select value from df group by value")
out.write.format("parquet").mode("overwrite").save("/tmp/testaqe_out")

# When ACQ is on, one pair of GpuColumnarToRow or GpuRowToColumnar is introduced.
spark.conf.set("spark.sql.adaptive.enabled", true)
out.write.format("parquet").mode("overwrite").save("/tmp/testaqe_out")

Basically to trigger this I need to make sure all of below 3 requirements are met:

AQE is on
There is aggregation before writing to parquet files
Write to parquet files eventually

I was able to get the test case to work by adding this method to Spark's AdaptiveSparkPlanExec:
def doExecuteMaybeColumnar(): Either[RDD[ColumnarBatch], RDD[InternalRow]] = {
  val plan = getFinalPhysicalPlan()
  if (plan.supportsColumnar) {
    val rdd = plan.executeColumnar()
    finalPlanUpdate
    Left(rdd)
  } else {
    val rdd = plan.execute()
    finalPlanUpdate
    Right(rdd)
  }
}


Design dynamic loading of new GUI.

In the MVP all the apps are more or less built in and loaded all the time.
We need a design for how to:

Build applications as hot plug-able modules.
Load and unload applications and widgets at runtime.
Support GUI elements that is not part of the core repository. "Application hosting"

We are archiving this repo.


Request param

How can i get request param in adonis 4.x?

http://dev.adonisjs.com/docs/4.0/routing#_route_parameters

Oh, thanks!


Version 5.0.2 bump

closing because 5.1 we haven't created a release branch for 5.0 yet, and when we do, we'll want to pip dev to either 5.1 or 6.0

closing because 5.1 we haven't created a release branch for 5.0 yet, and when we do, we'll want to pip dev to either 5.1 or 6.0


AAD B2B organisation details

Hi,
I'm currently working on a static web-app that is providing a client and a business portal. I configured the aad auth provider with the B2B active directory tenant details and added a custom open-id connect provider with the B2C tenant details.
As the JWT tokens do not exposed I wanted to understand how I can receive additional details for the loggedin user like i.e. organization details or name, last name and email from the graph API?
This is the configuration that I'm currently using
"auth": {
    "identityProviders": {
      "azureActiveDirectory": {
        "registration": {
          "openIdIssuer": "https://login.microsoftonline.com/<B2B_TENANT_ID>/v2.0",
          "clientIdSettingName": "AAD_CLIENT_ID",
          "clientSecretSettingName": "AAD_CLIENT_SECRET"
        }
      },
      "customOpenIdConnectProviders": {
        "aad-b2c": {
          "registration": {
            "clientIdSettingName": "AAD_B2C_CLIENT_ID",
            "clientCredential": {
              "clientSecretSettingName": "AAD_B2C_CLIENT_SECRET"
            },
            "openIdConnectConfiguration": {
              "wellKnownOpenIdConfiguration": "https://<B2C_TENANT_NAME>.b2clogin.com/<B2C_TENANT_NAME>.onmicrosoft.com/v2.0/.well-known/openid-configuration?p=B2C_1_signup_signin"
            }
          },
          "login": {
            "nameClaimType": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name",
            "scopes": [
              "openid profile email"
            ],
            "loginParameterNames": []
          }
        }
      }
    }
  }

We released the feature to programmatically assign roles with a function. Within the function, you have access to the token from the IdP. See the documentation for more details. There's also a tutorial with a sample app.
Currently you can use this function to assign roles. If you'd like to also store claims that you can access in the /.auth/me endpoint or in the user principal in a function app, let us know.


[New Feature]: Integrate Airflow with Cognito authentication (and authorization?)

Acceptance Criteria:
o Demonstrated use of Airflow API via Unity API Gateway URL
o Demonstrated login into Airflow UI using the Unity API Gateway URL
Dependencies:
From CS: Unity API Gateway deployed onto the target venues (mcp-venue-dev and mcp-venue-test)

@LucaCinquini I see this is still assigned to sprint 24.1.2, but you mentioned in your comment that you will move it to a later sprint.

@LucaCinquini are we still waiting on U-CS coordination before we move forward with this ticket?  What do we need to do to get this one moving?

Yes we are. Let's wait until Galen is back and then I hope Ramesh can do this, or it will be a combination or Ramesh and Drew.


2.4.0

this is prepping for a 2.4.0rc1 release

Looks like this will be stalled until we get a conda release of spopt.

Looks like this will be stalled until we get a conda release of spopt.

Codecov Report

Merging #1204 (97b351b) into master (8a7c9c9) will decrease coverage by 0.05%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1204      +/-   ##
==========================================
- Coverage   11.60%   11.55%   -0.06%
==========================================
  Files          23       23
  Lines         224      225       +1
==========================================
  Hits           26       26
- Misses        198      199       +1

Impacted Files
Coverage Δ

pysal/base.py
32.07% <ø> (ø)

pysal/model/__init__.py
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a7c9c9...97b351b. Read the comment docs.

Codecov Report

Merging #1204 (97b351b) into master (8a7c9c9) will decrease coverage by 0.05%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1204      +/-   ##
==========================================
- Coverage   11.60%   11.55%   -0.06%
==========================================
  Files          23       23
  Lines         224      225       +1
==========================================
  Hits           26       26
- Misses        198      199       +1

Impacted Files
Coverage Δ

pysal/base.py
32.07% <ø> (ø)

pysal/model/__init__.py
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a7c9c9...97b351b. Read the comment docs.

The staged-recipe for spopt is now passing so once that PR is reviewed+merged we will have an spopt-feedstock up on conda-forge. At that point we should cut a v0.1.0 release to include with the v2.4.4 release of PySAL.

The staged-recipe for spopt is now passing so once that PR is reviewed+merged we will have an spopt-feedstock up on conda-forge. At that point we should cut a v0.1.0 release to include with the v2.4.4 release of PySAL.

Once this is reviewed and merged, next ups is the release actions.

Once this is reviewed and merged, next ups is the release actions.

let's merge or wait for @knaaptime's review?

let's merge or wait for @knaaptime's review?

Can we remove codecov as it is more annoying than helpful. The imports are being tested, it just doesn't see that.

Can we remove codecov as it is more annoying than helpful. The imports are being tested, it just doesn't see that.


New Feature: Force HTTPS

Currently I don't see a way to force an Azure Function with an HTTP binding to only respond to HTTPS.
This request is for a new feature to support HTTPS only. With the security audits our applications go through, HTTPS is a requirement and we need to be sure that HTTP will not be used.

Instead of supporting a way to disable http, I think we should generally only support https across the board. The hard part is that it’s a breaking change if someone is relying on http today (which they shouldn't, but still).

Is there a way for me to programmtically tell if the function was  accessed via HTTP rather than HTTPS ? So I could thorw an error at least.

The request itself should have a "Scheme" property that you can use. HttpRequestMessage.RequestUri.Scheme. I'd be interested if this is a valid approach or if it leaves gaps.
I would still like for this to be handled at a global level. I'm in the same boat with security audits. Generic WebHooks already reject non-HTTPS traffic.

That is what I ended up doing, also curious if this is "safe enough".

We've experience similar behavior from the GenericWebhook function trigger:
https://github.com/aspnet/WebHooks/blob/4bfeb36657f4f52459d4006c390a8d45375a6731/src/Microsoft.AspNet.WebHooks.Receivers/WebHooks/WebHookReceiver.cs#L174

It seems like that section @Blackbaud-MitchellThomas referenced is as good as any protection I could layer on top aside from blocking things at the network layer.

Blocking at the network layer is definitely be much better, since there would never be a connection to the HTTP listener in the first place.
Theoretically if there are cookies without the "secure" flag set, if an attacker was able to make a user make a request to the HTTP Listener and it was listening at all; the cookies would be sent in the clear. If the attacker was on a network in between the user and the application he or she may be able to get that value.
There are a lot of "If's" in there, which should be sufficient to drop anything  on a security audit to a low or informational if you in fact make it so the application does not interact on HTTP. However, being able to block it and the network layer is a lot better, since at a minimum; there won't be any explaining to do.
More importantly, this is the type of stupid thing that old-school ops people grab onto to make an argument on why we shouldn't move to the cloud because its "less secure", and we "don't have enough control to make sure we're secure". Which is usually just a smoke screen for, "I don't want to learn something new". You can usually overcome it with logical arguments, but again; its even better to not have to explain it in the first place.

You hit the nail on the head. For some applications, this in-code level of protection can be seen as enough of a mitigation of risk, but for more sensitive applications, especially those that fall under HIPAA and PCI scope, we don't want any gaps if we can help it.

+1

+1

+1

+1

+1

+1

+1

+1

For now I'm using this utility method in my functions to reject any non-https requests.
        internal static void EnsureSecureConnection(HttpRequestMessage request)
        {
            if (!request.IsLocal() && !request.RequestUri.IsHttps())
            {
                var noHttps = request.CreateErrorResponse(HttpStatusCode.BadRequest, "Only HTTPS traffic is supported on this endpoint.");
                throw new HttpResponseException(noHttps);
            }
        }

This works for me:
In the Azure portal, go to your function app.

Platform features > Custom Domains > toggle HTTPS Only to 'On'.

Anyone using HTTP will receive a 301 Moved Permanently and be redirected to the HTTPS endpoint. You do not need to actually add a new hostname/domain to toggle this feature.

Looks like it was added recently to app services. Seen plenty of articles with that screenshots from that screen and that option was not there! Thanks for the information.

Yep, it's brand new indeed. Let's call this one done.

Is there any way to specify this option in an ARM template that creates the function app?

Yes, just set "httpsOnly": true on the site object (i.e. as a sibling to clientAffinityEnabled).

@davidebbo is the update to app service documented anywhere?

Not sure. I'd expect it to be mentioned on some post to https://blogs.msdn.microsoft.com/appserviceteam/, but not seeing one now.

By some reason "httpsOnly": true takes no effect during fresh new deployment. It only applies with redeployment to existing app. Is there any explanation for such behaviour?

@OGalaev-Quest that looks very much like a bug (I verified it). I will contact the code owners about it.

I cannot get it to work even after the app service has been created.
One approach is to use rewrite rule in web config (https://blogs.msdn.microsoft.com/benjaminperkins/2014/01/07/https-only-on-windows-azure-web-sites/) but I really don’t want go through all my services and do this. It would be good to know if this is accepted as a bug and some time scale for a fix?

Saying it doesn't work even after creation is basically equivalent to saying that the feature is generally broken and can't be used at all, and we know that's not the case. So I think something else is at play for you here. Try manually setting it post-creation via https://resources.azure.com/, and verify that the setting sticks.

Thanks David. It's actually the portal not refreshing and causing confusion. Resource explorer showed the updated status correctly. However the blade does not reflect the status, either after clicking the refresh button (in blade) or moving to another blade and back again. Only a browser CTRL+F5 refreshes the status.

@ChrisOwczarek yes, the Portal does some caching so it's common for recently updated settings to not be reflected right away.
so to summarize, the only known issue here is that it doesn't work on creation.

Thanks David. But I would say the refresh button (on the blade), when clicked, should result in the latest setting being shown and it does not. Cheers.

@ChrisOwczarek yes, I was focused on the backend side. There is definitely a Portal refresh issue, and it is more general (not specific to this setting).

I'd like to know by the end of which month it should be live? Because it is certainly not working for us at the moment.

@davidebbo I don't know if this goes here or in another feature request, but it probably makes sense for the HTTP Header Strict-Transport-Security to be set when this is enabled:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security
This will help keep the browser from ever sending anything over HTTP, even if the developer makes a mistake and has a reference to an unsecure resource.
I know the way the setting is working any call to HTTP will redirect to HTTPS, but cookies could still potentially be sent on the HTTP call if not configured properly.

@Vossekop This is definitely in place. I am using it.
Check Platform Features -> Custom Domains

@securityvoid Sorry, I was refering to @jeanpaulsmit 's issue of using the httpsOnly field in ARM templates. Which is what I am trying to use and isn't working as far as I can see.

The ARM issue is supposed to be fixed, but I'll let @jennylaw confirm.
@securityvoid for the Strict-Transport-Security, I think it's best to open a separate feedback item on https://feedback.azure.com/forums/169385-web-sites/ (there may already be one).

@davidebbo Okay, its posted here:
https://feedback.azure.com/forums/169385-web-apps/suggestions/33509452-strict-transport-security
Hopefully this can be implemented, as it will knock one more finding off of the Security Reports.
Since with regular websites you can set headers with a web.config, I'm not sure if they will view this as that high of a priority. Primarily it impacts functions where you can't use a web.config (At least that's been my understanding).
Here are instructions on how to do it with a web.config, that should work with the normal Azure Apps.
https://www.hanselman.com/blog/HowToEnableHTTPStrictTransportSecurityHSTSInIIS7.aspx
More details on what this feature does, if you're interesting in reading more:
https://www.owasp.org/index.php/HTTP_Strict_Transport_Security_Cheat_Sheet
Setting this header when a website is only available over HTTPS is considered industry best practices.

@securityvoid correct, you can't provide your own web.config. Furthermore, with Consumption functions, you can't use site extensions, so this (3rd party extension) won't work: https://www.siteextensions.net/packages/HstsSettings/


Problem with installation! (Installer.exe)

How exactly should Installer.exe be opened? Even if I open it with cmd as an administrator, it does not run. The game does not open even if I show the target folder? can you post a screenshoot for this tutorial or video?

Additionally, the version of AKI I play: <IP_ADDRESS>.12893

The installer requires .NET Framework 4.7+ to run. It will install necessary files, it will not act as a loader for the game.

I have the latest Net Framework, but I still can't get it to run properly. Is there a certain way to setup the installation? I'm not code savvy so use layman's terms.

I'm not sure to understand what is the exact problem, a lot of users are using this installer without any issue. Do you have screenshots or anything what could help?
Like what happens if you open a command prompt, and run the exe ? (do not run the exe directly, open a command prompt first).
My guess is : it is not able to find a EFT installation.
So you have to manually run like this:
`installer.exe "Path\To\Your\Eft\Installation"
But this is already covered in the documentation.

I'm sorry, I might just be missing something. I'd send you screenshots, but there's nothing to show. The program starts up and immediately closes. I've made sure to run a CMD prompt as Admin first. I've even tried running it manually through the CMD, but it doesn't recognize the EXE. Is there anything I'm missing? Does the installer.exe have to be in a certain place on my computer or something? I just don't know.

No the executable is completely standalone. You can run it from any directory. Perhaps an issue with your antivirus software ? (But I never saw that previously).

Well my issue is running the installer.exe "Path\To\Your\Eft\Installation" as an spi-aki user is the latest master branch is pulled, which is incompatible with version <IP_ADDRESS>.12893 which spi-aki needs.
Is there a way to tell installer.exe which branch to pull & compile, or create an installer.exe specific for spi-aki users?

Ok got it. We have support for <IP_ADDRESS>.12893, but changes I made for latest EFT introduced an issue with the installer. (after this it should try to compile with the <IP_ADDRESS>.12893 branch, but stops given the file not found error is fatal).

I published a new version of the installer:
https://github.com/sailro/EscapeFromTarkov-Trainer/releases/tag/installer-1.1
@make-believe-agency could you try this one on your side to see if it fixes your issue?
Thanks

@sailro Yep that worked thanks! I noticed something else doing this, on my machine Installer.exe defaults to the live installation.
This happens when launching through powershell without passing a directory to it, as well as launching the .exe directly. So result in my environment is double clicking the file means I have to delete my EFT install and redownload.

When I give Installer.exe the path of sptarkov, it then works as intended. Perhaps a quick fix would be to force users to give Installer.exe the correct path, otherwise it won't run?


[10.x] Fix Session missing() logic.

In the documentation under Session > https://laravel.com/docs/master/session#determining-if-an-item-exists-in-the-session, the missing() method has the following description:

To determine if an item is not present in the session, you may use the missing method. The missing method returns true if the item is null or if the item is not present:

However, it cannot be correct if the missing() method is the inverse of exists(), since exists() will also return true if the item is null.
The missing() method should be defined as the inverse of has() instead, since has() will only return true if the item is present AND is not null.
The test cases have also been fixed to match with the described behavior in the documentation.
Suggesting to merge into master branch due to potential breaking change.

I'm not making this breaking change. Just very little benefit. Better to just update the docs if they are incorrect.


dockerize app, add docker-compose

minimal docker image (using alpine) for running app
docker-compose will build/run the app and link w/memcached container

utACK


Issue regarding MdSidenav update: Accessing .toggle() method

Bug, feature request, or proposal:
Previously, I could access the toggle method using:

<md-sidenav id="left-sidenav" align="start" opened=true mode="side" #sidenav>
export class LeftSidenavComponent {

    //previously was MdSidenav
    @ViewChild('sidenav') sideNav: MdSidenavModule;

    //for demonstration purpose
    closeOrOpen() {
        this.sideNav.toggle();
    }

Now I can't access the property anymore. Any clue as to how?
What is the use-case or motivation for changing an existing behavior?
Being able to apply custom typescript logic to when the sidenav opens/closes, instead of just toggling it within the html
Which versions of Angular, Material, OS, browsers are affected?
"@angular/animations": "^4.0.2",
"@angular/common": "^4.0.2",
"@angular/compiler": "^4.0.2",
"@angular/compiler-cli": "^4.0.2",
"@angular/core": "^4.0.2",
"@angular/forms": "^4.0.2",
"@angular/http": "^4.0.2",
"@angular/material": "^2.0.0-beta.3",

I saw the MdSidenavToggleResult and was wondering if it was applicable. The documentation was a bit unclear though on how to apply it.

The type on the @ViewChild is supposed to be MdSidenav not MdSidenavModule.

@crisbeto Lmao, that worked. Thought the syntax changed and MdSidenavModule was the default import for sidenavs. Thanks!!!


statistics.yaml

ERROR [bot.ad_discordbot.modules.utils_files]: File not found: ad_discordbot\internal\statistics.yaml
If file does not exit, create empty file.
Cheers.

Hello, thank you for the issue report.
I agree, this error does look scary - in the next update, I'll be sure this is changed to either Info or Warning that the file will be created.
I'll be focusing on making the bot more easily updateable, as soon as I can.  Let me know if you have any other issues.
Come discuss the bot on discord if you have any insights.


Long startup time for influx CLI since introducing Flux 0.42

I noticed that it is taking about 2.5 seconds to run influx help on my workstation.
I bisected to the commit that upgraded to Flux 0.42. Before that it takes around a second, which is still too long IMO.
I have not yet taken a CPU profile to understand where exactly time is being spent.
bash-3.2$ git log -1
commit 454003841aabeb74396d73541378bfa59c75b5db (HEAD)
Author: Nathaniel Cook<EMAIL_ADDRESS>Date:   Fri Aug 30 12:22:48 2019 -0600

    build(flux): upgrade to Flux v0.42.0

    Skips a test that requires special dependencies to pass.
bash-3.2$ GO111MODULE=on go build ./cmd/influx && time ./influx help
Influx Client

Usage:
  influx [flags]
  influx [command]
<snip>

real	0m2.355s
user	0m3.832s
sys	0m0.120s

bash-3.2$ git checkout 45400384^
Previous HEAD position was 454003841 build(flux): upgrade to Flux v0.42.0
HEAD is now at 333dc1d19 Merge pull request #14883 from influxdata/fix/flux-data-check

bash-3.2$ GO111MODULE=on go build ./cmd/influx && time ./influx help
Influx Client

Usage:
  influx [flags]
  influx [command]
<snip>

real	0m0.981s
user	0m2.099s
sys	0m0.086s

When I am running “influx auth help”, it is taking between 4 to 10 seconds just to print the usage. strace reveals a lot of timing data being collected, along with futex.
It’s kind of a mood killer to wait 4 to 10 seconds to get help output.
root@120792:~# time influx auth help
… [ removed ] …
real 0m10.109s
user 0m9.637s
sys 0m0.154s
If I had to guess influx is running timing loops to try to discover timing granularity or resolution, it’s really going to be a bummer for people if they need to run x100 influx commands and the mean time to real work is ~5 seconds for each, meaning ~500 seconds or over 9 minutes waiting while system clock granularity is re-discovered on each run.

It's over 1min on raspberrypi, prohibitively slow to use cli in a script for me. Also seems to be CPU intensive like crazy on initialization alone.

I'm trying to reproduce this issue, but I'm seeing 1 second times using InfluxDB's latest commit, which includes Flux v0.53.0.

Does anyone have an idea as to why mine isn't slow?

@fchikwekwe one second to output a help message is still inappropriately slow. But, I suspect you are not running the build of influx that you think you are running.
cd /path/to/influxdb && GO111MODULE=on go build ./cmd/influx && time ./influx help still reports 2.5 real seconds on my machine, on current master (0d89697809974d1fa5c02311bb91f8039caf4d03), built with go version go1.12.9 darwin/amd64.

@fchikwekwe I would agree one second is too, long, but on some of my virtual machines, the time is ten seconds! I suspect there is a blocking calibration type routing occuring (I strace'd this)
which is blocking startup while calibration occurs.
As I built Ansible scripts to setup Influx, waiting 10 seconds for each test to understand how the cli/db/files interact was quite painful.

Yeah, thanks. Running your command allows me to reproduce the error. I'll work on trying to profile  it.

I applied this diff to influx:
diff --git a/cmd/influx/main.go b/cmd/influx/main.go
index 9f25ff33b..179540712 100644
--- a/cmd/influx/main.go
+++ b/cmd/influx/main.go
@@ -6,6 +6,8 @@ import (
 	"io/ioutil"
 	"os"
 	"path/filepath"
+	"runtime"
+	"runtime/pprof"

 	"github.com/influxdata/influxdb"
 	"github.com/influxdata/influxdb/bolt"
@@ -17,6 +19,20 @@ import (
 	"github.com/spf13/viper"
 )

+var profileFile *os.File
+
+func init() {
+	runtime.SetCPUProfileRate(50000)
+	f, err := os.Create("/tmp/influx.cpuprof")
+	if err != nil {
+		panic(err)
+	}
+	profileFile = f
+	if err := pprof.StartCPUProfile(f); err != nil {
+		panic(err)
+	}
+}
+
 var influxCmd = &cobra.Command{
 	Use:   "influx",
 	Short: "Influx Client",
@@ -74,6 +90,8 @@ func main() {
 	if err := influxCmd.Execute(); err != nil {
 		os.Exit(1)
 	}
+	pprof.StopCPUProfile()
+	profileFile.Close()
 }

 // Flags contains all the CLI flag values for influx.

But I am not seeing anything useful in the profiles -- I believe this is because imported packages' init functions (in this case, from flux) run before the init functions in the main package.
However, when I start influx and press ctrl-\ to send SIGQUIT, I seem to always see github.com/influxdata/influxdb/query/builtin.init.0 in the stack, which calls flux.FinalizeBuiltIns()
The simplest approach to fix this would be to lazily call flux.FinalizeBuiltins, probably via a sync.Once. I have no idea how many call sites would be affected by this.

Okay, that it pretty helpful. Let me get some input from the rest of the flux team on this.

Are nightly builds on download page working? Tried https://dl.influxdata.com/influxdb/nightlies/influxdb-nightly_linux_armhf.tar.gz but it seems to have modification date on 11th Jan 2019 on files within the package.

@Srokap From what I understand, it doesn't seem like nightly builds are being generated for 1.x anymore.

It seems like at some point the nightly builds may have inadvertently been killed. I've opened an issue for this here #15789.

Ok, was hoping to pick up nightly, but should be able to compile it myself to verify fix.


Text framework for OpenDP App

Targeted/dynamic text usage; integrating content from #89, e.g. https://docs.google.com/spreadsheets/d/1Yo4o_m9B9n74b5jAgVgDHT-XcYuorZ9JKvHuCmbnbUo/edit#gid=1839901367

use i18n hooks for future translation

Closed. In place .csv -> JSON file used by Vue


Error in data_pre.py: ValueError: Cannot set a DataFrame without columns

Hello, I encountered an issue when running the data_pre.py script as per the repository instructions. Below is the error I received:
(cfvd) user20@user20-virtual-machine:~/Desktop/counterfactual-vulnerability-detection-main/cfexplainer$ python data_pre.py
[Errno 2] No such file or directory: '/home/user20/Desktop/counterfactual-vulnerability-detection-main/cfexplainer/storage/cache/minimal_datasets/minimal_bigvul.pq'
(6 Workers) Run: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 188636/188636 [00:02<00:00, 73545.78it/s]
(6 Workers) Run: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 188636/188636 [00:02<00:00, 77460.00it/s]
(6 Workers) Run: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 188636/188636 [00:08<00:00, 21695.85it/s]
(6 Workers) Run: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 188636/188636 [00:12<00:00, 15051.73it/s]
Traceback (most recent call last):
File "/home/user20/Desktop/counterfactual-vulnerability-detection-main/cfexplainer/data_pre.py", line 171, in
bigvul()
File "/home/user20/Desktop/counterfactual-vulnerability-detection-main/cfexplainer/data_pre.py", line 131, in bigvul
dfv["mod_prop"] = dfv.apply(
File "/home/user20/anaconda3/envs/cfvd/lib/python3.9/site-packages/pandas/core/frame.py", line 4289, in setitem
self._set_item_frame_value(key, value)
File "/home/user20/anaconda3/envs/cfvd/lib/python3.9/site-packages/pandas/core/frame.py", line 4452, in _set_item_frame_value
raise ValueError(
ValueError: Cannot set a DataFrame without columns to the column mod_prop
It seems the script fails to find the dataset file minimal_bigvul.pq, and later encounters an issue with setting a column in an empty DataFrame.
Could you please provide guidance on how to resolve this issue?
Thank you for your assistance!

Thanks for following our work, and sorry for the late reply.
Regarding the error message “[Errno 2] No such file or directory: '/home/user20/Desktop/counterfactual-vulnerability-detection-main/cfexplainer/storage/cache/minimal_datasets/minimal_bigvul.pq’”, this is generated by line 86 in the data_pre.py script. However, this error doesn't actually impact the script’s execution. The file is generated in lines 157-163 after running the code, and its purpose is to save a cache for future runs. The missing file is just for optimization, so it's not causing the empty DataFrame issue.
The actual issue seems to be related to line 89, where the script tries to read a file. Please check if the Big-Vul dataset has been successfully downloaded or if the path is configured correctly. You can download the cleaned version of the Big-Vul dataset by fetching the MSR_data_cleaned.csv file from this link. For detailed information on Big-Vul, please visit its official repository.
Once downloaded, decompress and move the dataset to the appropriate location using the following commands:
unzip MSR_data_cleaned.zip
rm MSR_data_cleaned.zip
mv MSR_data_cleaned.csv cfexplainer/storage/external

You also need to configure the data storage path by running:
cd cfexplainer
export SINGSTORAGE=$(pwd)

Another possible issue could be related to mismatched versions of pandas or other dependencies. Please ensure you're using the required versions as specified in the README.
Additionally, if you encounter issues with generating the code graph, you might find this link helpful: https://drive.google.com/drive/folders/12iUTp4YFRG9qgAtqOV0hXRDpMZmvDp_O?usp=sharing. It contains key data for reproducing the results, including pre-generated code graph data in .pt format.
If the issue persists or if you encounter more problems, please feel free to let me know. I'll continue to monitor and help resolve your issue as much as possible.
Thank you!


Add support for exposing custom metrics generated by checks

Describe the feature you would like and why you want it
Currently kuberhealthy exposes only kuberhealthy_check and kuberhealthy_check_duration_seconds metrics per configured khcheck.
There are some cases where only these 2 metrics are not enough when defining KPIs or SLTs.
One such example could be an ingress or an egress check. It is not enough to have only the Fail/Success state and the duration of the check. You also need a metric like http call duration or if you have multiple http calls you probably want to count the percentage of the requests that failed/succeeded.
Some other examples might be to measure the k8s api latency or throughput or deployment scale time.
Additional context
One way to accomplish this freedom of exposing  custom metrics could be by extending the https://github.com/Comcast/kuberhealthy/blob/e712c07cee05d2d9f9f20352a5991a81fbcec636/pkg/health/state.go#L23 state of the check and adjust metrics generation method https://github.com/Comcast/kuberhealthy/blob/e712c07cee05d2d9f9f20352a5991a81fbcec636/pkg/metrics/exporter.go#L26
accordingly.
Then it will be the responsibility of the check itself to expose or not custom metrics. These metrics can be sent together with the report status
https://github.com/Comcast/kuberhealthy/blob/e712c07cee05d2d9f9f20352a5991a81fbcec636/pkg/checks/external/status/main.go#L7

I can give it a try if there are no other concerns regarding this approach.

Thanks for the carefully thought through proposal here.  My initial impression is that this would be complicated due to Kuberhealthy not having a database of any kind.  The metrics would have to either be stored in the khstate objects, or kept in memory, which would be wiped on any pod restart event.
We actually use pushgateway internally ourselves for metrics that don't come with Kuberhealthy out of the box...
I am leaving this one open for more thought/consideration from others, but I am leaning toward this not being on the critical path of what the project looks to accomplish.


update(client/deluge): broaden indicators determining completion in client

this accepts paused torrents if either marked with progress 100% or no remaining download, as well as adds total_remaining to the acceptable completion indicators
this may not entirely solve #711 but may increase likelihood that it is not show-stopping

I'm suspicious that this fixes it, but it seems fine to try.

Yea, I'm not convinced it will, but it can't hurt.
As I mentioned in #711 I'm currently testing a theory, but replicating the issue seems rather evasive.


String paragraph/line searching can attempt to form invalid indices

In our implementation of _getBlock which is used to find the ranges of paragraphs/lines within strings, we can potentially attempt to form an invalid index of a string resulting in a runtime crash. If the range provided begins at the start of the string and only contains one separator character (as is the case with a string that is only a separator character), then we attempt to form an index before the startIndex of the string while iterating backwards to find the contents end. This PR updates the _matchesSeparators function to use the formIndex(_:offsetBy:limitedBy:) function rather than calling index(before:) on what may potentially be the startIndex to ensure that we don't form invalid indices and instead break out of the loop as soon as we reach the end of the separator or the string.

@swift-ci please test

Addressed an off-by-one logic issue where separators would match even if their final UTF-8 scalar differed.
@swift-ci please test


Fix openblas static linking

Well, again another hack for the static linking with OpenBLAS + gfortran (Linux, my UMB claster).
Without the -lpthread flag I am getting  this linking error below.
Question: should these hacks remain in local CMakeFiles.txt  files ?
----------------------------- Captured stderr call -----------------------------
make: Warning: File `Makefile' has modification time 28 s in the future
make[1]: Warning: File `CMakeFiles/Makefile2' has modification time 28 s in the future
make[2]: Warning: File `src/CMakeFiles/example.dir/flags.make' has modification time 28 s in the future
make[2]: warning:  Clock skew detected.  Your build may be incomplete.
make[2]: Warning: File `src/CMakeFiles/example.dir/flags.make' has modification time 28 s in the future
make[2]: warning:  Clock skew detected.  Your build may be incomplete.
make[2]: Warning: File `src/CMakeFiles/example.dir/flags.make' has modification time 28 s in the future
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
/home/milias/bin/openblas/lib/libopenblas.a(memory.o): In function `blas_shutdown':
memory.c:(.text+0x23): undefined reference to `pthread_mutex_lock'
/home/milias/bin/openblas/lib/libopenblas.a(memory.o): In function `openblas_fork_handler':
memory.c:(.text+0x3e0): undefined reference to `pthread_atfork'
/home/milias/bin/openblas/lib/libopenblas.a(memory.o): In function `blas_memory_alloc':
memory.c:(.text+0x6f7): undefined reference to `pthread_mutex_lock'
memory.c:(.text+0x72b): undefined reference to `pthread_mutex_unlock'
/home/milias/bin/openblas/lib/libopenblas.a(memory.o): In function `blas_shutdown':
memory.c:(.text+0xb4): undefined reference to `pthread_mutex_unlock'
/home/milias/bin/openblas/lib/libopenblas.a(blas_server.o): In function `blas_thread_shutdown_':
blas_server.c:(.text+0x1a8): undefined reference to `pthread_mutex_lock'
blas_server.c:(.text+0x20c): undefined reference to `pthread_mutex_lock'
blas_server.c:(.text+0x21f): undefined reference to `pthread_cond_signal'
blas_server.c:(.text+0x22c): undefined reference to `pthread_mutex_unlock'
blas_server.c:(.text+0x259): undefined reference to `pthread_join'
blas_server.c:(.text+0x288): undefined reference to `pthread_mutex_destroy'
blas_server.c:(.text+0x292): undefined reference to `pthread_cond_destroy'
blas_server.c:(.text+0x2b0): undefined reference to `pthread_mutex_unlock'
/home/milias/bin/openblas/lib/libopenblas.a(blas_server.o): In function `goto_set_num_threads':
blas_server.c:(.text+0x3a5): undefined reference to `pthread_mutex_lock'
blas_server.c:(.text+0x3fc): undefined reference to `pthread_mutex_init'
blas_server.c:(.text+0x40d): undefined reference to `pthread_cond_init'
blas_server.c:(.text+0x429): undefined reference to `pthread_create'
blas_server.c:(.text+0x43d): undefined reference to `pthread_mutex_unlock'
/home/milias/bin/openblas/lib/libopenblas.a(blas_server.o): In function `blas_thread_server':
blas_server.c:(.text+0x501): undefined reference to `pthread_mutex_lock'
blas_server.c:(.text+0x516): undefined reference to `pthread_mutex_unlock'
blas_server.c:(.text+0x5b7): undefined reference to `pthread_cond_wait'
/home/milias/bin/openblas/lib/libopenblas.a(blas_server.o): In function `blas_thread_init':
blas_server.c:(.text+0x668): undefined reference to `pthread_mutex_lock'
blas_server.c:(.text+0x701): undefined reference to `pthread_mutex_init'
blas_server.c:(.text+0x70d): undefined reference to `pthread_cond_init'
blas_server.c:(.text+0x721): undefined reference to `pthread_create'

I guess it should be one commit (the second undoes the first). And why is this
not in the module but in the test sources? If all the workarounds are in the test
sources, then the question is what do they test?

Hi,
I will rebase it.
I was waiting for your statement as where it should go.  The strange think is that on travis-ci Linux installation this openblas static linking works without the -lpthread flag. So I put it into the local cmake settings.
My cluster has Scientifix Linux where this is needed. In between I will label this PR as WIP before asking for final approval.

I need to close it. Unfortunately WIP PRs flood my email inbox and GitHub notifications, too, and distract from "real" PRs. I really appreciate your work but I suggest to do WIP code reviews on forks and only submit "final" PRs. If you want my opinion on code done on forks, just involve me via "@bast".


Amd

This PR follows #247 and it is required in order to make sauce labs to run to all platforms /cc @mattiaerre

All is green after running in all browers, so merging this @Hobbit71


Disable proxy for consul-wait-for-leader.sh

For customers who have a proxy set in their environment variable, the consul-wait-for-leader.sh script will time out after attempting to connect to the proxy host.
Add --noproxy localhost to the curl command line

A better solution is to set the no_proxy=localhost environment variable where the other *_proxy vars are set. This way we don't have to modify individual commands.


[WIP] Data importer

This PR represents a major change in approach to how we import / export data.
Previously we have relied on external libraries for importing / exporting - which all come with various downsides.
The main idea of this PR is to utilize the existing DRF API framework for both importing and exporting of data. This should provide the following major advantages:

Significantly reduce code duplication
Hook in to existing API framework which is well tested
Improve speed of import / export
Allow finer control over import / export processes
Allow specification of custom import hooks
Expose import / export processes to custom plugins

Framework
Uses existing DRF serializers, just need to register with the @register_importer() decorator.
Related Issues

Closes https://github.com/inventree/InvenTree/pull/6792
Closes https://github.com/inventree/InvenTree/issues/6376
Closes https://github.com/inventree/InvenTree/issues/6389
Closes https://github.com/inventree/InvenTree/issues/6955
Closes https://github.com/inventree/InvenTree/issues/6956
Closes https://github.com/inventree/InvenTree/issues/7431
Closes: https://github.com/inventree/InvenTree/issues/6321
Ref: https://github.com/inventree/InvenTree/issues/5571
Ref: https://github.com/inventree/InvenTree/issues/6664
Ref: https://github.com/inventree/InvenTree/issues/6647
Ref: https://github.com/inventree/InvenTree/issues/3101

TODO

[x] Remove the APIDownloadMixin class (and all references)
[x] Remove the download_queryset method in a many existing views
[x] Use database model verbose_name if serializer does not provide a label attribute
[x] Add / update unit testing for new dataset export approach
[x] Add worker task to cleanup old import / export sessions
[x] Ensure all 'required' fields are mapped before progressing
[x] Add data import section to "admin center"
[x] Add in-line rendering of related model fields
[x] Improve query efficiency of "import session" endpoint (takes ~1 second!?)
[x] Add unit testing for new dataset import approach

Further Work

[ ] Support "default values" when importing dataset
[ ] Allow user to navigate back to previous data steps
[ ] Add unit test for cleanup backup task
[ ] Ensure user has correct permissions before creating a new import session
[ ] Expose import / export functions to plugins
[ ] Allow plugins to define custom export fields / processes
[ ] Allow bulk data export action to django admin interface
[ ] Replace existing "BOM Import" tool with new importer
[ ] Replace existing "BOM export" tool with new exporter
[ ] Replace existing "part import" tool with new importer
[ ] Replace existing "purchase order import" tool with new importer
[ ] Remove django-import-export framework entirely
[ ] Allow bulk "update" via import (override current records, not create new ones)
[ ] Remove all custom import UI template code (html templates, legacy)
[ ] Check that all existing "django-import-export" models are covered by new framework
[ ] Show "original row data" for each row when importing data
[ ] Allow user to create new related model "in place"
[ ] Add "context specific" importers for particular tables, rather than just in the admin center
[ ] Improve efficiency of data import process (currently takes ~1 minute for 1,000 records)
[ ] Impove user interface for data import - in particular, react hook efficiency
[ ] Enable per-column specification of "default" values in the admin portal
[ ] Documentation for new data import process

Very exciting! Is the removal of json/yaml a permanent change or something that was not in scope for the initial implementation?

@SchrodingersGat is this still slated for 1.0 or is 0.16 or sooner realistic?

I'm hoping for 0.16.0 release. Now that the refactoring is done I have some time to commit to this. Soon it should be ready for a first round review.

Very cool. There is a lot of untested frontend code that can be removed once the new importer works.

Yes once we can remove the old importer code, that will be fantastic.

Can we merge this?

@smithcv this is not finalized and tested. Please allow the time needed to get it done properly with the resources the project has.

@smithcv thanks for your interest but still a lot of work here. If you or your company would like to contribute to the development of this feature, you can provide funding against this issue :)

@SchrodingersGat I have created a branch that removes all import-export dependant things: https://github.com/matmair/InvenTree/tree/remove-imports
There are a lot of places where we use resource definitions for export - making this more generic (maybe as a view mixin?) would be great to remove code duplication

Data Export Profiling
The new data export functionality uses the optimized API serializers, and (at the current implementation) is already a huge improvement over the existing export tool which uses django-import-export

API Endpoint
Record Count
Old Exporter
New Exporter

/api/part/
5,000
did not return after multiple minutes
17.3s , 11 DB hits

/api/stock/
1,100
3.4s in 890 DB hits
4.0s in 16 DB hits

/api/stock/location/
20
0.4s in 38 DB hits
0.5s in 4 DB hits

/api/order/po/
10
0.4s in 48 DB hits
0.5s in 23 DB hits

However there is still some optimizing to be done here. Still need to profile where the time is being spent on the data export. Even though django-import-export uses a lot more DB hits their export is still faster in some instances.

@SchrodingersGat that looks very promising

@SchrodingersGat you need any assistance with this?

I need to remember where I was up to and what the next steps might be. There is still a lot of work to go..
So far I have got a framework in place for creating an import session, and extracting data from table. This data is split into rows which get "serialized" against the matching serializer for validation and ingest.
Major Tasks

[ ] The user interface elements (in PUI) still need a lot of work
[ ] Do we want to support this framework within the django admin interface?
[ ] Still need to work out the final steps for validating data rows and then saving to the database
[ ] No unit testing as yet

If you wanted to take a look at where I'm up to so far and provide input on the approach in general. I'm hoping to get back to this soon, now that the reporting has been refactored. It is a major blocker for a lot of outstanding issues.

This is now progressed far enough to merge into master. Exporting data from tables (in CUI and PUI) now uses the new data export tool and is much faster. Data are serialized into the file using the existing DRF layer, which is very query efficient and reduces code duplication.
There is now also a "Data Import" section in the PUI admin page - which is functional, but still requires some work:

There are a lot of outstanding tasks / improvements to do (see outstanding tasks above) which will be pushed out into new issues.
The goal is to remove the reliance on django-import-export entirely.

@SchrodingersGat thank you for pushing this ahead! Are any of these goals blocking for 0.16.0 release? We should probably log and tag those separately to not lose track over the next month.

I will make some separate issues soon, while it is still fresh on my mind.


How to name certain enum values

Example names of enum values we get from TCGC.
V7.6_preview.1
V2022_12_01_preview
Propose we name them as follows.
V7Dot6_Preview
V2022_12_01Preview
The . character isn't legal in a name (syntax error), so must be removed.
For patterns like 01_preview the linter complains that it should be 01Preview.

I'd prefer _ everywhere e.g., V7_6_Preview_1 or V2002_12_01_Preview. The "Dot" makes it harder to read and just looks funny, and we shouldn't sandwich "01" and "Preview" together like that - especially when they weren't before. That's also consistent with idiomatic rust naming for both members and constants.

Agreed on _ instead of Dot.
Note that the linter will complain about 01_Preview which is why I suggested the sandwich.

Have an example of the linter message handy? We can always override it in those cases.

pub enum Versions {
    V2002_12_01_Preview,
}

warning: variant `V2002_12_01_Preview` should have an upper camel case name
  --> sdk\keyvault_secrets\src\generated\enums.rs:33:5
   |
33 |     V2002_12_01_Preview,
   |     ^^^^^^^^^^^^^^^^^^^ help: convert the identifier to upper camel case: `V2002_12_01Preview`
   |
   = note: `#[warn(non_camel_case_types)]` on by default

If we're happy with the name as is then yeah we can suppress.

Cringe: "upper camel case". It's called PascalCase!
Yeah, I prefer the look of the underscores, and it jives with some of our other languages as well. I think just slap the #[allow(non_camel_case_types)] on the enum (or each member, but less noisy on the parent).

Actually, why are we generating an enum for versions at all? Per guideline discussions (and written guidelines), we always target the latest version but do allow the customers to pass in an api_version as a string. Isn't this similar to Go? So why should we define an enum? Just informational purposes or more as a backing store for supported versions?

I added guidelines, but we should answer my question above before merging. /cc @ronniegeraghty

What if we just don't emit version enums at all?
We don't use these in Go or C++, for example.
We need to set the api-version corresponding to the SDK we are code generating but I suggest we just let customers pass in a string for any other api-version they want to try (but we do not officially support).
Also, while TypeSpec files will have multiple versions in them, we will remove preview versions from TypeSpec files as new previews/GAs ship and we will (probably) also remove old GA version if/when a new breaking GA version is introduced - a new breaking GA is tantamount to a new V 1.0. We're still discussing this last proposal.

The enums for the applicable API versions are defined by the tsp. Here's an example from KV.
https://github.com/Azure/azure-rest-api-specs/blob/56e1117535d28d9f0489c93d92a1ce01c3eeac2f/specification/keyvault/data-plane/Security.KeyVault.Secrets/main.tsp#L41-L53
What I'm reading is that we want to skip emitting these and just emit a String param instead, is that correct? Is it mandatory, or do we make it optional, with a client-side default value based on what the tsp tells us?

Correct. Ignore the enum in the TypeSpec and use somehow set the default api-version value for clients to the version that is being used to generate the code. There must be some way to tell the code generator what api-version to generate code for, right?

Yes, we know the default value to use. I will make this change in the codegen today.


error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550 with Scan Assoc Grps

I'm using version 0.08 from date 2017-03-16. After couple of iterations of scanning and setting values I hit  Clean Up button. Now every time when I hit Scan Assoc Grps button I see following error in Live logging:
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:36: error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:36: info Association Group #1 contains nodes: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:35: info Association Group #1 contains destinations: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:34: error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:33: info Association Group #1 contains nodes: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:32: info Association Group #1 contains destinations: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:32: error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:31: info Association Group #1 contains nodes: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:30: info Association Group #1 contains destinations: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:29: error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:28: info Association Group #1 contains nodes: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:27: info Association Group #1 contains destinations: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:27: error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:26: info Association Group #1 contains nodes: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:25: info Association Group #1 contains destinations: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:24: error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:23: info Association Group #1 contains nodes: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:23: info Association Group #1 contains destinations: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:22: error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:21: info Association Group #1 contains nodes: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:20: info Association Group #1 contains destinations: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:19: error java.lang.IndexOutOfBoundsException: toIndex = 3 @ line 1550
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:19: info Association Group #1 contains nodes: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:18: info Association Group #1 contains destinations: [01] (hexadecimal format)
97773ca0-3a2e-4d9f-b6d0-57ef1c287b25  22:20:17: info scanAssocGroups(): Scanning Association Groups (#0 to #10).

The device is Fibaro Wall Plug. This is what I see I hit Print General button (I have reversed order of log entries):
General Information:
Device Name: ...
Raw Description: zw:Ls type:1001 mfr:010F prod:0602 model:1001 ver:3.02 zwv:4.05 lib:03 cc:5E,22,59,56,7A,32,71,73,98,31 sec:85,70,72,5A,8E,25,86 role:05 ff:8700 ui:8700
Supported Command Classes: [APPLICATION_STATUS (0x22), SWITCH_BINARY (0x25), SENSOR_MULTILEVEL (0x31), METER (0x32), CRC_16_ENCAP (0x56), ASSOCIATION_GRP_INFO (0x59), DEVICE_RESET_LOCALLY (0x5A), ZWAVEPLUS_INFO (0x5E), CONFIGURATION (0x70), NOTIFICATION (0x71), MANUFACTURER_SPECIFIC (0x72), POWERLEVEL (0x73), FIRMWARE_UPDATE_MD (0x7A), ASSOCIATION (0x85), VERSION (0x86), MULTI_CHANNEL_ASSOCIATION (0x8E), SECURITY (0x98)]
Security: Device is paired securely.
=> Command classes supported with security encapsulation: UNKNOWN (0xNULL)
=> Command classes supported for CONTROL with security encapsulation: UNKNOWN (0xNULL)
Manufacturer ID: 010F
Manufacturer Name: null
Product Type ID: 0602
Product ID: 1001
Firmware Metadata: Firmware ID: 0612, Checksum: 5432
Application (Firmware) Version: 3.02
Z-Wave Protocol Version: 4.05
Z-Wave Library Type: 03 (Enhanced Slave)
Powerlevel: NormalPower
Protection Mode: [ Local: null (null), RF: null (null) ]
Switch_All Mode: null (null)

Discovery Stats:
Number of association groups discovered: 3 [Print Assoc Groups]
Number of endpoints discovered: 0 [Print Endpoints]
Number of parameters discovered: 22 [Print Parameters]
Number of unique command types received: 16 [Print Commands]
Number of MeterReport types discovered: 2 [Print Sensor]
Number of NotificationReport types discovered: 0 [Print Sensor]
Number of SensorMultilevelReport types discovered: 1 [Print Sensor]

According to manual it has indeed three association groups that has been discovered. I'm not sure why it  prints those errors.

This also appears to be a bug with the physicalgraph.zwave.commands.securityv1.SecurityMessageEncapsulation.encapsulatedCommand() method, which is part of the SmartThings core Z-Wave library.
I recommend you raise this on the SmartThings community forum and tag Duncan.

I posted here also: https://community.smartthings.com/t/z-wave-library-changes-feb-2017/79021/11

Thanks! I haven't realized that this could be an issue with the platform.


[on-hold] [needs SDK update] update trending to use trending_score

probably wait for sdk release + test

Closing for now, I'll re apply when we move over to scribe.


feat(test): Created test workflow for adding a new Torlist entry

PR related to this trello task --> https://trello.com/c/24GJsY5h

:tada: This PR is included in version 1.1.0 :tada:
Your semantic-release bot :package::rocket:


Eridium not being updated

Everytime I edit eridium, by tens, hundreds, thousands, even just +1, and whenever I load my game it reverts to the original value, any ideas ?

Found a workaround by creating a new save to prevent cloud save from ruining it


undefined method "followers" for nilClass

I assume this gem is deprecated?
clearbit from clearbit-ruby works fine, but this tool throws an error on every email address I put in. Just thought I'd share in case you didn't know
/Users/jamie/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/clearbit-leadscore-0.0.5/lib/clearbit/lead_score/score.rb:35:in `calculate': undefined method `followers' for nil:NilClass (NoMethodError)
        from /Users/jamie/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/clearbit-leadscore-0.0.5/lib/clearbit/lead_score.rb:51:in `lookup'
        from /Users/jamie/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/clearbit-leadscore-0.0.5/bin/clearbit-leadscore:45:in `<top (required)>'
        from /Users/jamie/.rbenv/versions/2.2.2/bin/clearbit-leadscore:23:in `load'
        from /Users/jamie/.rbenv/versions/2.2.2/bin/clearbit-leadscore:23:in `<main>'

Also getting this error! :)


Use F.fold within decoder for image reconstruction

nn.Fold (https://pytorch.org/docs/1.7.1/generated/torch.nn.Fold.html?highlight=fold) is designed for stitching together a large canvas from subimages that overlap.
This PR replaces our previous reconstruction method, which used nested for-loops within Python. The biggest advantage is performance and scalability, since I believe nn.Fold takes advantage of the GPU. Moreover, this will simplify the most complex part of decoder.py.

Just as sanity checks, did you check if all tests are passing in the GPU ? Have you looked at some of the images by eye just to ensure they look ok / similar to before ?

I ran all tests ensuring that "torch.allclose()" never failed for the original and new method, but I didn't do it on the GPU. I'll make sure tests pass on GPU before merging.

I ran all tests ensuring that "torch.allclose()" never failed for the original and new method, but I didn't do it on the GPU. I'll make sure tests pass on GPU before merging.

Sounds good if you checked it with torch.allclose() you can probably merge. It wouldn't hurt to run in the GPU just to make sure. Thanks!

This passed all GPU tests (other than multi GPU, which was taking hours so I cancelled it)


Why is (isLeaf) function called on every single node in the tree when a node is selected?

Why is (isLeaf) function called on every single node in the tree when a node is selected?

It does seem like a potential problem for me. I have yet to look into it more, but with a tree of 5600 (56 * 10 * 10), it behaves as follows:

Reload the page, tree is built fast, expand/collapse works fast
Select a node label
Expand/collapse as well as everything else (e.g. entering text to input fields) has a serious delay.


README: Add Nxt Games modules

Added the open source modules created at Nxt Games for the Panda3D game engine

I believe these will eventually make a nice addition to the list. However, I would prefer to see a few things first:

A release that makes it easy to get the module (e.g., a wheel on PyPI)
The Travis badges not reporting an error

More docs would be nice, but I see that there are least examples, which is good.

I can work on the PyPi releases. Travis might be more of an issue. That was not my doing originally. I'm not familiar with Travis enough to setup the environments and dependencies for that.

Then I would recommend taking the badge out of the readme.

I took them out. I also starting prepping the modules for PyPi. However, I don't know if there is a clean way to do p3d-discord. From what I've learned before interrogate modules are picky about where they are placed on Mac OS. I don't know as though the standard site packages install will run on a Mac.


Fablib uses the same key for all slices.

Users may use different keys for different slices. But currently, fablib assumes that all slices are created with the same key.
We need to somehow store the different keys that are used to access the different slices. Or maybe, we don't have to go over all slices when we are working with one slice.

This one probably needs us to wait until proper key management is implemented in FABRIC/portal.  Then we can implement a solution that works with the long-term key management system.


Coverage remained the same at 93.781% when pulling 4fac57f8998dce57610fd65691c8320845197546 on eatonphil:no-fetch-in-header into 278704210c3c3dc84dcf80bfbadb51ff6ef7092c on linode:master.

:/


Bot DMs unchanged Locations

Bug Report
Description
Locations are repeatedly DM'd despite showing no apparent change. This has already discouraged a few users from using the system.
Steps to Reproduce
(Not easily replicable since it's data and time dependent.)
Expected behavior
Locations should be DM'd only if they show a change visible to the user.
Screenshots

https://twitter.com/messages/1267652725782663170-1364634278861053957
Additional context
This occurs presumably because the changes are limited to non-user-facing fields.

Looks like we lost as many as five users this week because of this bug:

https://twitter.com/messages/1216116764741791745-1364634278861053957
https://twitter.com/messages/1267652725782663170-1364634278861053957
https://twitter.com/messages/935780726-1364634278861053957
https://twitter.com/messages/613583276-1364634278861053957
https://twitter.com/messages/76654639-1364634278861053957


LICENSE v LICENCE2 confusion

Hi,
I'm confused about how LICENSE2 applies. Is it a license only for logos? => if you do not use Kanaries logos it isn't relevant?
Or is it a license which is intended to add additional restrictions to graphic walker overall. If that's the case, I guess it would be useful if not critical to add clarity, but IANAL. +if that's the case, is Apache 2.0 a meaningful label to associate with the repo?

It is required not to apply any changes/remove to the logos in graphic-walker.
The software is basically under apache2 except the logo part. This is a common aproach in many open source softwares and make it more sustainable for the project.


Updatedata

Abhishek,
I updated the data using parth's migration tool. I will add the tests using this data in a different pr

Rahul

Looks good. +1


Terminal command error for both versions after 0.2.0

Hi! First off I'm loving this mod. However, I seem to encounter an issue when entering "moons" into the Terminal for every version after 0.2.0
This is the error I receive in the cmd window the second I try to send 'moons'
[Error  : Unity Log] IndexOutOfRangeException: Index was outside the bounds of the array.
Stack trace:
TerminalPlus.Nodes.MoonsPage () (at <2d85965c653f4f40b949ecbb28eb9380>:IL_01FB)
TerminalPlus.TerminalPatches.TerminalEventPrefix (TerminalNode node, Terminal __instance) (at <2d85965c653f4f40b949ecbb28eb9380>:IL_03EA)
(wrapper dynamic-method) Terminal.DMDTerminal::RunTerminalEvents(Terminal,TerminalNode)
Terminal.LoadNewNode (TerminalNode node) (at :IL_0007)
(wrapper dynamic-method) Terminal.DMDTerminal::OnSubmit(Terminal)
UnityEngine.Events.InvokableCall.Invoke () (at :IL_0010)
UnityEngine.Events.UnityEvent1[T0].Invoke (T0 arg0) (at <e27997765c1848b09d8073e5d642717a>:IL_0049) TMPro.TMP_InputField.SendOnEndEdit () (at <9baa8119dbcd498f902c1a92f10843c1>:IL_000E) TMPro.TMP_InputField.ReleaseSelection () (at <9baa8119dbcd498f902c1a92f10843c1>:IL_001B) TMPro.TMP_InputField.DeactivateInputField (System.Boolean clearSelection) (at <9baa8119dbcd498f902c1a92f10843c1>:IL_00B0) TMPro.TMP_InputField.OnUpdateSelected (UnityEngine.EventSystems.BaseEventData eventData) (at <9baa8119dbcd498f902c1a92f10843c1>:IL_007C) UnityEngine.EventSystems.ExecuteEvents.Execute (UnityEngine.EventSystems.IUpdateSelectedHandler handler, UnityEngine.EventSystems.BaseEventData eventData) (at <17de86d66286445fa95af59fa8fa644d>:IL_0000) UnityEngine.EventSystems.ExecuteEvents.Execute[T] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction1[T1] functor) (at <17de86d66286445fa95af59fa8fa644d>:IL_0067)
UnityEngine.EventSystems.EventSystem:Update()
Is this possibly from a conflict with another mod installed after one of your updates or is this from the update itself?
Thanks!

Hi! First off I'm loving this mod. However, I seem to encounter an issue when entering "moons" into the Terminal for every version after 0.2.0
This is the error I receive in the cmd window the second I try to send 'moons'
[Error : Unity Log] IndexOutOfRangeException: Index was outside the bounds of the array. Stack trace: TerminalPlus.Nodes.MoonsPage () (at <2d85965c653f4f40b949ecbb28eb9380>:IL_01FB) TerminalPlus.TerminalPatches.TerminalEventPrefix (TerminalNode node, Terminal __instance) (at <2d85965c653f4f40b949ecbb28eb9380>:IL_03EA) (wrapper dynamic-method) Terminal.DMDTerminal::RunTerminalEvents(Terminal,TerminalNode) Terminal.LoadNewNode (TerminalNode node) (at :IL_0007) (wrapper dynamic-method) Terminal.DMDTerminal::OnSubmit(Terminal) UnityEngine.Events.InvokableCall.Invoke () (at :IL_0010) UnityEngine.Events.UnityEvent1[T0].Invoke (T0 arg0) (at <e27997765c1848b09d8073e5d642717a>:IL_0049) TMPro.TMP_InputField.SendOnEndEdit () (at <9baa8119dbcd498f902c1a92f10843c1>:IL_000E) TMPro.TMP_InputField.ReleaseSelection () (at <9baa8119dbcd498f902c1a92f10843c1>:IL_001B) TMPro.TMP_InputField.DeactivateInputField (System.Boolean clearSelection) (at <9baa8119dbcd498f902c1a92f10843c1>:IL_00B0) TMPro.TMP_InputField.OnUpdateSelected (UnityEngine.EventSystems.BaseEventData eventData) (at <9baa8119dbcd498f902c1a92f10843c1>:IL_007C) UnityEngine.EventSystems.ExecuteEvents.Execute (UnityEngine.EventSystems.IUpdateSelectedHandler handler, UnityEngine.EventSystems.BaseEventData eventData) (at <17de86d66286445fa95af59fa8fa644d>:IL_0000) UnityEngine.EventSystems.ExecuteEvents.Execute[T] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction1[T1] functor) (at <17de86d66286445fa95af59fa8fa644d>:IL_0067) UnityEngine.EventSystems.EventSystem:Update()
Is this possibly from a conflict with another mod installed after one of your updates or is this from the update itself?
Thanks!

Hi! Sorry for the delay, I had finals this last week. I can't see what the problem is off the top of my head, but my guess is that my mod isn't recognizing the full list of moons; could you send me a list of the mods you were using? I recognize the error type and will try to fix it with this next update :)

Sure thing! Was rather surprised to see the error suddenly pop up after an update. Here are the moons;

Big Boos Haunt Moon (https://thunderstore.io/c/lethal-company/p/PanMan/Big_Boos_Haunt_Moon/)
BikiniBottom (https://thunderstore.io/c/lethal-company/p/G7Exid/BikiniBottom/)
Ether (https://thunderstore.io/c/lethal-company/p/LethalMoonTeam/Ether/)
Generic Moons (https://thunderstore.io/c/lethal-company/p/Generic_GMD/Generic_Moons/)
LethalCenterHospitalMap (https://thunderstore.io/c/lethal-company/p/Kimukirano/LethalCenterHospitalMap/)
Luigis Mansion (https://thunderstore.io/c/lethal-company/p/PanMan/Luigis_Mansion_Moon/)
SR Marooners Bay (https://thunderstore.io/c/lethal-company/p/FissionMetroid101/SR_Marooners_Bay/)
Wesleys Moons (https://thunderstore.io/c/lethal-company/p/Magic_Wesley/Wesleys_Moons/)
Zelda Moons (https://thunderstore.io/c/lethal-company/p/Dafini/Zelda_Moons/)

Thanks for the help! Really enjoy the organization in your mod.

Thanks! One last thing, were you using this exact list before 0.2.0? That'll help me narrow down whether it's an incompatibility problem or just a bug from the mod

I believe Big Boos, SR Marooners Bay, and possibly Zelda Moons / BikiniBottom were not included at first, but I added them after receiving the error and downgrading back to 0.2.0.

The newest update allows everything to work properly. Thank you very much for your hard work!


3DMark 99 can't detect DirectX

First of all, i would like to thank you for this impressive project.
After creating a vm in Virtualbox, and installing the driver, i tried to run 3dmark99, but it seem to not detect direct x, even though in the setup it is recognized.

Dxdiag works, it also completes the rotating cube test.
This Happens with every accelerator Virtualbox has, and Vmware too (dont remember if i ever tested qemu, but it was giving me some problems), maybe ill' try to see what's going on with dependency walker at a later time.

Forgot to mention: i am running Virtualbox 6.1 on Linux Mint 21.1, on a Ryzen 3700X cpu

Hello, 3DMark99 has known issue with DX detection – I don’t know if it is problem with DX9 or 3D Mark simply doesn’t like my replacement of ddraw.dll – here is the issue and workaround: https://appdb.winehq.org/objectManager.php?sClass=version&iId=6776&bShowAll=true
Workaround: download collection of 3D Mark patchers from here: https://www.vogons.org/viewtopic.php?p=641489#p641489 – you need only “3dmark.exe” and replace original file. After it you should be able to test your virtual GPU performance :-)

Oh wow, now it works really well, thanks! :)
Since it seems to be an issue not specific to softgpu, can i close the issue?

@JHRobotics  You can also change the DLL version reporting in version.rc of DDRAW.DLL to match the final DirectX runtime on Windows 98 SE. This relieves the need to patch 3DMark EXE and has the benchmark run in pristine condition.
Here's what I use
#define WINE_FILEVERSION 4,09,00,0904
#define WINE_FILEVERSION_STR "<IP_ADDRESS>4"
#define WINE_PRODUCTVERSION 4,09,00,0904
#define WINE_PRODUCTVERSION_STR "<IP_ADDRESS>4"