panicked at 'overflow when subtracting duration from instant'

I have included bkt in one of our internal tool. I got at least 5 complaints already from users having such error (MacOs users):
thread 'main' panicked at 'overflow when subtracting duration from instant', library/std/src/time.rs:424:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

or (distinct column number on this one) Here it's bkt v0.7.1
thread 'main' panicked at 'overflow when subtracting duration from instant', library/std/src/time.rs:424:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Clearing the bkt-cache folder was our quick fix.
I'm sorry I don't have a backtrace yet, but will try to provide it next time it happen. I thought it might be worth to already open that issue.

Ooh interesting, yes please let me know if you uncover a backtrace. I have never observed this but we should definitely fix it. Will take a deeper look this evening but if you identify a fix in the meantime feel free to send a PR.

I think this is https://github.com/rust-lang/rust/issues/100141 - surprising!

Sounds to be our issue. Do you think the workaround will help?

Yeah I'm sure we can work around it :) If you're able to get a backtrace or reproduction steps that'll help pinpoint things but I'll do some more investigating this weekend.

Unfortunately not soon. I'm a Linux user, so I'm not concerned. And no one reported it since I asked internally…
:crossed_fingers:

Could you add RUST_BACKTRACE=1 to your internal tool (if you haven't already)?

@kumy can you also share any more details about how your tool is invoking bkt? E.g. is it using a long --ttl, or passing any other flags? I tried to reproduce the trace by caching a command with a long TTL and rebooting, but I think my Mac isn't affected by this bug - it appears to use a signed type on my machine rather than a u64.
I suspect the most reasonable fix will unfortunately be to stop using Instant and instead swap to chrono::DateTime or similar. I don't see a reasonable way to fix a line like this one without changing either the type or the behavior.

(From memory, more details to come on monday)
From the last user report, the failing call was with a --ttl 15min. But it may not be related especially to this cache entry, I think this is the first call we have in all the script, so maybe related to pruning routine when bkt is launched.
I hope to be able to provide a trace as soon as I have a new user report.

can you also share any more details about how your tool is invoking bkt?

We're using that pattern:
bkt \
  --scope="${ENVIRONMENT}.${params[0]}.foo" \
  --ttl="${TTL}" \
  --discard-failures \
  -- \
  $THE_COMMAND "${params[@]}"

is it using a long --ttl

We're using ttl: 15min, 1h, 1d, 1w, depending on the queried data

Anything else we can do to help here?

😆 tell my employer to prioritize this bug! I will try to figure out the right fix for this soon, it's a little messy.

Given the few number of occurrences we had on our side. I think we could leave with: destroying the failing cache entry and return the real content from "the command". Maybe displaying a line on stderr to warn when this error arise :shrug: .  (That is my own personal preferred choice)
I asked users, and they cold boot every morning. We are not storing the cache in a volatile dir, and we would like to keep it's content between reboots.
Sorry I'm not in the internal of bkt, I won't comment.

Thanks for bkt and for all your time!

Thanks for the context on #2, that does make sense. I would actually suggest changing the cache dir to be cleared on reboot to align with the behavior your Linux users experience. According to this post /tmp is cleared on boot while /var/tmp (which I believe is what OSX uses by default as $TMPDIR) is not, so passing --cache_dir=/tmp for your OSX users might be sufficient to prevent this issue.
I played around a little more with swapping to SystemTime everywhere, and despite its limitations it's looking like a viable change. I could swap to e.g. time but I'm not yet sure if there's sufficient value to doing so. Will flesh it out and publish a fix soon, probably this weekend.

I would actually suggest changing the cache dir to be cleared on reboot to align with the behavior your Linux

Hum, I may have badly explained it. We want to keep the cache between reboots. The cache (on all platform) is not in $TMP but somewhere in $HOME.

Oh you didn't mention using a custom cache directory above. In that case your OSX users will need to pick up this fix (once it's released).

FTR, one user reported the error this morning after powering his Mac...

@kumy apologies for the delay resolving this, I just pushed a commit that should mitigate this issue. I will cut a new release soon but if you / any of your users would like to test the updated binary before I do so that would be great! Artifacts can be found here.


fix: TopK operator should return correct results on dictionary column with nulls

Which issue does this PR close?

Closes #1030.
Rationale for this change

What changes are included in this PR?

How are these changes tested?

cc @huaxingao With this patch, the failed Iceberg test can pass now.

I ran TPC-H benchmarks comparing this PR to main and did not see any regression

Thanks @huaxingao @andygrove


feat: tinywl: Simple static borders

Nice feature, unfortunately the borders are formed inside the window, it will cover window surface. Would be nice if the borders are formed outside the window.

Hi! This patch was just for testing purposes I think we can go ahead and close this.


chore(release): add release workflow

Thank you for contributing to golib!
Please add a summary of your change
Does your change fix a particular issue?
Fixes #(issue)
Check list:

[ ] Mark if documentation changes are required.
[ ] Mark if tests were added or updated to cover the changes.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 75.84%. Comparing base (887ba58) to head (b76ca1a).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #66   +/-   ##
=======================================
  Coverage   75.84%   75.84%
=======================================
  Files          22       22
  Lines         828      828
=======================================
  Hits          628      628
  Misses        125      125
  Partials       75       75

Flag
Coverage Δ

unittests
75.84% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


NetherPortal glitch

-Bug: Glitched Netherportal and no item drops from player.
-How it happened: Go through nether portal and have a creeper explode on the other side as soon as passing through.
-NOTE: It happend on an older version. But I didn't find a bug fix in the patchnotes for it, so I thought it might still be in there.
-How it looks: (Dirt block placed by hand after)

Update on this. It still happens in the 2.0 Version.
I believe it is because the area gets unloaded when the player dies before all the portal blocks are rebuild.

I believe I have found a fix which will be implemented in the 1.20.1 release very soon! Happy days!


Is there any widget available for selecting Xaxis and yaxis from the combo box for creating any type of graph?

I have table of data with column names.
Is there any available componenet to acheive the below.
Combox box shows with list of graph in D3.
THe user selects bar graph, then based on the selection , it will show 2 more combo box with the list of columns.
1 for X axis, another 1 for Y axis. After selecting the columns it will render the bar graph.
Thank you for your time :+1:

@vijayakumarmariyappan This might be of interest: https://www.youtube.com/watch?v=j4VI47GlmGA&list=PL9yYRbwpkykuK6LSMLH3bAaPpXaDUXcLV&index=32
You may also like https://vega.github.io/voyager/
Generally this is not the best place to ask for help. Try the #help channel of the D3 Slack, or StackOverflow (D3 tag).

@curran Thanks again :+1:  Could you please update that if you have any other list of tools similar to voyager. Thanks for your help:)

@vijayakumarmariyappan Voyager is part of the Vega ecosystem, worth checking out all that stuff:

https://vega.github.io/vega/
https://github.com/vega - poke around in here

Al the best!

@curran
Thank you so much :+1:


Peer Review by Parth Samant

Summary
This repo aims to use data from Toronto's Watch Your Speed Program (WYSP) to investigate the relationship between traffic direction, prevalence of high-speed vehicles, and median driving speeds. It was found that North/Southbound traffic was correlated with higher median speeds then eastbound, and that more high-speed vehicles were correlated with a higher median speed.
Strong positive points:

Good job making sure that preambles of scripts are updated
Good job with the formatting of the paper - it is quite well organised.
Very good prose, it is quite well-written.

Critical improvements needed:

table 1 should be in the model section. I'm assuming it uses the modelsummary() function, so it would likely be better there.
From Figure 2 it is quite hard to decipher a between the amount of vehicles over 100 km/h and the median speed.
make sure to update sketches of graphs/dataset from the default

Suggestions for improvement:

If possible, transform the 'count of vehicles over 100km/h' in figure 2 to into a proportion/percentage (might show a better relationship)
Consider having more information on the model's limitations (in the model section)
Consider having more tables/graphs/etc. in the results section (instead of just the modelsummary() output)
Try to reference more literature in the surveys appendix section
In the abstract it may be good to include that although north/south speeds have a higher median, it is not by much

Evaluation:
R is appropriately cited (1 pts):
1/1: R is appropriately cited in the bibliography and in the paper
Data  is appropriately cited (1 pts):
1/1: data is appropriately cited in the bibliography and in the paper
Class paper (1 pts):
1/1: No indication this is a class project
LLM usage documented (1 pts):
1/1: It is mentioned that LLM usage was not involved for this paper.
Title (2 pts):
2/2: The title is concise and informative
Author, date, and repo (2 pts):
2/2: The author, date, and GitHub repo link are included.
Abstract (4 pts):
3/4: Overall pretty good, but I believe it could be better by more accurately showing the results that you found (as mentioned in critical improvements needed)
Introduction (4 pts):
4/4: The introduction shows the significance and the broader context to motivate. It also shows the paper's structure and results of the paper as well. Nice!
Estimand (1 pts):
1/1: The estimand (conditional median speed) is mentioned and explained.
Data (10 pts):
8.5/10: Quite well done! Consider the changes I mentioned above that you could use to make the graphs more obvious as to what they are trying to show. This could help the reader have a better grasp of the dataset and how it connects to your results/discussion.
Measurement (4 pts):
4/4: Quite in depth and provides a good overview of how this data was obtained/measured.
Model (10 pts):
8/10: This is quite well done, but consider having some more information on the limitations of the model. Also make sure to have evidence of model validation.
Results (10 pts):
7/10: Although the results have been summarised in table 1, table 1 is the output of modelsummary(). Perhaps more of the results could be conveyed as a table? For example, you could portray the average speed depending on direction as a table.
Discussion (10 pts):
9.5/10: Very well done! Some small nit-picky things such as mentioning that the difference in median speed based on direction is quite small.
Prose (6 pts):
5.9/6: Overall great pose — there are a very small number of spelling mistakes, though.
Cross-references (1 pts):
1/1: Cross-references are included
Captions (2 pts):
2/2: Captions are descriptive and communicate what the graph is about
Graphs/tables/etc (4 pts):
3/4: Overall quite good but could be improved to give a better grasp of the data.
Surveys, sampling, and observational data appendix (10 pts):
8/10: Quite good, but make sure to reference more literature that relates to the study.
Referencing (4 pts):
4/4: No errors
Commits (2 pts):
1/2: Make sure to have more descriptive commit messages.
Sketches (2 pts):
0/2: The paper does not have relevant sketches.
Simulation (4 pts):
4/4: The paper has good and relevant simulations
Tests (4 pts):
3/4: Tests were provided, but consider having more thorough/advanced tests.
Parquet (1 pts):
1/1: raw data and analysis data are both present.
Reproducible workflow (4 pts):
3/4: Generally reproducible workflow but make sure to add set.seed() in simulations
Miscellaneous (3 pts):
3/3: Presence of API's in model folder is quite impressive.
Enhancements (4 pts):
4/4: API enhancement was done for this paper.
Estimated overall mark:
95.9/112.
Other Comments
This was overall a very well done paper, and I am impressed you are able to already finish this much of the paper despite the deadline being December 1st.
Good luck with the paper! I hope some of my feedback can help. :)

Thank you for your review, changes were made accordingly.


New kotools.types.api Gradle plugin

Description
We would like to create a kotools.types.api binary plugin for configuring the API binary checks.
This plugin should use the Kotlin/binary-compatibility-validator Gradle plugin for providing the following tasks:

apiCheck for checking the backward compatibility of the API binaries
apiDump for updating the snapshot of the API binaries.

Checklist

[ ] Create an empty binary plugin and apply it to the library project.
[ ] Implement and test the new binary plugin.

We will not implement this binary plugin, because we don't need to configure the original plugin.


pin rtree version

This PR pins rtree v0.9.3 to prevent Windows failures in CI.

If this doesn't work it's time to seriously consider #522 and/or using pygeos.

Running into a ton of messy conflicts now. Not sure the best path forward.


Instruction for Kotlin/JS declarations

Looking at the deploy preview (https://react-query-git-fork-turansky-patch-1-tannerlinsley.vercel.app/overview), I don't think the page is visible anywhere? I think you'd need to wire it to the menu or so, but I'm not sure it deserves it's own page ... ?

I have used this page as example.
It also unavailable on site

Then it’s probably not a good source of inspiration 😅

@TkDodo You are right. New PR required


preserveTOC for rst (rst2html)

Is there a preserveTOC option for rst (as is for asciidoc)?
If not, would it be worth to implement it?

The TOC is preserved by default.
This .rst document
.. contents::

Section 1
=========

Section 2
=========

is rendered as:

Contents

        Section 1

        Section 2

Section 1

Section 2

Yes, but I would like to set preserveTOC: false and use built-in variable .TableOfContents.


Fixing a bunch of tests

This PR depends on #1530 so review/merge that one first.
Fixes the tests for the artifact, base, job, logger, ontology, parameters, reference, sql_connection objects. It also fixes test_setup.py.
It should be fast to review as there where almost no changes required to the objects themselves. Most of the changes are related to the import changes in the test files.

:+1:

Looks good, thanks @josenavas :+1:


Explorer frontend throwing "Internal server error"

I'm using the latest Docker images and the Explorer frontend will come up but accessing it throws an internal server error. The container logs are:
><EMAIL_ADDRESS>start

> dotenv -c -e ../frontend.env -- next start

ready - started server on <IP_ADDRESS>:3000, url: http://localhost:3000

warn  - You have enabled experimental feature(s).

warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

TypeError: Cannot read property '0' of undefined

    at getNearNetworkName (/near-explorer/frontend/.next/server/chunks/210.js:63:36)

    at Object.App.getInitialProps (/near-explorer/frontend/.next/server/pages/_app.js:1979:99)

TypeError: Cannot read property '0' of undefined

    at getNearNetworkName (/near-explorer/frontend/.next/server/chunks/210.js:63:36)

    at Object.App.getInitialProps (/near-explorer/frontend/.next/server/pages/_app.js:1979:99)

TypeError: Cannot read property '0' of undefined

    at getNearNetworkName (/near-explorer/frontend/.next/server/chunks/210.js:63:36)

    at Object.App.getInitialProps (/near-explorer/frontend/.next/server/pages/_app.js:1979:99)

I'm not sure why it's throwing this, as my envvars contain a NETWORK_NAME value:
                "NEAR_EXPLORER_CONFIG__NETWORK_NAME=localnet",
                "NEAR_EXPLORER_CONFIG__BACKEND_SSR__HOSTS__GUILDNET=explorer-backend",
                "NEAR_EXPLORER_CONFIG__BACKEND_SSR__HOSTS__MAINNET=explorer-backend",
                "NEAR_EXPLORER_CONFIG__BACKEND_SSR__HOSTS__TESTNET=explorer-backend",
                "NEAR_EXPLORER_CONFIG__BACKEND_SSR__SECURE=false",
                "NEAR_EXPLORER_CONFIG__BACKEND__HOSTS__GUILDNET=<IP_ADDRESS>",
                "NEAR_EXPLORER_CONFIG__BACKEND__HOSTS__TESTNET=<IP_ADDRESS>",
                "NEAR_EXPLORER_CONFIG__BACKEND__SECURE=false",
                "NEAR_EXPLORER_CONFIG__BACKEND_SSR__PORT=8080",
                "NEAR_EXPLORER_CONFIG__BACKEND__HOSTS__MAINNET=<IP_ADDRESS>",
                "NEAR_EXPLORER_CONFIG__BACKEND__PORT=49970",
                "PORT=3000",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "VERSION=v16.4.2",
                "NPM_VERSION=7",
                "YARN_VERSION=v1.22.10",
                "HOME=/tmp"

We can use a message for that.

Actually, we have a message.
But we didn't throw it because of a bug: https://github.com/near/near-explorer/commit/924c8320b6a311b8d17f9a29bcd581958258dd73


GMC: response timeout with some specific models

In GMC e2e tests, there are some failed case caused by response timeout,
tgi2.2.0 + meta-llama/CodeLlama-7b-hf  on xeon in codegen test
tgi2.2.0 + meta-llama/CodeLlama-7b-hf  on guadi is ok
switch to another model "HuggingFaceH4/mistral-7b-grok" also ok
due to the manifest test (without GMC) cannot detect this issue, we need to find out whether it's GMC's specific.

similar issue to #258


Can't import proto file using relative path using the CMake "nanopb_generate_cpp" macro

I'm using the CMake "nanopb_generate_cpp" macro but I can't import a .proto
file from a relative directory in the .proto source.
See my example:
Example
Input files:
a.proto
subDir/b.proto

Contents of a.proto:
import "subDir/b.proto";

The generated output would be:
/build/<path to output>/a.pb
/build/<path to output>/a.pb.c
/build/<path to output>/a.pb.h
/build/<path to output>/b.pb
/build/<path to output>/b.pb.c
/build/<path to output>/b.pb.h

But the a.pb.h file has an #include "subDir/b.pb.h".
This fails to compile because the header can't be found because there is no
subDir sub directory.
Expected behavior
The expected behavior would be that the generated output is as follows:
/build/<path to output>/a.pb
/build/<path to output>/a.pb.c
/build/<path to output>/a.pb.h
/build/<path to output>/subDir/b.pb
/build/<path to output>/subDir/b.pb.c
/build/<path to output>/subDir/b.pb.h

This would be the same behavior as protoc which has the --proto_path parameter
to pass the root path of all the proto files.
Proposed solutions
My proposed solution is to add an extra parameter to the nanopb_generate_cpp
to set a root path or use the cmake CMAKE_CURRENT_LIST_DIR variable.
This is used to keep the same directory structure in the generated output directory.
The user now needs to make sure to include all of those sub-directory's in the
target_include_directories CMake command.
A command could be used to extract all the directory's from the output source/header files
and add them to a variable using something like the following (taken from this):
MACRO(HEADER_DIRECTORIES return_list)
    FILE(GLOB_RECURSE new_list *.h)
    SET(dir_list "")
    FOREACH(file_path ${new_list})
        GET_FILENAME_COMPONENT(dir_path ${file_path} PATH)
        SET(dir_list ${dir_list} ${dir_path})
    ENDFOREACH()
    LIST(REMOVE_DUPLICATES dir_list)
    SET(${return_list} ${dir_list})
ENDMACRO()

If you're open for this change I can submit a PR.

 What you say seems sane and I'm willing to accept a pull request.
@kylemanna, @kfitch : Any thoughts?

If I'm gonna edit this would it be ok too also edit the cmake_simple example to have a subdirectory example?
This would than have an extra proto file in a subdirectory to get the following:
syntax = "proto2";
import "unlucky/unlucky.proto";

message SimpleMessage {
    required int32 lucky_number = 1;
    required UnluckyNumber unlucky_number = 2;
}

Or would it be better to add an extra test to the tests directory that does the above as a test case?

Hmm, currently there isn't much stuff testing the different build systems. But that might not be very easy to do through scons. I guess modifying cmake_simple works for now.

@idlesink I have made an attempt at adding an optional "RELPATH" parameter to NANOPB_GENERATE_CPP in my nanopb branch https://github.com/kfitch/nanopb
The intent is you would do something like
NANOPB_GENERATE_CPP( SRCS HDRS proto/a.proto proto/sub/b.proto RELPATH proto)
Could you give it a try and give some feedback?

After trying it out in my project I was happy to see that it was working as expected.
What you could do, and I recommend this, is to add this feature to the tests and/or an example and make sure to update the documentation at the top of the "FindNanopb.cmake" file.

I added some docs and an example in my branch. I will go ahead and submit a pull request.

After updating my own project with the new merge and testing it again, everything seems to be working fine.
I think that we can close this issue for it has been fixed.


Update Localizable.strings (Italian)

Added the Italian translation of the new string "Enable Scroll Reverser"

Merged in cd4d297, thanks!


System Beeps when Copy function not available

System beeps when copy function is not available.
System beeps when moving/re-positioning windows on the screen.
MacOs Sierra 12.1

I would use the program if it would not do this, but I can not find the line of code that creates the beep.


Add lg_netcast turn_on_action configuration variable

Proposed change

Allow to turn on the TV using an external service
Type of change

[ ] Dependency upgrade
[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New integration (thank you!)
[x] New feature (which adds functionality to an existing integration)
[ ] Breaking change (fix/feature causing existing functionality to break)
[ ] Code quality improvements to existing code or addition of tests

Example entry for configuration.yaml:

  - platform: lg_netcast
    name: LGigi
    host: <IP_ADDRESS>
    access_token: 123456
    turn_on_action:
      service: switch.turn_on
      data:
        entity_id: switch.lgigi

Additional information

This PR fixes or closes issue: fixes #
This PR is related to issue:
Link to documentation pull request: https://github.com/home-assistant/home-assistant.io/pull/12075

Checklist

[x] The code change is tested and works locally.
[x] Local tests pass. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.
[x] I have followed the development checklist
[x] The code has been formatted using Black (black --fast homeassistant tests)
[ ] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

[x] Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] The manifest file has all fields filled out correctly.
Updated and included derived files by running: python3 -m script.hassfest.
[ ] New or updated dependencies have been added to requirements_all.txt.
Updated by running python3 -m script.gen_requirements_all.
[ ] Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

[ ] No score or internal
[ ] 🥈 Silver
[ ] 🥇 Gold
[ ] 🏆 Platinum

@MartinHjelmare I squashed the fixups 🙇

Please don't squash commits after review has started to make it easier for readers to track changes. We'll squash when merging. :+1:

Oh, sorry @MartinHjelmare I always pay attention to this. That's why I left them until you approved this PR. I'll leave them in the future 🙂

I need to check your squashed commit too, so I basically have to go through the PR again and for a big PR it would mean a lot of work.
No worries! Thanks for your contribution and I hope to see more!


feat: added support to set the rotation limits

We wanted to have the ability to rotate an image with no limits. It was hardcoded to 45 degrees. Now the default is 45 degrees but users of this package can override it and set there own limits or no limits.

made DialConfig its own file again...

Thank you!


Q: Support for v4.8+

Is someone working on bringing support for ember-data 4.8+?

I'm also curious, tried investigate by our self but internal changes in ember-data killed me

Based on this blog post, I believe efforts are going to target ember-data 4.12.

@dwickern and @runspired probably have the most insight into this

ember-data 3.28 - 4.6 uses RecordData v1 which is implemented by model-fragments v6
ember-data 4.7 - 4.11 uses RecordData v2 (RFC 461) which is not currently supported by model-fragments
ember-data 4.12 - 5+ has built-in support for fragments (?)
In https://github.com/adopted-ember-addons/ember-data-model-fragments/pull/417#issuecomment-1352220446 Chris suggested we end support for this addon at ember-data 4.6 and have users upgrade straight to 4.12. @runspired is this still what you would recommend? How would a migration look?

TL;DR The path ahead is as follows. I'll be working with @rsg and anyone else interested to achieve this.
The Promised Land: 4.12+
Our main goal should be to make ModelFragments compatible with 4.12. The release with 4.12 support will be ModelFragment's last, though obviously patches as necessary as long as some folks are on it.
Users of ModelFragments would then upgrade to 4.12, install (at least) @ember-data/schema and @ember-data/schema-model which are the two packages we are guaranteeing the 5.x versions of will support 4.12, and then begin incrementally re-writing their apps to use SchemaModel instead of Model+ModelFragments.
The Road to 4.12
First Stop, Improve 4.6 support
As the Compatibility table points out, currently ModelFragments hopes to support up to 4.6. In practice so far @rsg has noted that at least for Square there seem to be blockers to getting beyond 4.0. So for starters we'll want to figure out what those are and ensure they and anyone else that hasn't quite been able to get to 4.6 gets to 4.6.
Second Stop, Choose Our Path
Option 1 - Write A Custom Cache for 4.7+
In this approach the bulk of the application's refactoring responsibility occurs in 4.12.
In this path, we focus on keeping the public DX and APIs that ModelFragments has today, but change the underlying implementation to conform to expected cache behaviors.
For non-ModelFragments users, 4.11 was a pretty solid landing version before more substantive changes to the internals in 4.12 introduced a number of dragons.
The V2 RecordData version in 4.7-4.11 is very similar to the final version that shipped in 4.12. If we focus on building a custom singleton RecordData to support 4.7-4.11, the leap to supporting 4.12 is likely such that we can handle 4.7-4.12 in one go, but even if we can't easily we should be close enough that it won't be a big ModelFragments rewrite to get to 4.12
But this is also dicey, as this means one big-bang rewrite for ModelFragments. And that rewrite will inevitably come with its own unexpected, unintentional but observable and frustrating deviations in behavior. And to be clear, I do mean a full rewrite, because ultimately the way ModelFragments reasons about and uses the cache and transforms today is incorrect and is what drives the private API usage.
Option 2 - Write A Custom Model for 4.6+
In this approach the bulk of the application's refactoring responsibility occurs in 4.6
We keep EmberData's cache (likely zero changes needed) and instead focus on changing the ModelFragments DX and APIs. We would effectively be attempting to adopt EmberData's SchemaModel ~6 versions early. I think it is possible; though it would not be an exact match due to some limitations of the cache in versions before 4.12.
The migration path for this then involves users getting to 4.6 + Today's ModelFragments, and then incrementing ModelFragments to the major that supports 4.6+ and incrementally rewriting from ModelFragments to the "FragmentSchemaModel", and finally bumping to 4.12 and replacing model-fragments with just EmberData.
I believe in this approach the risks are likely similar, but that if we are careful we can keep SchemaModel and FragmentSchemaModel close enough aligned that the 4.12 swap is relatively painless. The downside is that users won't gain access to any other new features of EmberData (Request, Collection, Pagination primitives, builders) until after a potentially long migration spent on 4.6, and EmberData's patterns will change substantially during 5.x pushing towards newer DX that 4.x supports today.
Parting Thoughts
I leave it to the community to decide which of these paths feels best. I have no apps which use ModelFragments and thus cannot accurately judge the right path before you.
I'll be talking quite a bit in depth about the DX and Feature set of SchemaModel at EmberConf this year. Likely that talk will serve as a good grounding point to reflect on which approach feels more correct.

and then begin incrementally re-writing their apps to use SchemaModel instead of Model+ModelFragments.

I don't know if I understood correctly, but I assume that v4.12 is the last version of Ember Data this library is going to support because a similar behavior is available to develop in ED v5.x+. @runspired Is it possible to have docs of how to migrate from ModelFragments to SchemaModel?

@Gorzas the migration path hasn't been built as its something we'll discover together as we build SchemaRecord / figure out the nice path for folks here. This said, 4.12 is a key part of that path. I recently added a wg-model-fragments channel to the ember discord and posted more about our 4.12 support on the blog


Q: Automatic application of non-zero byte

Using latest version (20230401)
========== Check non-zero byte of the sector ==========
	There is not non-zero byte in the (Track AA)
	There is non-zero byte in the (Track 01)(-LBA): -46 byte => -12 sample
	Last non-zero byte before (Track AA): -49 byte => -12 sample

Is there an option to automatically apply the offset obtained here and dump it?
Or do I need to dump again using the /a option? (In this case /a -12)

Or do I need to dump again using the /a option? (In this case /a -12)

Yes if you want to dump using "user specified offset".

Okay, thanks for the reply.


FEATURE: Google Font from ShadTheme must also be feed to the Material Theme

When we are not using the Material Theme by using these ShadApp or ShadApp.router, and we are configuring the ShadTheme in instance inserting the Google Font in ShadTheme, so it must be automatic feed to the Material Theme.

I already solved it in the following issue https://github.com/nank1ro/flutter-shadcn-ui/issues/87
Are you using the latest version of shadcn_ui? (v0.6.3)

I already solved it in the following issue #87 Are you using the latest version of shadcn_ui? (v0.6.3)

Ohh I see I will check


[WL-975] Fix "Max" button bug

@devtronghieu LGTM, I approved the PR, but we need to wait for the UX content before merging this PR


Use full URLs in docs/data2.json

Currently the processed URLs point to https://toni-heittola.github.io/js-datatable/www.google.com. See the "Linking" section under https://toni-heittola.github.io/js-datatable/#value-formatters.
This PR changes the "www.google.com" URLs to "https://www.google.com" which fixes the issue.

Thanks!


Ninject kernel is not disposed on application stop

There needs to be a way to dispose the kernel when the application stops in topshelf. This allows all of the objects created by the kernel to be disposed and cleaned up.
Perhaps create a wrapper round the service bound in ConstructUsingNinject and then NinjectBuilderConfigurator.Kernel.Dispose(); when the wrapper stops?

This is an open issue since 1 Jun 2017. Why no one looks into this? @lukemcgregor Did you manage to solve this problem?


PWSTATS setup (adapting config files)

Moved from jasonacox/pypowerwall  repo.
...  I have 3 pricing periods for the whole year (no season usage)
09.00 -> 15.00 : Super Off peak $ 0.08
15.00-> 21.00 : Peak $ 0.52
21.00-> 09.00 : Mid Peak $ 0.20
I'm not getting paid any In feed tariff ($ 0.00 ).
In the example_usage.jason you defined the 4 periods:
"22:00": "Off-Peak",
"08:00": "Peak",
"11:00": "Off-Peak",
"17:00": "Peak"
Can I just rename the first 3 periods to suit my needs and do something like this:
"09:00": "Super Off-Peak
"15:00": "Peak"
"21:00": "Mid-Peak"
and what to do with the existing 4th defined period? Does anything also needs to be changed in other file(s) to follow this?
I can see that the other parts of this file needs to be altered as well but let's start from the basics.
Also, what is the meaning (and importance ) of:
"week_anchor": "MONTH",
"year_anchor": "JUN"
?
Thank you.
PS. thanks for the comment and your time in the other thread. I closed it as suggested.

Hi,
I preppared the modified example_usage.jason to suit my needs. Can someone with the experience verify if it does make sense please.
Also, what is the purpose and meaning of ""calendar": {  "2024-11-19": ..." ?
{
"settings": {
"influx_url": "http://localhost:8086",
"bucket": "powerwall/kwh",
"timezone": "Australia/Perth",
"supply_priority": [
"GRID_SUPPLY",
"PW_SUPPLY",
"SOLAR_SUPPLY"
],
"cost_unit": "$",
"energy_unit": "kWh",
"rename": {
"GRID_SUPPLY": "Grid supply---",
"GRID_EXPORT": "Grid export+++"
},
"resample": true,
"week_anchor": "MONTH",
"year_anchor": "JANUARY"
},
"plans": [
{
"name": "Synergy_Midday-Saver",
"report": [
"GRID_SUPPLY",
"GRID_EXPORT",
"SELF_PW_NET_OF_GRID",
"SELF_SOLAR_PLUS_RES"
],
"agent": "Simple",
"seasons": {
"All_Year": [
{
"schedule": "Week",
"days": [
0,
1,
2,
3,
4,
5,
6
],
"periods": {
"09:00": "Off-Peak",
"15:00": "Peak",
"21:00": "Mid-Peak"
}
}
]
}
}
],
"calendar": {
"2024-11-19": {
"plan": "Synergy_Midday-Saver",
"season": "All_Year",
"tariffs": {
"Peak": {
"GRID_SUPPLY": -0.525313,
"GRID_EXPORT": 0.00,
"SELF_PW_NET_OF_GRID": 0.525313,
"SELF_SOLAR_PLUS_RES": 0.525313,
"SUPPLY_CHARGE": -1.26075
},
"Mid-Peak": {
"GRID_SUPPLY": -0.231138,
"GRID_EXPORT": 0.00,
"SELF_PW_NET_OF_GRID": 0.231138,
"SELF_SOLAR_PLUS_RES": 0.231138,
"SUPPLY_CHARGE": -1.26075
},
"Off-Peak": {
"GRID_SUPPLY": -0.08405,
"GRID_EXPORT": 0.00,
"SELF_PW_NET_OF_GRID": 0.08405,
"SELF_SOLAR_PLUS_RES": 0.08405,
"SUPPLY_CHARGE": -1.26075
}
}
}
}
}
Thank you.

and... we got first problem (I think):
michal@NUC:~/Powerwall-Dashboard/tools/usage-service$ bash test_service.sh
Warning for developers: this script will delete any existing pwdusage container.
However, is does not delete the pwdusage docker image.
For general end users, this is what you want to happen for trouble shooting.
Enter y/Y to run test> y
Error response from daemon: No such container: pwdusage
Error response from daemon: No such container: pwdusage
3f1091c46479d7989058d08e8453bccdc357fa4c42f9951bc298e154db82fefc
Waiting. Please hit enter to clean up test container.>
pwdusage
pwdusage
the  previous bash build.sh went as follow:
michal@NUC:~/Powerwall-Dashboard/tools/usage-service$ bash build.sh
This script cleans up old versions of the pwdusage docker image,
and builds a new image using the latest version pwdusage from pypi.
Normal end users should enter 'y' when asked to clean up.
Developers should probably avoid this script - it's a bit terminal.
Clean up old versions of pwusage before building? (Y/n)>y
Error response from daemon: No such container: pwdusage
Error response from daemon: No such container: pwdusage
Error response from daemon: No such image: jasonacox/pwdusage:latest
[+] Building 35.1s (8/8) FINISHED                                                                    docker:desktop-linux
=> [internal] load build definition from Dockerfile                                                                 0.1s
=> => transferring dockerfile: 411B                                                                                0.0s
=> [internal] load metadata for docker.io/library/python:3.11-slim                                                 10.6s
=> [auth] library/python:pull token for registry-1.docker.io                                                        0.0s
=> [internal] load .dockerignore                                                                                0.1s
=> => transferring context: 2B                                                                                0.0s
=> [1/3] FROM docker.io/library/python:3.11-slim@sha256:370c586a6ffc8c619e6d652f81c094b34b14b8f2fb9251f092de23f16e  4.8s
=> => resolve docker.io/library/python:3.11-slim@sha256:370c586a6ffc8c619e6d652f81c094b34b14b8f2fb9251f092de23f16e  0.0s
=> => sha256:5f20101c4b63a3742b4155dcffc296c8ee0e2c7b83dbe6c7687441d991f358d4 250B / 250B                           0.7s
=> => sha256:51e2bd9c4b085ec5483ea37c7aff2b4fa3e49a780ff4db8f971fa68c9539db9c 16.20MB / 16.20MB                     3.4s
=> => sha256:3a75594a45d1452af40f08d7e4d4fa96572c3a09fb46ae142cf45e98973b3c69 3.32MB / 3.32MB                       2.4s
=> => sha256:bc0965b23a04fe7f2d9fb20f597008fcf89891de1c705ffc1c80483a1f098e4f 28.23MB / 28.23MB                     3.3s
=> => extracting sha256:bc0965b23a04fe7f2d9fb20f597008fcf89891de1c705ffc1c80483a1f098e4f                            0.6s
=> => extracting sha256:3a75594a45d1452af40f08d7e4d4fa96572c3a09fb46ae142cf45e98973b3c69                            0.1s
=> => extracting sha256:51e2bd9c4b085ec5483ea37c7aff2b4fa3e49a780ff4db8f971fa68c9539db9c                            0.4s
=> => extracting sha256:5f20101c4b63a3742b4155dcffc296c8ee0e2c7b83dbe6c7687441d991f358d4                            0.0s
=> [2/3] WORKDIR /app                                                                                0.2s
=> [3/3] RUN pip3 install --no-cache-dir pwdusage                                                                  12.1s
=> exporting to image                                                                                6.9s
=> => exporting layers                                                                                5.3s
=> => exporting manifest sha256:9691331f944d4828d5a4579f089b754b64b425c5fb99d4a5009bc8c9eb1c1697                    0.0s
=> => exporting config sha256:c34ace2ef66f89042dce59c834094103057f23412b73e1cf675bf4b409a969da                      0.0s
=> => exporting attestation manifest sha256:9d6e71268e0dae58823cd4bfcd760d2a8fbb55b2e9a35a8a05e19a77666ead84        0.1s
=> => exporting manifest list sha256:50211461f4ab4c107c8b0d303b16b544ad6d4d7d7a572f2b6ca20d874dbdba5c               0.0s
=> => naming to docker.io/jasonacox/pwdusage:latest                                                                 0.0s
=> => unpacking to docker.io/jasonacox/pwdusage:latest                                                              1.4s
View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/fpf76yif0ogj3qojcntict89d
Thanks.

When running the bash test_service.sh, I'm getting:
2024-12-25 10:26:09 ERROR: Error loading usage engine configuration file.
2024-12-25 10:26:09 ERROR: Details: <urllib3.connection.HTTPConnection object at 0x7fa6e5bb6510>: Failed to establish a new connection: [Errno 111] Connection refused
?
Thanks

Also, not sure what it is but influxdb does not specify an user for that querry, is it the part of the problem?
~/Powerwall-Dashboard/tools/usage-service$ docker inspect $(docker ps -q) --format '{{.Config.User}} {{.Name}}'
/pwdusage
1000:1001 /telegraf
1000:1001 /weather411
1000:1001 /pypowerwall
/influxdb
1000:1001 /grafana

Please post your pwdusage.extended.yml and usage.json files. I'll check when I have time.

Hi, thanks for  answering during this time of the year !
pwdusage.extended..yml:

version: "3.5"

services:

pwdusage:

    image: jasonacox/pwdusage:latest

    container_name: pwdusage

    hostname: pwdusage

    restart: always

    user: "1000:1001"

    volumes:

        - type: bind

          source: ./tools/usage-service

          target: /var/lib/pwdusage

          read_only: true

    ports:

        - target: 9050

          published: 9050

          mode: host

    environment:

        - USAGE_JSON=/var/lib/pwdusage/usage.json

    depends_on:

        - influxdb

usage.jason

{

"settings": {

    "influx_url": "http://localhost:8086",

    "bucket": "powerwall/kwh",

    "timezone": "Australia/Perth",

    "supply_priority": [

        "GRID_SUPPLY",

        "PW_SUPPLY",

        "SOLAR_SUPPLY"

    ],

    "cost_unit": "$",

    "energy_unit": "kWh",

    "rename": {

        "GRID_SUPPLY": "Grid supply---",

        "GRID_EXPORT": "Grid export+++"

    },

    "resample": true,

    "week_anchor": "MONTH",

    "year_anchor": "JUN"

},

"plans": [

    {

        "name": "Aurora-TAS-ToU",

        "report": [

            "GRID_SUPPLY",

            "GRID_EXPORT",

            "SELF_PW_NET_OF_GRID",

            "SELF_SOLAR_PLUS_RES"

        ],

        "agent": "Simple",

        "seasons": {

            "Summer": [

                {

                    "schedule": "Weekday",

                    "days": [

                        0,

                        1,

                        2,

                        3,

                        4

                    ],

                    "periods": {

                        "22:00": "Off-Peak",

                        "08:00": "Peak",

                        "11:00": "Off-Peak",

                        "17:00": "Peak"

                    }

                },

                {

                    "schedule": "Weekend",

                    "days": [

                        5,

                        6

                    ],

                    "periods": {

                        "00:00": "Off-Peak"

                    }

                }

            ],

            "Winter": [

                {

                    "schedule": "Weekday",

                    "periods": {

                        "21:00": "Off-Peak",

                        "07:00": "Peak",

                        "10:00": "Off-Peak",

                        "16:00": "Peak"

                    }

                }

            ]

        }

    }

],

"calendar": {

    "2022-10-06": {

        "plan": "Aurora-TAS-ToU",

        "season": "Summer",

        "tariffs": {

            "Peak": {

                "GRID_SUPPLY": -0.33399,

                "GRID_EXPORT": 0.08883,

                "SELF_PW_NET_OF_GRID": 0.33399,

                "SELF_SOLAR_PLUS_RES": 0.33399,

                "SUPPLY_CHARGE": -0.04579291666

            },

            "Off-Peak": {

                "GRID_SUPPLY": -0.15551,

                "GRID_EXPORT": 0.08883,

                "SELF_PW_NET_OF_GRID": 0.15551,

                "SELF_SOLAR_PLUS_RES": 0.15551,

                "SUPPLY_CHARGE": -0.04579291666

            }

        }

    },

    "2023-04-02": {

        "season": "Winter"

    }

}

}

Thanks.

Take a quick look at issue #10 - the same error came up and the debug steps there may sort you out.

Also, if the ip address works correctly, try replacing it with the name of the machine hosting docker - e.g. in my case orac.local (this is going to depend on your LAN/Wireless DNS working though).

Hi,
still, no success
2024-12-26 19:54:58 INFO: Powerwall-Dashboard usage proxy server [1.0.0] - HTTP Port 9050
2024-12-26 19:54:58 INFO: Powerwall-Dashboard usage engine proxy Started
2024-12-26 19:56:42 ERROR: Error loading usage engine configuration file.
2024-12-26 19:56:42 ERROR: Details: <urllib3.connection.HTTPConnection object at 0x7fc241e36e10>: Failed to establish a new connection: [Errno 111] Connection refused

so what about that ERROR: Error loading usage engine configuration file ?

the docker internal addresses are:
influxdb: 172.18.06
telegraf: <IP_ADDRESS>
grafana: 172.18.04
I can play with them, but I need to know where to use it.
Thanks

now, if I put "influx_url": "http://<IP_ADDRESS>:8086" in usage.jason, I get:
"599 Error loading usage engine configuration file ((<urllib3.connection.HTTPConnection object at 0x7f69dcccd6d0>, 'Connection to <IP_ADDRESS> timed out. (connect timeout=10.0)'))."
So no more "refused" but "timed out"
thanks

Those are the files permissions in the /var/lib/pwdusage, that looks ok I hope.

and the influxdb is listening on 8086...
2024-12-26 23:43:40 ts=2024-12-26T15:43:40.962854Z lvl=info msg="Listening on HTTP" log_id=0thv2XDW000 service=httpd addr=[::]:8086 https=false
I restarted the influxdb then pwdusage and finally tried to load the http, no progress. I'm running out of ideas, there is not even a trace of connecting attempts in the influxdb logs. Time for bed.
Thanks

Michal, you need to set the influxdb url to the IP address of the docker host, not the internal docker IP i.e. http:///:8086.

Hi,
new day, new challanges :)
so, I put in usage.jason: "influx_url": "http://nuc:8086"
and in the local browser: http://<IP_ADDRESS>:9050/usage_engine
that resulted in: Usage Engine Status	"Engine OK, tariffs (re)loaded"
Time to follow the rest of the steps... :)
thanks

I changed it to "influx_url": "http://<IP_ADDRESS>:8086/" and it works again. Hmmm....
Usage Engine Status	"Engine OK, tariffs (re)loaded"

OK, after finishing the test, I did:
./compose-dash.sh stop
./compose-dash.sh up -d
which went trough with no problems. I noticed thou, that the jasonacox/pwdusage (latest) is not running.
Is that how it should be?
Thanks

I changed it to "influx_url": "http://<IP_ADDRESS>:8086/" and it works again. Hmmm....

So your problem with the nuc url is a problem with name resolution on your local network. Which I can't help with - you'll need to check your router docs and do some more general reading on this. However, if your nuc has a static ip, then there is no problem with just using that (and you know it works).

I noticed thou, that the jasonacox/pwdusage (latest) is not running.

Sorry - I gave incomplete instructions - check this for setting up the .extend file. https://github.com/BuongiornoTexas/pwdusage?tab=readme-ov-file#add-pwdusage-to-powerwall-dashboard

Thanks for that.
Actually, when I used "nuc", it was the first tiime I got:
Usage Engine Status "Engine OK, tariffs (re)loaded".
It stopped working thou after subsequent page reloads. Go figure!
I can use "nuc" for enything else so I don't understand what is happening. There is a discussion at https://github.com/influxdata/influxdb/issues/15321 (if you interested) about influx troubles itself.
Going to read the link you posted now!
Thanks

Update:
the pwdusage is running now in docker
I was able to add data source in grafana (2 green info squares shown up on saving)
I succesfully tested http://<IP_ADDRESS>:9050/usage_engine,  Usage Engine Status	"Engine OK, tariffs (re)loaded"
Now I will try to get the data to actually display in the Grafana browser screen.
Thanks.

Hi,
just probably one querry before I close that thread. I did some changes in the use.jason to adopt to my Synergy plan.
Can you please have a glimpse at it and let me know if it does have a chance to work. That file is currently in use but I can't see if it is ok before I set up new a dashboard in Grafana (which is the new, uncharted territory for me!).
Anyway, I got 3, daily price brackets, no seazons, no weekends, zero export $, 24/7. And that is what I did:
{
"settings": {
"influx_url": "http://<IP_ADDRESS>:8086",
"bucket": "powerwall/kwh",
"timezone": "Australia/Perth",
"supply_priority": [
"GRID_SUPPLY",
"PW_SUPPLY",
"SOLAR_SUPPLY"
],
"cost_unit": "$",
"energy_unit": "kWh",
"rename": {
"GRID_SUPPLY": "Grid supply---",
"GRID_EXPORT": "Grid export+++"
},
"resample": true,
"week_anchor": "MONTH",
"year_anchor": "JAN"
},
"plans": [
{
"name": "Synergy_Midday-Saver",
"report": [
"GRID_SUPPLY",
"GRID_EXPORT",
"SELF_PW_NET_OF_GRID",
"SELF_SOLAR_PLUS_RES"
],
"agent": "Simple",
"seasons": {
"All_Year": [
{
"schedule": "Week",
"days": [
0,
1,
2,
3,
4,
5,
6
],
"periods": {
"09:00": "Off-Peak",
"15:00": "Peak",
"21:00": "Mid-Peak"
}
}
]
}
}
],
"calendar": {
"2024-11-19": {
"plan": "Synergy_Midday-Saver",
"season": "All_Year",
"tariffs": {
"Peak": {
"GRID_SUPPLY": -0.525313,
"GRID_EXPORT": 0.00,
"SELF_PW_NET_OF_GRID": 0.525313,
"SELF_SOLAR_PLUS_RES": 0.525313,
"SUPPLY_CHARGE": -1.26075
},
"Mid-Peak": {
"GRID_SUPPLY": -0.231138,
"GRID_EXPORT": 0.00,
"SELF_PW_NET_OF_GRID": 0.231138,
"SELF_SOLAR_PLUS_RES": 0.231138,
"SUPPLY_CHARGE": -1.26075
},
"Off-Peak": {
"GRID_SUPPLY": -0.08405,
"GRID_EXPORT": 0.00,
"SELF_PW_NET_OF_GRID": 0.08405,
"SELF_SOLAR_PLUS_RES": 0.08405,
"SUPPLY_CHARGE": -1.26075
}
}
}
}
}
It would be great if you can comment. If all good I will close this thread and probably open one for the implementation in Grafana. You gave pretty good decription but I'm missing the very basic stuff.
Thanks

I just find out from the other closed thread that the supply charge is based on the hourly calculation so I will need to correct this.
Thanks.

So I got everything running in the end.
Thanks for the excellent utility and your support 😀

Excellent - I took a quick look at your json file, and it looks correct.


Weakness/Resistances being color-codified in the wrong order

The image exemplifies the issue quite clearly:
The resistances and weaknesses are calculated correctly in the order they are input in the actor's sheet, but are applied as if they follow the same order. In reality, the weaknesses and resistances are alphabetized on the actor's sheet.

In this example, the weaknesses were added in ascending order from 1 to 11.

Nice catch!  Fixed in v1.4.5


chore(deps): update renovate/renovate docker tag to v35.15.0

This PR contains the following updates:

Package
Update
Change

renovate/renovate
minor
35.14.1 -> 35.15.0

Release Notes

renovatebot/renovate
v35.15.0
Compare Source
Features

customize merge confidence host (#21061) (623f2ca)

Code Refactoring

merge confidence default undefined (#21059) (3b60c93)
MergeConfidence types (#21058) (25e0fa2)

Miscellaneous Chores

deps: update dependency @types/node to v18.15.3 (#21060) (1b88fe4)

v35.14.4
Compare Source
Miscellaneous Chores

deps: update dependency @types/node to v18.15.2 (#21053) (12e2653)
deps: update linters (#21054) (5ba02a0)
deps: update node.js to 3c89fc4 (#21055) (f399efb)

Build System

deps: update opentelemetry-js monorepo (#21050) (aebf72d)

v35.14.3
Compare Source
Bug Fixes

data: automatic update of static data (#21030) (52c2ce0)

Miscellaneous Chores

deps: update actions/dependency-review-action action to v3.0.4 (#21043) (80ccbb0)
deps: update dependency @types/node to v18.15.1 (#21042) (3514ba5)

v35.14.2
Compare Source
Bug Fixes

asdf-pulumi: Extract version (#21038) (fba9013)

Documentation

update references to renovate/renovate to v35.14.1 (#21040) (5f32cdb)

Miscellaneous Chores

deps: lock file maintenance (#21041) (2f5a4d2)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 4.4.5 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


add_n operator with MXNet-MKL producing wrong results when input count >4

Problem:
With mxnet-mkl (1.4.0)
If number of input symbols > 4 and I perform add_n after a FC layer produces wrong results.
i.e.,
data_0 -> fc_0  \
data_1 -> fc_1   \
data_2 -> fc_2      => add_n
data_3 -> fc_3  /
data_4 -> fc_4 /

Minimum reproducible code below:
Run below code which is full network:
import mxnet as mx

num_inp_symbols = 5
data_shape = (5,5)
hidden_layer_size = 8

input_symbols = [mx.sym.var('data_'+str(i)) for i in range(num_inp_symbols)]
fully_connected_symbols = [mx.sym.FullyConnected(data=input_symbols[i],
                                                                                num_hidden=hidden_layer_size,
                                                                                name='fc_'+str(i))
                                                for i in range(num_datasets)]

#Create final symbol
net = mx.sym.add_n(*fully_connected_symbols)
#Validate topology
#mx.viz.plot_network(net)

mod = mx.mod.Module(symbol=net, data_names=['data_0', 'data_1', 'data_2', 'data_3', 'data_4'], label_names=None)
mod.bind(for_training=False, data_shapes=[('data_0', data_shape), ('data_1', data_shape), ('data_2', data_shape), ('data_3', data_shape), ('data_4', data_shape)])
mod.set_params(full_module.get_params()[0], full_module.get_params()[1])

mod.forward(mx.io.DataBatch([mx.nd.ones(data_shape), mx.nd.ones(data_shape), mx.nd.ones(data_shape), mx.nd.ones(data_shape), mx.nd.ones(data_shape)]))
print(mod.get_outputs()[0])

Output
[[ 2.2989948  -3.3271918   0.64880913  2.2778904   0.9859241   2.0046096
  -1.6065626   1.5986269 ]
 [ 2.2989948  -3.3271918   0.64880913  2.2778904   0.9859241   2.0046096
  -1.6065626   1.5986269 ]
 [ 2.2989948  -3.3271918   0.64880913  2.2778904   0.9859241   2.0046096
  -1.6065626   1.5986269 ]
 [ 2.2989948  -3.3271918   0.64880913  2.2778904   0.9859241   2.0046096
  -1.6065626   1.5986269 ]
 [ 2.2989948  -3.3271918   0.64880913  2.2778904   0.9859241   2.0046096
  -1.6065626   1.5986269 ]]
<NDArray 5x8 @cpu(0)>

However, Let us now compute output of each FC in above network (fc0_output, fc1_output,... fc4_output). What I observe is the if I do individual fc output calculation and sum it up it is not same result as running everything together.
constituent_fc0 = fully_connected_symbols[0]
print(constituent_fc0.get_internals().list_outputs())

mod_cons_fc0 = mx.mod.Module(symbol=constituent_fc0, data_names=['data_0'], label_names=None)
mod_cons_fc0.bind(for_training=False, data_shapes=[('data_0', data_shape)])
mod_cons_fc0.set_params(mod.get_params()[0], mod.get_params()[1])
mod_cons_fc0.forward(mx.io.DataBatch([mx.nd.ones(data_shape)]))
o1 = mod_cons_fc0.get_outputs()[0]

#and so on for fc1, fc2, fc3, fc4
#and then do
print(nd.add_n(o1, o2, o3, o4, o5))

@ZhennanQin @pengzhao-intel - Can you please help debug this issue?
Please Note:

storage type is all dense
Number of inputs > 4
Happens only from Module APIs and from mxnet-mkl 1.3.0 version onwards.

we are on the public holiday this week :)
@rongzha1 please help take a look for this issue after the holiday.

add_n output mem overlap with input mem (due to FInplaceOption), but in Forward function : ElementwiseSumContainsDnsImpl( ),  output mem was set_zero which makes input zero:     Kernel<set_zero, cpu>::Launch(s, out_data.Size(), out_data.dptr());
In fact, this is not a MKLDNN Bug.
@pengzhao-intel @zheng-da @TaoLv

@rongzha1 thanks for the analysis. Would you mind to file a PR and fix this issue?

btw, please add the example as a test case.

Good catch and fixed now :)


In Handle.py, recommend moving len() function to the SimBaseHandle class

Currently, only "NonConstantObject" handles support the len() function.  This means in tests have to be careful of the handle type before invoking the len(handle) function.  This function will have the same implementation for all handle types so if we just move to the SimBaseHandle class, it will at least not throw an exception if someone calls len() on a handle.  The GPI interface already supports this for all handles and defaults returning 0, so it will not crash even if the capability for particular handle types is not supported by the different interfaces.

@leftink makes sense

Fixed in latest.  Closing.


refactored a lot

I think you don't need pointers to channels and a global srv. That's really begging for nil pointer dereference.
I don't think that you need the private server type and a exported one. You mainly had the same api in lrserver.go and server.go just one exported that wrapped the other.
you can startup the lrserver http server in NewLRServer() and still get an error if Listen() fails.
I rewrote the tests in GoConvey. The do the same testing (setup > handshake > shutdown) but with less code

It might be a bit much for one PR but I can reformat it if you want.

@cryptix
Thanks again for your good advice, I finally found some time to write v2 incorporating your suggestions.
If you have a chance, check out the 2.0.0 branch and feel free to submit any issues or pull requests before I merge it. Not everything is in the README, so you'll have to generate the docs or look through the code to see all the new features.
Also, this version will break the current API so keep an eye out if you're currently using this package.


Swagger autogen is not recognizing the path name which is defined in another file

I have a page with path names defined that we are using in the router.get( pathname from fille, methodname from controller). But in swagger.json file,  its coming as empty.
If we change pathname in the router.get as router.get('/s', controllermethod name) then it working fine.
Also, status code mentioned in the controller method is not getting read by swagger autogen.
requestbody also is empty

Hi @RajalakshmiRamatchandirane. Thank you for opening the issue. Sorry for the delay. I couldn't reproduce this problem here. Could you send me a sample project that reproduces this problem, please?
That way is easier to identify and handle the problem.

Hi, did you find a solution to this in the end?

Hi @Detan-L. Not yet. Any news about it, I'll post it here.


Introduce joint torque regressor

Summary:
this PR introduces a joint torque regressor, such that

where

and  has its origin at the barycenter.
Also, it introduces several methods which, other than being employed by the function, might be useful per se, and a couple of minor additions and modifications.
This may be useful for calibration, but also for the computation of partial derivatives w.r.t. the dynamic parameters.
In detail:

introduce methods inertia.toDynamicParameters() and Inertia.FromDynamicParameters(Vector10) to convert between a Pinocchio Inertia and its corresponding
introduce algorithm bodyRegressor, computing the regressor for the dynamic parameters of a single rigid body, of which the velocity and acceleration are given
introduce algorithm jointBodyRegressor, computing the regressor for the dynamic parameters of a rigid body attached to a given joint, assuming RNEA has been called
introduce algorithm frameBodyRegressor, computing the regressor for the dynamic parameters of a rigid body attached to a given frame, assuming RNEA has been called
introduce algorithm computeJointTorqueRegressor, computing the joint torque regressor described above, which is written in data.jointTorqueRegressor
expose data.staticRegressor in Python
move computeStaticRegressor to main pinocchio namespace. Deprecating the version in regressor namespace. I did this because the existence of this namespace has always seemed incoherent with the style of the rest of Pinocchio, which is having all algorithms in the main Pinocchio namespace

NOTE 1: notice that, as forward step of the algorithm, I am using RneaForwardStep. I did it to go quickly, and also to avoid the need of compiling another step. I now realize calling RneaForwardStep will modify data.f, leaving it in an inconsistent state. This should be fixed. Therefore, I will either create a custom forward step or find another way to fix the problem which does not require creating a new step type (I can think of at least two,both of which would slightly increase execution time).
NOTE 2: because the matrices are small enough, I used fixed-size matrices: Matrix<Scalar,10, 1> and Matrix<Scalar,6,10>. As for Matrix<Scalar,10, 1>, I typedefed it to Vector10 inside InertiaTpl. Matrix<Scalar,6,10>, I do not know where to put a convenient typedef. Also, neither of them is exposed in eigenpy. Should we think about it, especially for Vector10?

@jcarpent thanks for your comments. Concerning the final two notes I left to the original post, do you have suggestions?

@jcarpent thanks for your comments. Concerning the final two notes I left to the original post, do you have suggestions?

I have answered in the post. The rest seems fine to me.

@jcarpent thanks for your comments. Concerning the final two notes I left to the original post, do you have suggestions?

I have answered in the post. The rest seems fine to me.

I think you might have missed what I mean when I mentioned the "final notes". I meant this:

NOTE 1: notice that, as forward step of the algorithm, I am using RneaForwardStep. I did it to go quickly, and also to avoid the need of compiling another step. I now realize calling RneaForwardStep will modify data.f, leaving it in an inconsistent state. This should be fixed. Therefore, I will either create a custom forward step or find another way to fix the problem which does not require creating a new step type (I can think of at least two,both of which would slightly increase execution time).

Do you agree it might be a concern? I think I will need to make a custom forward step for it, which is identical to RNEA except that it does not touch data.f

Do you agree it might be a concern? I think I will need to make a custom forward step for it, which is identical to RNEA except that it does not touch data.f

Yes, I think this is not a problem. I will prefer that to have minimum number of operations, and self contained algorithms

Do you agree it might be a concern? I think I will need to make a custom forward step for it, which is identical to RNEA except that it does not touch data.f

Yes, I think this is not a problem. I will prefer that to have minimum number of operations, and self contained algorithms

Fixed

@gabrielebndn Do you want and have time to address the last comments have made?
Otherwise, we can keep it as it is, but it won't be the best we can perform, which is not an issue in its own.

@gabrielebndn Do you want and have time to address the last comments have made?
Otherwise, we can keep it as it is, but it won't be the best we can perform, which is not an issue in its own.

Sorry no time, I'll be away the whole weekend and I really have to run

It is fine for me. I will wait travis to merge.

@gabrielebndn Thanks a lot again for this very nice work!


Add colourized output.

Now if tests fail that are coloured as red and passing tests are green.
Tests that pass marginally, by roughly 20% will be marked as yellow.
This percentage is higher the lower the threshold, and lower the higher
the threshold.

An example is included below
before

after

:+1: was just looking for this since coverage's are already coloured by default.


Make log file name configurable

When I run a robot-service in Cloud Run in Google Cloud, I want to monitor the logs. I therefore would like to configure the log file name and location, in order to monitor it with Cloud Logging.

Not sure, if that would even work. But there is ROBOT_SYSLOG_FILE environment variable that does seem to do the job.


Corrections to example GUI

Here is the correction to example_GUI.py that lets it run without the 'bases' directory, as discussed in #275. Moreover, now it catches all exceptions occurring in Transform() and shows them in the GUI messages pane, so that if something wrong happens, the user will surely notice that.
Another problem was that loading "from transform" in fact did not work, since the code assigned the Transform object instead of its transformed image (.transform). This is also corrected (and seems to work).
Please test.

New improvements:

Don't show the confusing warning for the methods that do not need bases directory.
Disabled selecting the forward transform for the methods that don't support it.
Disabled editing of combo boxes, with are really used as drop-down lists.

Hopefully, all this is done correctly. Please check again. :–)

Yeah, this looks good to me. It makes me think that we should define the forward and basis methods for all of PyAbel. But until we find these variables useful somewhere else, let's just stick with what you have here.

Thanks, all!

Nice work!
...uh oh, all of the PRs are merged now. Probably @MikhailRyazanov has another new mega-PR waiting... :)


Fix bug in autocrop with leaveBorder enabled

What's Changing and Why

Centralise application of border for a better overview
Add safety check for crops to be greater than or equal to 0
Currently the crop size is always reduced by leaveBorder value, which can lead to negative values if the crop size is smaller than the border.

What else might be affected
nothing
Tasks

[x] Add tests
[x] Update Documentation => not needed
[x] Update jimp.d.ts => not needed
[x] Add SemVer Label

Looks good! Just remove the only and I'll merge + release

@hipstersmoothie, any progress on this? :)


[US] (i18n) Show RAW Names

As an application user, I want to display throughout the application the RAW i18n default translations, so that I can utilize the internationalization system.

MoSCoW
Risk
Story Points

Should
Medium
5

Acceptance Criteria

Displaying the RAW db names throughout the application via the default language, as per Stakeholder Requirements.

Risks

Not displaying all the raw db names, if the user hasn't filled the default language

Pre-requisites

[ ] Meet with the stakeholder to clarify the requirements
[ ] Complete the requirements of this US

Tasks

[ ] Default language column, that always remains on the Internalization tab.
[ ] Dropdown default language selector, for the user to choose the default language to be used for the application.
[ ] User being able to add a translation for each label, and being able to modify it. Changes reflected to the database.
[ ] Raw names being reflected on the application

UI Mockup

Use Case Diagram

Before starting this US #280 would need to be completed to test the full functionality, hence it will be moved to next iteration


Implement the meta store interface and its impl for its file-backed implementation

Purpose and Challenge
Quickwit needs a way to keep track of the list of splits available for a given index. Currently the metastore's implementation is backed by PostGreSQL.
However, deploying PostgreSQL might be a cumbersome requirement for the user who just wants an index for a single large static dataset in their datalake, or who wants to test out whether Quickwit's search is working as advertised before throwing themselves into deploying multiple services.
For this reason, there will be different implementation of the datastore, the most constrained one, being stored directly on S3.
API Definition
Functional need

Indexer need to publish the splits they have finished building.
Searcher need to list the splits in an index, possibly pruning them based on some metadata.
Mergers need to publish the outcome of a merge, (generally the atomic publication of a split and the deletion of some other).
Retention manager need to be able to remove split based on their timestamps (and possibly some metadata?).

Note that some of these operation imply some interaction with the storage. This is not the role of the meta store, and the correct implementation of its client application is assumed. The meta store semantics however, need to make this coordination possible... (See section below)
File management need, and the split lifecycle
Quickwit needs a way to ensure that we can cleanup unused files, and this process needs to be resilient to any fail-stop failures.
Because our metastore are typically more constrained than a filesystem (e.g.: object storage), we cannot rely in a write ahead log. We do not want to rely on a strategy that would require to list the files in the store as well, because:

we might end up removing some user file non related with quickwit.
listing files is notoriously slow and clumsy on object storage.

Instead, we rely atomically shifting the status of splits.
A split goes into the following lifecycle
Building
   -stage->
Staged
   -publish->
Published
   -markAsDelete->
MarkAsDeleted
   -delete->
Deleted

Building splits and deleted splits are not states that are registered in the metastore, and should have no durable left-over anywhere.
We maintain the following invariant:
If a split has a files in the storage, it MUST be registered in the meta store, and its state can be either

Staged:  the splits is almost ready. Some of its files may have been uploaded in the storage.
Published: the splits is ready and published.
ScheduledForDelete: the split is scheduled to be deleted.

Before creating any file, we need to stage the split. If there is a failure, upon recovery we schedule for delete all of the staged splits.
A client may not necessarily remove file from storage right after marking it as deleted. A CLI client may delete files right away, but a more serious deployment should probably only delete these files after a period of time to give some time to running search queries to finish.
API Definition
Stage splits (W)
/// Stages a splits.
/// A split needs to be staged BEFORE uploading any of its files to the storage.
/// The SplitId is returned for convenienced but it is not generated
/// by the metastore. In fact it was supplied by the client and is present in the split manifest.
fn stage_split(split_manifest: SplitManifest) -> Result<SplitId, StageError>;

enum StageError {
    InvalidManifest(...),
    ExistingSplitId(...),
    InternalError(...),
}

/// A split manifest carries all meta data about a split
/// and its files.
struct SplitManifest {
    /// the index it belongs to or is destined to belong to.
    index_id: String,

    /// The set of information required to open the split,
    /// and possibly allocate the split.
    metadata: SplitMetaData,

    /// The list of files associated to the split.
    files: Vec<ManifestEntry>,
}

struct SplitMetaData {
    /// Split uri. In spirit, this uri should be self sufficient
    /// to identify a split.
    //  In reality, some information may be implicitly configure
    /// in the store uri resolver, such as the S3 region.
    split_uri: String

    /// Number of records (or document) in the split
    num_records: u64,

    /// Weight in bytes of the split
    size_in_bytes: u64,

    /// if a time field is available, the min / max timestamp in the segment.
    time_range: Option<Range<i64>>

    /// Number of merge this segment has been subjected to during its lifetime.
    generation: usize,
}

Publish splits (W)
/// Records a split as published.
///
/// This API is typically used by an indexer who needs to publish a new split.
/// At this point, the split files are assumed to have already uploaded.
/// The metastore only updates the state of the split from staging to published.
///
/// It has two side effetcs:
/// - it makes the split visible to other clients via the ListSplit API.
/// - it guards eventual recovery procedure from removing the split files.
///
/// Unless specified otherwise in the implementation, the metastore DOES NOT
/// do any check on the presence, integrity or validity of the metadata of this split.
///
/// A split successfully published MUST eventually be visible to all clients.
/// Stronger consistency semantics should be documented in the implementation.
///
/// If the split is already published, this API call returns a success.
fn publish_split(split: SplitId) -> Result<(), PublishError>;

enum PublishError {
    DoesNotExist(SplitId),
    SplitIsNotStaged {
        split: SplitId,
        previous_state: State
    },
    InternalError(...)
}

List splits (W)
/// Returns the list of published splits intersecting with the given time_range.
/// Regardless of the timerange filter, if a split has no timestamp it is always returned.
/// Splits are returned in any order.
fn list_splits(index: IndexId, state: State, time_range: Option<TimeRange>) -> Result<Vec<SplitMeta>, ListSplitsError>;

enum ListSplitsError {
IndexDoesNotExist(IndexId),
InternalError(...)
}
List splits is used:

by searcher with state=Published, and often with some timestamp range.
by recovery procedure with state=Staging, and state=ScheduledForDelete
by eventual garbage collection procedure with state=ScheduledForDelete

Delete split (W)
/// Marks a split as deleted.
/// This function is successful if a split was already marked as deleted.
fn mark_as_deleted(split_id: SplitId) -> Result<(), MarkAsDeletedError>;

enum MarkAsDeleteError {
    DoesNotExist(SplitId),
    InternalError(...)
}

///
/// This function only takes split that are in staging or in mark as deleted state.
fn delete_split(split_id: SplitId) -> Result<, DeleteSplitError>;

enum DeleteSplitError {
    DoesNotExist(...),
    Forbidden(...), // deleting a published split is forbidden
    InternalError(...)
}

File-in-object-storage based implementation
We simply serialize all of the splits associated to a single index into one large file.
We do not enforce any lock, and having several concurrent writer can lead to corruption (similarly to several datalake meta store).
The file has the following format.
{
    "index": {
        // Possible index metas
    },
    "splits": {
        <split_id_1>: {
            "metadata": <SplitMetaData>,
            "files": [<manifest_entries>],
        }
    }
}

@fulmicoton on the last part regarding the lock. how do we want to avoid multiple IndexWriter from corrupting the meta-store file without locking? Are we just settling with one IndexWritter per Index?

We don't do anything... Only one Indexwriter per index supported for S3 stuff

@fulmicoton @guilload
What is the ManifestEntry introduced in Stage splits code? Is there a detailed explanation or code sample somewhere?
And, what exactly do you mean by a split needs to be staged before uploading any of its files to the storage?

Manifest Entry a struct that contains some information about each file metadata. They checksum, their size, their filepath if I recall correctly. You can have a look at the struct in barrel.

what exactly do you mean by a split needs to be staged before uploading any of its files to the storage?

In tantivy and in Quickwit, we need to be tolerant with to failure, at least to the extent of the Fail-stop assumption.
As long as the memory is not corrupted... The system should still work as expected.

You should not have dangling file on S3 either.
For this reason, before registering any files, we need to record somewhere our intent to upload files.
When we restart the CLI, the CLI will check which files are in the Staged state and remove them...
Another approach would be to list the files from the stoarge, and remove all of the files in the directory that are not useful to the index according to the index meta. We do not use this approach because  a user that added its own files to the directory could see their file deleted by Quickwit. Besides, listing files on S3 can rapidly get slow, and it would require to add an extra method to the Storage trait.

@fulmicoton @guilload
I have a question about list_splits.
Assume that there are splits that contain the following range in the meta store.
If I call the list_splits function with the following range, which splits should the return value contain?
split_1 : std::opts::Range { start : 0, end : 100 }, SplitState::Published
split_2 : std::opts::Range { start : 100, end : 200 }, SplitState::Published
split_3 : std::opts::Range { start : 50, end : 150 }, SplitState::Published
list_splits( SplitState::Published, std::opts::Range { start : 50, end : 200 } )
Is the above list_splits returns value expected to be split_2 and split_3? or only split_3 ?
Can you tell me the specification of the filter using time_range in list_splits?

We want to retrieve any split that overlaps with the time range. So for a [50, 200) range, every split that has range.end >= 50 or range.start < 200 is a candidate.


NavController.viewWillLeave emits spuriously on application start

Short description of the problem:
NavController.viewWillLeave emits a value when the application starts. I'm not sure where this value is coming from, but it's a ViewController.
What behavior are you expecting?
No value is emitted.
Steps to reproduce:

Create a blank Ionic v2 app.
Replace app.ts with the below code.
Run the application.

import { Component, ViewChild, OnInit, OnDestroy } from '@angular/core';
import { ionicBootstrap, Platform, NavController } from 'ionic-angular';
import { StatusBar } from 'ionic-native';
import { Observer, Subscription } from 'rxjs';

import { HomePage } from './pages/home/home';

@Component({
    template: '<ion-nav [root]="rootPage" #myNav></ion-nav>'
})
export class MyApp implements OnInit, OnDestroy {
    @ViewChild('myNav') public navCtrl: NavController;
    public rootPage: any = HomePage;

    private viewWillLeaveSubscription: Subscription;

    constructor(public platform: Platform) {
        platform.ready().then(() => {
            StatusBar.styleDefault();
        });
    }

    public ngOnInit() {
        this.viewWillLeaveSubscription = this.navCtrl.viewWillLeave.subscribe((value) => {
            console.log('viewWillLeave next(' + value + ')');
            if (value) {
                console.log(value);
            }
        });
    }

    public ngOnDestroy() {
        this.viewWillLeaveSubscription.unsubscribe();
    }
}

ionicBootstrap(MyApp);

The output is this. The componentType is undefined, so I don't know where this ViewController object is coming from.
app.ts:26 viewWillLeave next([object Object])
app.ts:28 ViewController {
_emitter: EventEmitter
_hdAttr: null
_leavingOpts: null
_loaded: false
_onDidDismiss: null
_onWillDismiss: null
_tbRefs: Array[0]
componentType: undefined
data: Object
didEnter: EventEmitter
didLeave: EventEmitter
didUnload: EventEmitter
fireOtherLifecycles: true
index: -1
instance: Object
isOverlay: false
name: ""
state: 2
willEnter: EventEmitter
willLeave: EventEmitter
willUnload: EventEmitter
__proto__: Object
}

Other information: (e.g. stacktraces, related issues, suggestions how to fix, stackoverflow links, forum links, etc)
Which Ionic Version? 1.x or 2.x
2.0.0-beta.11
Plunker that shows an example of your issue
http://plnkr.co/edit/5UV6VgfAz14uVHIr9zr0?p=preview
Run ionic info from terminal/cmd prompt: (paste output below)
Your system information:

Cordova CLI: 6.3.1
Gulp version:  CLI version 3.9.1
Gulp local:   Local version 3.9.1
Ionic Framework Version: 2.0.0-beta.11
Ionic CLI Version: 2.0.0
Ionic App Lib Version: 2.0.0-beta.20
OS:
Node Version: v4.4.7

Hello, thanks for opening an issue with us! I have made a plunker here which shows the solution for this issue. As you can see from the plunker the change was to use our Ionic 2 lifecycle events instead of the default Angular lifecycle events. You can read more about our lifecycle events here. Thanks for using Ionic!


Issue 2066: Fix the conv for stride > ksize case.

Issue 2066: Fix the conv for stride > ksize case. Passed all the tests and locally verified it fixed the problem.

Can one of the admins verify this patch?

@tensorflow-jenkins test this please

I think this is a known flaky test.
@zheng-xq: shouldn't we add a test to conv_ops_test.py for this?

@tensorflow-jenkins test this please


Fix #1446 : validate meshes before export, removing NaN

Fix #1446 : validate meshes before export, removing NaN for example

Any updates on that matter?

This will be merged soon


[ACS][CallAutomation] 1.2.0 alpha version update and changes to align latest beta + test updates

Versioning update to 1.2.0 for alpha package build
With release of 1.1.0-Beta, update to main package to match
Updated all tests and re-recorded

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-communication-call-automation


oidc login with existing kubeconfig that works with kubectl and openlens does not work with headlamp desktop application

This is a fragment of my kubeconfig (redacted) that works perfectly with kubectl and open lens
apiVersion: v1
clusters:
    - cluster:
        certificate-authority-data: LS0t...
        server: https://stagnig:6443
      name: staging
contexts:
    - context:
        cluster: staging
        namespace: user-asavinykh
        user<EMAIL_ADDRESS>      name: staging
current-context: staging
kind: Config
preferences: {}
users:
    - name<EMAIL_ADDRESS>      user:
        auth-provider:
            config:
                client-id: redacted.apps.googleusercontent.com
                client-secret: redacted
                id-token: eyJ...
                idp-issuer-url: https://accounts.google.com
                refresh-token: redacted
            name: oidc

However it does not let me log in with the headlamp desktop app.

Hey @AndrewSav When you say it doesn't let me log in is it not showing the Sign In page or It doesn't work after completing the signin flow.

@yolossn it does show the sign in page, but it should not since the kubeconfig is already in the usable state there is no need for the signin UI flow, it should just work. Neither OpenLens nor kubectl require any visible signin flow with this config.
The sign-in page does not work, either but I'm not surprised about this because it could not possibly know how to login to my cluster, so I'm not concerned about it.

Related developer documentation

Hey
also similar issue on my side:
We are authenticating via OpenUnison: https://openunison.github.io/
I can either copy a command from their dashboard and it then generates the whole kubeconfig or via their CLI.
The kubeconfig then looks like this:
- name: my-username
  user:
    auth-provider:
      config:
        client-id: kubernetes
        client-secret: ""  #this is empty
        id-token: eyJh...
        idp-certificate-authority-data: ""
        idp-issuer-url: OUR-DOMAIN-URL/auth/idp/k8sIdp
        refresh-token: q1b....
      name: oidc

When i manually create/refresh the kubeconfig, it works via openlens.
But with headlamp when i then press "Sing In" it opens following URL :
http://localhost:4466/oidc-callback?error=unauthorized_client
and says
invalid request state is empty
I don't see from where this URL is getting picked up or how i could adjust it to my idp-issuer-url value from the kubeconfig.
Or could we even just skip this check, because we can outside of headlamp create/refresh the token manually ? That worked would work fine for me now as it's also the state with openlens


Add an option to restic backup to override the snapshot's "paths" field

Output of restic version
restic 0.9.6 compiled with go1.13.4 on linux/amd64
What should restic do differently? Which functionality do you think we should add?
Add an option to restic backup to override the snapshot's "paths" field.
What are you trying to do?
My first-time restic experience went like this:

Looks like it has some features for excluding patterns etc., but I want more control over which files are backed up.
Ah, no problem, it has an option for that: restic backup --files-from lets me just tell it exactly which files to backup.  Great!
Hmm, it's actually treating the contents not as files, but as patterns.  Never mind, I'll just escape any glob characters in the file paths, and...
Ok, it worked!  Uploading the snapshot manifest at the end took a strangely long time, but :shrug:.  Now let's take a look at restic snapshots, and OOOH sweet jesus!

As you can probably guess, I had created a snapshot with 140k items in its paths field :grimacing:. As far as I can tell, this is not actually causing any real problems.  I'd just like to be able to override the "path" field when I make a backup, so that the output of restic snapshots is more usable.
Did restic help you today? Did it make you happy in any way?
Restic seems really good.  I'm a long-time bup user, and bup is great too - but I wanted encryption so I could use a cloud storage provider.  Restic seems to follow mostly the same design principles as bup, so I thought I'd give it a try.  I'm liking it so far!

We are looking for the same feature! Currently at CERN we are backing up around 170M files everyday with restic. The problem is that sometimes we are traversing a quite big folders just for a few changes that we can get quickly from namespace information prior to the backup. This is causing a lot of activity on our instances and affects user experience. Would be nice to pass directly these files to restic but linked to the main path of the backup. Something like restic backup —files-from file.txt /my/path, assuming that files in file.txt would be in the form /my/path/file and using /my/path internally in the repo so then is linked to previous backups.
And @asayers, is not just a visual thing, because if you restore that snapshot you will only get the files specified in the file, not the incremental view of the path. The same for the fuse view.

I would add that it also seems to interfere with restic selecting a parent snapshot at backup as the paths field differs for each snapshot (if you have any new/deleted files). In fact this has been filed as a bug already: #2246 so a solution here could fix two bugs at once.
Perhaps instead of allowing the user to set the paths field restic could set the field to the smallest unique path(s) from all the files that have been added to the snapshot? i.e if the snapshot contains:
/home/a/foo
/home/b/bar
/var/log/a
/var/log/b

It would set paths to /home/, /var/log.

Thanks for the suggestions. Marking as feature enhancement for now. There's lots of other issues to look at currently.

@chrahunt reported a few further issues caused by the large number of paths:

These are the problems I have:

Every backup, a new 11MB snapshot file is created and uploaded. This results in about 1 GB/day extra storage usage.
restic snapshots --compact latest is slow, up to 15 seconds. I have to run this to retrieve the parent snapshot and avoid backup delays, similar to #2246.
When running restic forget, I need to specify --group-by host, since the snapshot paths are not usable to me for grouping.
If I want to list snapshots, I must always use --compact, so paths are excluded. I must never use --json, since paths are always included.

If restic accepted an option like I described above, then I can pass --snapshot-path /.

Something like restic backup —files-from file.txt /my/path, assuming that files in file.txt would be in the form /my/path/file and using /my/path internally in the repo so then is linked to previous backups.

In #3200 I propose a new option --set-path to backup which completely overwrites the paths. IMO this is the most flexible way. In your case, you could then use restic backup --files-from file.txt --set-path /my/path, but also something like restic backup --files-from file.txt /other/path --set-path /my/path --set-path /other/path is working (just make sure you --set-path all paths that are included in the backup).

@aawsome Great, that's exactly what I was looking for!

#1376 is a duplicate of this
I am running into this issue too, I am importing snapshots from another system into Restic, and I can't do that at the original path (which contains current data).

+1


🎉 Added My Name to Contributors List & Enhanced Intro Section

Added my name to the list of contributors.
Improved the introduction to make it more welcoming and user-friendly.

Kindly merge it @AKSHITHA-CHILUKA  @sanjay-kv

Create a new PR


remove python-exp

resolve conflict, reopen #2999 , close #2962

For beta version of python, one can use versions/python-beta


[SPARK-25574][SQL]Add an option keepQuotes for parsing csv file

What changes were proposed in this pull request?
In this PR, I added a new option for csv file - keepQuotes.
In our project, when we read the CSV file, we hope to keep quotes.
For example:
We have such a record in the CSV file.:
ab,cc,,"c,ddd"
We hope it displays like this:
+----+---+----+---+
| _c0 | _c1 | _c2  |   _c3 |
+---+---+----+----+
|  ab  |  cc   |  null     | "c,ddd" |
Not like this:
+----+---+----+----+
| _c0 |  _c1  | _c2 |   _c3  |
+---+----+----+----+
|  ab   |  cc   |  null   | c,ddd |
+----+---+----+---+
How was this patch tested?
Added a unit test.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3593/
Test PASSed.

Test build #96793 has started for PR 22590 at commit 9c46a72.

Test build #96793 has finished for PR 22590 at commit 9c46a72.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96793/
Test PASSed.

@10110346 Could you describe the use case when you need this, please. As @HyukjinKwon said in one of PR, uniVocity parser support many config options, and we cannot expose everything from underlying libraries otherwise we couple so much to third-party libs.

Test build #97156 has finished for PR 22590 at commit 4e9cd6d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97156/
Test PASSed.

Normally, it's better to have no quotes, but in our production environment, the user requests quotes to be displayed, so we need this option.

Test build #98076 has started for PR 22590 at commit 59c1034.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4526/
Test PASSed.

Test build #98076 has finished for PR 22590 at commit 59c1034.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98076/
Test FAILed.

retest this please

Test build #98473 has started for PR 22590 at commit 59c1034.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4770/
Test PASSed.

They should be documented in API doc like DataFrameReader.scala. For site, we should avoid doc duplication - It's a general issue to document options.

I wonder how important it is. I know spark-csv at Databricks supported different quote modes and that's gone when we ported that into Spark - the root cause was due to replacing the library from apache-common into univocity.
After few years, I only saw one request about reviving the quote mode proposed here - so I suspect how important it is.
Basically, @MaxGekk described my stand correctly. Can we investigate a way to set the arbitrary parse settings options?

@HyukjinKwon I think it is not important. but our customers need this feature.
Yeah, it is better to find a way to set the arbitrary parse settings options

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5430/
Test PASSed.

Test build #99353 has finished for PR 22590 at commit 9e3c4bd.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99353/
Test FAILed.

Hm, let's try to find a way to expose other parse options. I think we shouldn't allow every options available on Univocity ...

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/8163/
Test PASSed.

In that case, shouldn't your input be something like ab,cc,,"\"c,ddd\""? quotes are inherently not part of the value. This doesn't make sense to me semantically.

In that case, shouldn't your input be something like ab,cc,,"\"c,ddd\""? quotes are inherently not part of the value. This doesn't make sense to me semantically.

Yeah, I will close it ,thanks


all not being respected from an import *

Microsoft/vscode-python#494
# A.py
import B
B. # intellisense shows `foo` and `bar`, but should only show `foo`.

# B.py
from C import *

# C.py
__all__ = ['foo']
foo = 42
bar = 24

Works in the new language server.

We have basic __all__ support, though not for more complicated expressions (like those which combine multiple modules' __all__s into a single list).

An unfortunate side effect of handling this but only partially means that we may under complete in some situations (especially projects like Django which do lots of fun stuff). I've opened #620 to track improving the support.

Repoening, the code which filters __all__ has been temporarily removed as commonly used libraries are using it in ways we can't handle yet.


Wrapping of TextInput works in iOS fixes #4

I have set the prop to "none", it could also be set to {this.state.modalVisible ? 'auto' : 'none'}, that would make the children receive any touch-events while the modal is visible, but since the modal is covering the screen I don't think that would make any sense, and just add more logic with no use.

Looks good. Thanks @mikaello


MailMessage->from not set in ResetPassword notification

Laravel Version: 5.3.18
PHP Version:  5.6.25
Database Driver & Version: MySQL 5.7.13

Description:
I was trying to reset password using mailgun on my localhost but the emails wouldn't send.  Upon inspection, "from" field was empty in mail headers.  Then I noticed that in ResetPassword.php notification, in toMail() method, we're not setting the "from" field and the MailMessage object is missing "from" info.  When I manually set it, it sends emails fine.  I'm not sure if this is a bug or if I missed a step in sending reset email.
Steps To Reproduce:

Set up to send email with mailgun
Generate auth scaffolding (php artisan make:auth)
Reset password
No email sent

Global "From" Address can config in config/mail.php
Please take a look at this:
https://github.com/laravel/laravel/blob/master/config/mail.php#L58
Hope this help. :)

Check @minkbear's comment.


Support Conv3d layer in LoRA and IA3

Support Conv3d layer in LoRA and IA3

Add Conv3d layers for LoRA and IA3
Add test cases in test_custom_models

Issue GH-2079

Thanks a lot for the updates. I have a few more comments:
...

I implemented all of these changes. Although in the Dora case I wonder if having those 2 sub-classes just to avoid a single trinary statement is a bit over-engineered, if we're going to do that for LoRA (and IA^3 and maybe more) it probably makes sense to be consistent.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@BenjaminBossan I don't have a specific realistic example at the moment. I'm working on analyzing CT images (in the vein of https://github.com/reginabarzilaygroup/Sybil) but nothing ready to be published. If you think a toy example would be useful I can add that to my todo list; video classification would be the obvious choice.

Thanks for the info, I agree something like video classification would be a good example. Would that be something you could take a look at soon or rather do it separately from this PR?

I'm not sure when I'll be able to get to it so I'd prefer to keep it separate if that's alright.