[R] Predictions with base margin are off by constant number

ref https://github.com/dmlc/xgboost/issues/9810
It seems in the R interface, adding a base margin to a DMatrix somehow results in the predictions being off by exactly 1.48.
Example:
library(xgboost)
data("agaricus.train")
x <- agaricus.train$data[1:100, ]
y <- agaricus.train$label[1:100]
n_threads = 1
dm_labelled <- xgb.DMatrix(
    data = x,
    label = y,
    nthread = n_threads
)
dm_only_x <- xgb.DMatrix(
    data = x,
    nthread = n_threads
)
dm_w_base_margin <- xgb.DMatrix(
    data = x,
    nthread = n_threads,
    base_margin = rep(0, nrow(x))
)
params <- list(
    objective = "binary:logistic",
    tree_method = "hist",
    nthread = n_threads
)
model <- xgb.train(
    data = dm_labelled,
    params = params,
    nrounds = 5L
)
pred_wo_margin <- predict(model, dm_only_x, outputmargin = TRUE)
pred_w_margin <- predict(model, dm_w_base_margin, outputmargin = TRUE)
print(pred_wo_margin[1] - pred_w_margin[1])

[1] -1.48

In contrast, the parameter seems to work as expected on the python interface:
import numpy as np
rng = np.random.default_rng(seed=123)
X = rng.standard_normal(size=(100,10))
y = rng.integers(2, size=100)
import xgboost as xgb
dmat_labelled = xgb.DMatrix(data=X, label=y)
dmat_only_x = xgb.DMatrix(data=X)
dmat_w_base_margin = xgb.DMatrix(data=X, base_margin=np.zeros(y.shape[0]))
model = xgb.train(
    dtrain=dmat_labelled,
    params={
        "objective" : "binary:logistic",
        "tree_method" : "hist",
    },
    num_boost_round = 5
)
pred_wo_margin = model.predict(dmat_only_x, output_margin=True)
pred_w_margin = model.predict(dmat_w_base_margin, output_margin=True)
print(pred_w_margin[0] - pred_wo_margin[0])

Hmm, there are some inconsistencies between the base margin handling and base score handling, I will take a deeper look.

basically, whether we should apply the inverse link function to the user-provided base margin. At the moment, it's no. But I'm wondering whether this is a right thing to do.

I do not think this is related to transformations involved in the objective, since here it happens also for outputmargin=TRUE and doesn't happen under the same settings in python.
Curiously, while the python interface does give the expected result for objective binary:logistic, it seems to suffer from the same problem as the R interface for objective reg:squarederror.
From some testing, it seems there are two issues involved here:

When the DMatrix has base_margin, the model doesn't add its base_score to it (but note that this would not explain the issue with the binary objective mentioned earlier, since the base_score there is not equal to the difference in predictions).
The base_margin is substracted from the output instead of being added.

Here's an example:

R

library(xgboost)
library(testthat)
data(mtcars)
y <- mtcars$mpg
x <- as.matrix(mtcars[, -1])
n_threads = 1
dm_labelled <- xgb.DMatrix(
    data = x,
    label = y,
    nthread = n_threads
)
dm_only_x <- xgb.DMatrix(
    data = x,
    nthread = n_threads
)
set.seed(123)
base_margin <- rnorm(nrow(x))
dm_w_base_margin <- xgb.DMatrix(
    data = x,
    nthread = n_threads,
    base_margin = base_margin
)
params <- list(
    objective = "reg:squarederror",
    tree_method = "hist",
    nthread = n_threads
)
model <- xgb.train(
    data = dm_labelled,
    params = params,
    nrounds = 5L
)
models_base_score <- as.numeric(jsonlite::fromJSON(xgb.config(model))$learner$learner_model_param$base_score)
pred_wo_margin <- predict(model, dm_only_x)
pred_w_margin <- predict(model, dm_w_base_margin)
expect_equal(
    pred_wo_margin,
    pred_w_margin + models_base_score - base_margin,
    tolerance = 1e-5
)

Python

import numpy as np, xgboost as xgb
import json
mtcars = np.array([[21,6,160,110,3.9,2.62,16.46,0,1,4,4],
[21,6,160,110,3.9,2.875,17.02,0,1,4,4],
[22.8,4,108,93,3.85,2.32,18.61,1,1,4,1],
[21.4,6,258,110,3.08,3.215,19.44,1,0,3,1],
[18.7,8,360,175,3.15,3.44,17.02,0,0,3,2],
[18.1,6,225,105,2.76,3.46,20.22,1,0,3,1],
[14.3,8,360,245,3.21,3.57,15.84,0,0,3,4],
[24.4,4,146.7,62,3.69,3.19,20,1,0,4,2],
[22.8,4,140.8,95,3.92,3.15,22.9,1,0,4,2],
[19.2,6,167.6,123,3.92,3.44,18.3,1,0,4,4],
[17.8,6,167.6,123,3.92,3.44,18.9,1,0,4,4],
[16.4,8,275.8,180,3.07,4.07,17.4,0,0,3,3],
[17.3,8,275.8,180,3.07,3.73,17.6,0,0,3,3],
[15.2,8,275.8,180,3.07,3.78,18,0,0,3,3],
[10.4,8,472,205,2.93,5.25,17.98,0,0,3,4],
[10.4,8,460,215,3,5.424,17.82,0,0,3,4],
[14.7,8,440,230,3.23,5.345,17.42,0,0,3,4],
[32.4,4,78.7,66,4.08,2.2,19.47,1,1,4,1],
[30.4,4,75.7,52,4.93,1.615,18.52,1,1,4,2],
[33.9,4,71.1,65,4.22,1.835,19.9,1,1,4,1],
[21.5,4,120.1,97,3.7,2.465,20.01,1,0,3,1],
[15.5,8,318,150,2.76,3.52,16.87,0,0,3,2],
[15.2,8,304,150,3.15,3.435,17.3,0,0,3,2],
[13.3,8,350,245,3.73,3.84,15.41,0,0,3,4],
[19.2,8,400,175,3.08,3.845,17.05,0,0,3,2],
[27.3,4,79,66,4.08,1.935,18.9,1,1,4,1],
[26,4,120.3,91,4.43,2.14,16.7,0,1,5,2],
[30.4,4,95.1,113,3.77,1.513,16.9,1,1,5,2],
[15.8,8,351,264,4.22,3.17,14.5,0,1,5,4],
[19.7,6,145,175,3.62,2.77,15.5,0,1,5,6],
[15,8,301,335,3.54,3.57,14.6,0,1,5,8],
[21.4,4,121,109,4.11,2.78,18.6,1,1,4,2]])
y = mtcars[:, 0]
X = mtcars[:, 1:]

rng = np.random.default_rng(seed=123)
base_margin = rng.standard_normal(size=X.shape[0])

dm_labelled = xgb.DMatrix(data=X, label=y)
dm_only_x = xgb.DMatrix(data=X)
dm_w_base_margin = xgb.DMatrix(data=X, base_margin=base_margin)
params = {
    "objective" : "reg:squarederror",
    "tree_method" : "hist",
}
model = xgb.train(
    dtrain=dm_labelled,
    params=params,
    num_boost_round=5
)
pred_wo_margin = model.predict(dm_only_x)
pred_w_margin = model.predict(dm_w_base_margin)
model_json = json.loads(model.save_raw("json").decode())
model_base = float(model_json["learner"]["learner_model_param"]["base_score"])

np.testing.assert_almost_equal(
    pred_wo_margin,
    pred_w_margin + model_base - base_margin,
    decimal=5
)

Actually the comment above is wrong, I messed up the signs, calculation in xgboost is correct. Just doesn't include base_score.

On further experiments, looks like this is not an R-specific issue. It just seems to be the case that it doesn't sum the intercept when base_margin is added.
For large enough datasets, the intercept that the model will end up using seems to be easily calculable as the constant prediction that minimizes the objective, and adding this to the prediction with margin does seem to produce the expected numbers.

library(xgboost)
data("agaricus.train")
x <- agaricus.train$data
y <- agaricus.train$label
dm_labelled <- xgb.DMatrix(
    data = x,
    label = y,
    nthread = n_threads
)
dm_only_x <- xgb.DMatrix(
    data = x
)
zeros <- rep(0, nrow(x))
dm_w_base_margin <- xgb.DMatrix(
    data = x,
    base_margin = zeros
)
params <- list(
    objective = "binary:logistic"
)
model <- xgb.train(
    data = dm_labelled,
    params = params,
    nrounds = 1L
)
pred_wo_margin <- predict(model, dm_only_x, outputmargin = TRUE)
pred_w_margin <- predict(model, dm_w_base_margin, outputmargin = TRUE)
optimal_intercept <- log(sum(y == 1)) - log(sum(y == 0))

expect_equal(pred_wo_margin - pred_w_margin, rep(optimal_intercept, nrow(x)), tolerance=1e-4)

On further experiments, looks like this is not an R-specific issue. It just seems to be the case that it doesn't sum the intercept when base_margin is added.

Indeed, when a user-supplied base margin is available, the intercept is ignored. They are both global bias. I don't have a strong opinion on how they should mix. I will take a closer look today.


Change default values in containers dashboard to dashes

BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1372318
This change will help QE automation to know when the values were received from the server.
@martinpovolny @himdel Please review
cc @simon3z

@serenamarie125 from what I've read, this seems to be affecting QE testing being able to tell when the dashboard components are done loading.  Currently, this PR puts a - in place, which is replaced with the actual value.  Do you have any UX recommendations?  I would have thought a small spinner or something might be shown for components that are fetching data.  Thx, Dan

is how it looks during loading..
So maybe it'd make sense to use the - in the data, but in the UI, add a condition to display a spinner like in the other fields in that case...

is how it looks during loading..
So maybe it'd make sense to use the - in the data, but in the UI, add a condition to display a spinner like in the other fields in that case...

@himdel I think a spinner there wouldnt look as nice. Empty strings instead of dashes maybe?

@himdel @zeari @dclarizio Where did these dashes coming from, it wasn't suggested from UX.  Staying in line with the PatternFly Dashboard Card pattern, we should be using that same loading indicator in all cards.  Was an issue found with that approach ?

@serenamarie125 We used to have zeroes instead of dashes while loading the page. That's bad for QE because they have no way to determine when the page has finished loading. So that's where we are now :).
@zeari I don't know, maybe it'd be less nice but it would definitely be unambigously clear that it's still loading. Would you mind trying and posting a screenshot for @serenamarie125 to see?

@zeari I don't know, maybe it'd be less nice but it would definitely be unambigously clear that it's still loading. Would you mind trying and posting a screenshot for @serenamarie125 to see?

There we go

@zeari I meant a sceeenshot with the spinners..

@zeari can you reply to @himdel? https://github.com/ManageIQ/manageiq/pull/11359#issuecomment-248930572

@dclarizio @himdel Sorry for the belated response.
do we really want all those spinners? feels a lot more clunky than having dashes\spaces\nothingץ
Remember all those spinners are moving...

@serenamarie125 I'm thinking the spinners are the correct way to do this, but if they take a while, not sure it's the best UX overall.  Thoughts?

ping @serenamarie125

@dclarizio @himdel Changed to spinners

@zeari do we still need the dashes there? Seems like that change is no longer needed, is it?
Otherwise, the only concern I have is that by doing 2 conditions instead of N*2, people might add a new pf-card without adding it to the loading screen, and there's lot of reuse that might be deduplicated.
(I mean, maybe something like %magic-card{:loading => "!loadingDone", :status => "objectStatus.pods"} and having that handle the details for each, might be a bit cleaner ..and shorter :).)

@serenamarie125 I'm thinking the spinners are the correct way to do this, but if they take a while, not sure it's the best UX overall. Thoughts?

@serenamarie125 bump :smile: any thoughts before we merge this, that's a lot of spinners?
@zeari comments on https://github.com/ManageIQ/manageiq/pull/11359#issuecomment-259685879 ?

@dclarizio that's recommended from PF.  The problem is each take a different amount of time, so it's not like you can say show an entire "dashboard loading" message, because you may have inventory data accessible way before trend data for instance.

Thx @serenamarie125, I figured as much, plus the fact that hopefully they won't ALL be spinning for a long time.

@zeari from @himdel above:

Otherwise, the only concern I have is that by doing 2 conditions instead of N*2, people might add a new pf-card without adding it to the loading screen, and there's lot of reuse that might be deduplicated.

(I mean, maybe something like %magic-card{:loading => "!loadingDone", :status => "objectStatus.pods"} and having that handle the details for each, might be a bit cleaner ..and shorter :).)

Any response, you want to clean this up or create a GH issue to do it later?

@dclarizio @himdel im having some trouble getting to this one. Ill Definitely get to this in a week or two.

@zeari I think this should be moved to manageiq-ui-classic

@zeari Please re-create this PR in the https://github.com/ManageIQ/manageiq-ui-classic repo.


Fix minor inconsistencies in tutorial

I think I saw a couple of typos or inconsistencies in the tutorial:

Grammar error in commands/one-or-multiple.md
Wrong command listed in command/options.md
What looks to be missing words in first-steps.md (I doubt a typer "parameter" or "argument" refers to a python function)

Thanks a lot for your work on this great tool. I hope this helps!

Thanks for your feedback.
Oops sorry, I realize I had not updated the python files associated with the code block impacted by my update in commands/one-or-multiple.md. I just corrected this with my second commit.
(I had not thoroughly read and applied the contributing guide that describes this link between docs and python files/tests. That should teach me!)


What is difference between Sleipnir and Quick?

What is difference between Sleipnir and Quick? They seem very similar to me. What are some trade offs and advantages of using Sleipnir?
Thanks!

A key difference seems to be that https://github.com/Quick/Quick actually supports Swift 3.0 and isn't stuck at a 8 months old stage.


Display QRCode for ngrok url

When running npm run start:tunnel display a QR code with the ngrok URL
to make it easier to test your app on mobile devices
Please tell me if you find this feature useful as I do before I add a test 🙂
Edit: here's a screenshot

Coverage remained the same at 98.958% when pulling 9f95dcee95ccc1e12cfe232a7204a314ee17a339 on posva:add-qr-code into d4e0052f101aca14e4f89b3e474edeab056e25fb on mxstbr:dev.

Nice idea! If we merge this it could need some tests yes, and also add a console message to the user? Also personally I don't even know how I should scan QR codes from my phone, it would probably help to update the docs to mention and explain this cool feature.

Glad you like it! I'll add the info log, a test and some docs in some hours

Would be nice to know what @mxstbr thinks, even if he's quite busy ATM 😉

Then I'll wait for his input 😄

Not a fan of QR codes personally, to be completely honest. What benefit does this have? I have to have a separate app installed, then open that app and scan that QR code – might as well enter the short URL we have?

Yes, but it takes more time to manually enter the url. It's even worse if you make any mistake given the nature of ngrok urls

I guess that makes sense, I still don't like just throwing that huge blob up into a users terminal though. Is there a nicer way to do this? Maybe with some sort of link that, when clicked on, shows the QR code?

I did a quick search and found about a service that does that: https://chart.googleapis.com/chart?cht=qr&chl=github.com&chs=360x360&choe=UTF-8&chld=L|2 (replace github.com with the actual url)
I still think directly having the QR code is better because not all terminals allow clicking on the console and it extends the time to reach the website on your phone.
That link can be added below the proxy link. It's a bit long though

Coverage remained the same at 99.177% when pulling a15be6c77c7da3734cda685c41dc0c865c733570 on posva:add-qr-code into a6ea1ded0196bbe0ecb237f3c7656815c10d71a7 on mxstbr:dev.

If we display QR code in console user have to scan QR right after start dev server because webpack will display a bunch of building message.
How about add QR code in "/about" page, or display it in the footer.

@nhducit No, it won't. It looks as in the screenshot

@posva  the QR code is displayed when we run npm run start right?  whenever we modify source code console display new webpack log message then have to scroll up to see the QR code.

It display a new line when you changes something. Try it to actually see it 🙂

Coverage remained the same at 98.605% when pulling c021e5e5f7e3cdd6ec294ee171e236ce4c896af6 on posva:add-qr-code into 33485f41053caa6e90b9cbede8b51c168b38bda4 on mxstbr:dev.

The only thing holding me back is the size of that QR code. Is there any way we can reduce it?

@KarandikarMihir Unfortunately no, it's already using Unicode chars to make it look smaller. None of my teammates found it big while using it. Since it's displayed only in the tunnel mode and only when the server is launched, I didn't have any issue about it. Keep in mind that if the QR code is too small it may be hard to scan it.

When I first saw this I was like woah that's pretty cool a QRCode in the console, but after thinking about it  I tend to agree with @mxstbr and @KarandikarMihir, it's fairly large and distracting and it seems like it's actually more effort to use than copying and pasting the link. Thanks for sharing, today I learned you can put a QRCode in the console, much awesome 😍

it's actually more effort to use than copying and pasting the link

Unfortunately, you cannot copy/paste the link from a mobile device (without any extra app)
It's something my team really enjoyed so I shared it. I'm sure people will find useful to have access to a qr code of any kind (it can be a link to generate the QR code as said below).
The PR being outdated I'm closing this. More than 3 months have passed 😆


[react-snap-carousel]: Unsupported scroll margin value, expected  value, received

Using react-snap-carousel in older browser, from our issue tracker seems to be below iOS 15: [react-snap-carousel]: Unsupported scroll margin value, expected <length> value, received  is thrown, making our nextjs app go Application error: a client-side exception has occurred (see the browser console for more information).
I can't quite figure out, what in the library that's not supported by iOS 14 (both scroll-padding and scroll-margin should be supported), would it make sense to either be able to pass a fallback value or bypass the lib throwing errors?
Update
Patched like this, and now it works in iOS 14
diff --git a/node_modules/react-snap-carousel/dist/use-snap-carousel.js b/node_modules/react-snap-carousel/dist/use-snap-carousel.js
index 48c97d5..2c3162e 100644
--- a/node_modules/react-snap-carousel/dist/use-snap-carousel.js
+++ b/node_modules/react-snap-carousel/dist/use-snap-carousel.js
@@ -212,7 +212,7 @@ var _getOffsetRect = (el) => {
 };
 var getScrollPaddingUsedValue = (el, pos) => {
   const style = window.getComputedStyle(el);
-  const scrollPadding = style.getPropertyValue(`scroll-padding-${pos}`);
+  const scrollPadding = style.getPropertyValue(`scroll-padding-${pos}`) || "0px";
   if (scrollPadding === "auto") {
     return 0;
   }
@@ -231,7 +231,7 @@ var getScrollPaddingUsedValue = (el, pos) => {
 };
 var getScrollMarginUsedValue = (el, pos) => {
   const style = window.getComputedStyle(el);
-  const scrollMargin = style.getPropertyValue(`scroll-margin-${pos}`);
+  const scrollMargin = style.getPropertyValue(`scroll-margin-${pos}`) || "0px";
   const invalidMsg = `Unsupported scroll margin value, expected <length> value, received ${scrollMargin}`;
   assert(scrollMargin.endsWith("px"), invalidMsg);
   const value = parseInt(scrollMargin);
diff --git a/node_modules/react-snap-carousel/dist/use-snap-carousel.mjs b/node_modules/react-snap-carousel/dist/use-snap-carousel.mjs
index 4f87451..4bd9f61 100644
--- a/node_modules/react-snap-carousel/dist/use-snap-carousel.mjs
+++ b/node_modules/react-snap-carousel/dist/use-snap-carousel.mjs
@@ -188,7 +188,7 @@ var _getOffsetRect = (el) => {
 };
 var getScrollPaddingUsedValue = (el, pos) => {
   const style = window.getComputedStyle(el);
-  const scrollPadding = style.getPropertyValue(`scroll-padding-${pos}`);
+  const scrollPadding = style.getPropertyValue(`scroll-padding-${pos}`) || "0px";
   if (scrollPadding === "auto") {
     return 0;
   }
@@ -207,7 +207,7 @@ var getScrollPaddingUsedValue = (el, pos) => {
 };
 var getScrollMarginUsedValue = (el, pos) => {
   const style = window.getComputedStyle(el);
-  const scrollMargin = style.getPropertyValue(`scroll-margin-${pos}`);
+  const scrollMargin = style.getPropertyValue(`scroll-margin-${pos}`) || "0px";
   const invalidMsg = `Unsupported scroll margin value, expected <length> value, received ${scrollMargin}`;
   assert(scrollMargin.endsWith("px"), invalidMsg);
   const value = parseInt(scrollMargin);

@KasperAndersson thanks reporting this -- I'd seen this in some error logs on a site I was working on a while back but didn't have the time to investigate.
I've pushed up your fix here https://github.com/richardscarrott/react-snap-carousel/pull/26

@KasperAndersson I've published a new beta version with your fix which you can try out using npm add<EMAIL_ADDRESS>I'm going to ship it to production on a site I'm working on and if it's all good will release it under the latest tag as 0.4.1 sometime next week 👍

Published as 0.4.1


fix: alarm type/level sensors were ignored if zwavejs was found

Proposed change

Restore monitoring of alarm level/types for locks that are not throwing ZwaveJS notifications.
Type of change

[ ] Dependency upgrade
[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (which adds functionality)
[ ] Breaking change (fix/feature causing existing functionality to break)
[ ] Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: fixes #317

Initial test shows my issue is resolved.


Tenant.GetSiteCollections only returns first 300

As described here: https://www.yammer.com/itpronetwork/#/Threads/show?threadId=508865792
This needs to iterate by 300.
I will tackle this quick fix.

Fix is provided in #523, waiting for someone else from the core team to confirm changes.

#523 has been merged. Thanks @SharePointRadi


Update applicator_patch.rb and compatibility with redmine 5.0.2

I suggest to change module Deface to module ApplicatorPatch
Under Redmine 5.0.2 it doesn't work with the Zeitwerk loader:
/usr/local/bundle/gems/zeitwerk-2.6.0/lib/zeitwerk/loader/callbacks.rb:25:in `on_file_autoloaded': expected file /usr/src/redmine/plugins/redmine_base_deface/lib/applicator_patch.rb to define constant ApplicatorPatch, but didn't (Zeitwerk::NameError)

  raise Zeitwerk::NameError.new("expected file #{file} to define constant #{cpath}, but didn't", cref.last)

  ^^^^^

from /usr/local/bundle/gems/zeitwerk-2.6.0/lib/zeitwerk/kernel.rb:28:in `require'

Hello.
I wasn't able to reproduce the error. However, the issue comes from the filename, which does not match the module name.
Does it resolve the problem if you just add an empty module at the end of the file?
module ApplicatorPatch;end

The latest version of the plugin now address this issue. Thank you for bringing it to my attention.


Migrate to minitest

fixes #316

@seuros I'd like to take up this task and continue the minitest migration.

@alagram Cool. Looking forward your contribution. Let me know if you need assistance.
I will merge this PR so you can open a PR against the primary branch.

@alagram Cool. Looking forward your contribution. Let me know if you need assistance.
I will merge this PR so you can open a PR against the primary branch.

Will do, thank you!

Open smaller PRs, it will be better for review.
On Fri, Aug 5, 2022 at 3:34 PM Albert Agram @.***>
wrote:

@alagram https://github.com/alagram Cool. Looking forward your
contribution. Let me know if you need assistance.
I will merge this PR so you can open a PR against the primary branch.
Will do, thank you!
—
Reply to this email directly, view it on GitHub
https://github.com/ClosureTree/closure_tree/pull/403#issuecomment-1206526041,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASIUT722TCJIMB6ZBA4NCLVXUQ6HANCNFSM55WFX4GQ
.
You are receiving this because you modified the open/close state.Message
ID: @.***>


Warning: Accessing PropTypes via the main React package is deprecated. Use the prop-types package from npm instead

I migrated react-tabs-redux to the latest version (1.5.0) from v1.4.1 but I'm still getting this warning.

Hi @rahul-desai3, sorry for a late response. Would you like to send a PR fixing this? If not, I'll have some spare time next week.

I'll merged the PR, I'll review and publish possibly tomorrow.


[Bugfix][FE 318] pronounciation guide

Added word guide

How about the LanguageForm too?

How about the LanguageForm too?

You're right.  Changed...not sure about the placeholder though? placeholder="Enter the Pronunciation Guide of Public Name"

Hmm yeah that looks weird. Maybe keep the placeholder how it was.

Hmm yeah that looks weird. Maybe keep the placeholder how it was.

changed


Using with firebase auth flow

I do not have any sample code  but it's more of a question or theory. On my app I already have a google flow login with firebase auth. But firebase does not provide a new accessToken after 1 hour. i want to use this lib so i can ge the new access token evey hour
So with a user already logged with the firebase auth flow in is there a way I can call a method in this library and get the new access token?

I'm not 100% sure if I understand your question, but generally this library is meant to handle OIDC login flows from start to finish. It's not very well suited to jump in halfway through a login process or specific situation. I recommend writing custom JavaScript for the scenario you describe, and skip using this library.

@jeroenheijmans Thanks for reply
Basically the whole auth firebase auth flow is in place using Google Provider Can your lib detect if a user is already logged in and get any account information from your lib
Will look into it

Based on the bit of information you've provided, I don't think this library will be useful to you.


Add free resources feature

In complex resources, there are tons of additional resources like iam policies and ... which have no cost but fill the skipped resources warning. It would be great if we can add a free resource concept.

@sinabakh at the moment we can do this with the NoCost field. See https://github.com/infracost/infracost/blob/110d91f8edde971b88967188b21d739bad4e0748/internal/providers/terraform/aws/ecs_service.go#L22-L27 for an example.
I'm happy for a better approach to be added if we want.

Seems solid, I was not aware of this feature. I'll check the skipped resources logic to see if sth is missing there.

@sinabakh I hope you don't mind me keeping this open. I've been thinking about this a little more and think we could have a much better/light-weight approach. We had one user have 89 out of 95 resources show as unsupported where most of them are free resources (IAM policies, security groups, etc).
I think we could very quickly build a string array of free terraform resource types and exclude them. What do you think?

@aliscott I think the general approach is so clean, but creating files and functions to return a single no cost component might take a while (although I think it might be cleaner). For now, we can work on some sort of tricks in our registry so we can provide an array of free resources.
It's a trade off between speed and clean code. I think we can choose speed for now, if you agree I will work on it today but it would be great if you can provide the resources that you are aware of which are free.

Yeah I think an array will be easier to maintain going forward.
Known free resources:

aws_iam_*
aws_security_group*
null_resource


doc: documentation page on installing Stone from sources

This PR resolves #22

Unfortunately, the PR seems to be created by ChatGPT without testing. I don't see a way for the project to be built with cargo build --release.

@dmirgaleev okay I'll redo it please


Comments

I recommend removing annotated parts such as
"      //Log.info(Arrays.toString(message.getWith().toArray()));
//msg = String.format(msg, );
//return msg;
" which are completely useless here.

Hi, thank you for advise.
8e89e2accbc8825dcf9bbefd93ec2b2d4b071606
eaaf092ae5ad1c7cb48a3ac0130f4d9d51d77a5c


How to connect with TLS?

I can't seem to establish a connection with TLS. I have generated my self-signed certificates with the following commands: https://gist.github.com/drsect0r/3e73d2d96e2da801354e
I have configured my server and client as detailed below and I'm able to execute commands and manage the docker client. I have imported the CA certificate in my docker host system.
server options
--tlsverify --tlscacert=ca.pem --tlscert=server-cert.pem --tlskey=server-key.pem -H tcp://<IP_ADDRESS>:2375 -H unix:///var/run/docker.sock

client
docker --tlsverify --tlscacert=ca.pem --tlscert=client-cert.pem --tlskey=client-key.pem -H=hostname:2375

The following code results in a stack trace and and docker log message remote error: unknown certificate authority
$client = new Docker\DockerClient([
    'remote_socket' => 'tcp://<IP_ADDRESS>:2376',
    'ssl' => true,
    'stream_context_options' => [
        'ssl' => [
            'cafile' => storage_path() . '/ssl/ca.pem',
            'local_cert' => storage_path() . '/ssl/client-cert.pem',
            'local_pk' => storage_path() . '/ssl/client-key.pem',
        ],
    ],
]);
$docker = new Docker\Docker($client);

"exception": {
    "code": 0,
    "message": "Cannot enable tls: ",
    "file": "/con/data/http/root/vendor/php-http/socket-client/src/Client.php",
    "line": 125,
    "trace": [
      {
        "file": "/con/data/http/root/vendor/php-http/socket-client/src/Client.php",
        "line": 86,
        "function": "createSocket",
        "class": "Http\Client\Socket\Client",
        "type": "->",
        "args": [
          {},
          "tcp://<IP_ADDRESS>:2376",
          true
        ]
      },
      ...
}

Had the same problem try
$context = stream_context_create([
            'ssl' => [
                'cafile' => storage_path() . '/ssl/ca.pem',
                'local_cert' => storage_path() . '/ssl/client-cert.pem',
                'local_pk' => storage_path() . '/ssl/client-key.pem',
                'verify_peer_name' => true
            ],
        ]);

        $client = new \Docker\DockerClient([
            'remote_socket' => 'tcp://<IP_ADDRESS>:2376',
            'stream_context' => $context,
            'ssl' => true
        ]);

        $this->client = new \Docker\Docker($client);

@krzaczek thanks for you quick reply! (I clearly did not have enough sleep; regarding my stream_context_create in an array) I still get the error remote error: unknown certificate authority

@krzaczek
This did the trick;
$context = stream_context_create([
    'ssl' => [
        'cafile' => storage_path() . '/ssl/ca.pem',
        'local_cert' => storage_path() . '/ssl/client-cert.pem',
        'local_pk' => storage_path() . '/ssl/client-key.pem',
        'verify_peer_name' => true,
        'allow_self_signed' => true,
    ],
]);

$client = new \Docker\DockerClient([
    'remote_socket' => 'tcp://docker:2376',
    'stream_context' => $context,
    'ssl' => true,
]);

$docker = new Docker\Docker($client);

Ok it's a allow_self_signed cert --- I missed that :)


chore: update changelog for v0.1.4

Proposed Changes

Update changelog to record all changes for publishing v0.1.4

@kikuomax this PR is now ready for review 😎

@wesdevpro Sorry for directly modifying your branch, but I updated the CHANGELOG according to my preference. My commit also covers a couple of missing changes. I opened a discussion #263 about the CHANGELOG guideline.


Add file, add folder do not work.

<EMAIL_ADDRESS>when  Add file, add folder
do not work
atom 0.183, 0.184
work
atom 0.182

I can confirm this as well. When used from the remote-ftp panel the prompt is empty where it would usually contain the base folder's name in it and typing does not show any text.
However Atom does seem to understand that text is being typed and whatever you do type is accepted and hitting enter does create a file of that name.
So it seems like there is something wrong with the display.
Atom 0.187.0
Theme: One Dark
Syntax Theme: Base16 Tomorrow Dark

This appears to be require-ftp's DOM:
<atom-panel class="modal overlay from-top">

        Enter the path for the new folder.
        <atom-text-editor class="editor mini" tabindex="-1" callattachhooks="true" data-grammar="text plain null-grammar" mini="" data-encoding="utf8">

        </atom-text-editor>

</atom-panel>

This appears to be the DOM of the standard new file dialog (which works):
<atom-panel class="modal overlay from-top">

        Enter the path for the new file.
        <atom-text-editor class="editor mini" tabindex="-1" callattachhooks="true" mini="" data-grammar="text plain null-grammar" data-encoding="utf8">
        </atom-text-editor>

</atom-panel>

Fixed in eb46f06911c402703e945b11c26fe21617984d9b


Control Panel Crashes on continuing process-shutdown-dialog

Soo... this took me a while to figure out why this is happening to me since it used to run just fine, but I finally got it (yay).
This is an issue that happens with a custom MariaDB (or MySQL) installation. I have my MariaDB set up as a service to start with Windows Boot, now when trying to start WPN-XM I get the "processes are already running" window, with the option to either shutdown the process(es) or continue without. When pressing continue the Control Panel crashes (standard Windows "Application has stopped working" window) and I can't get it to start without manually stopping the service.
This used to work just fine (even tho I wish there was a possibility to skip that dialog, since I know the process is running), but it won't work anymore. I can't say which version broke it, since I skipped a few, but I'm currently running 0.8.0 (Lite Installer).
I have none of the services set to start on control-panel start (just to see if it was an issue). I also reinstalled a few times with the same result. For now I'm using the start-wpnxm.bat (with mariadb and browser lines deleted) and it's working fine.

Thank you for reporting this. This issue came up before and was already fixed.
Sorry, that the new relase took so long, but i had to rewrite toolchain for builds on Travis-CI.
You find the new release here: https://github.com/WPN-XM/WPN-XM/releases/tag/v0.8.1
If you want to keep your v0.8.0 you might just drop a new ServerControlPanel in.
You find it over at https://github.com/WPN-XM/server-control-panel/releases/tag/0.8.1
The boxed zips are one-file executables - just unzip into the root folder.
Please give me a feedback, if the new SCP solved this issue.
If you encounter other problems, please keep me updated. I will address them asap.

Doesn't seem like it worked, but I'm a little confused since the Window Title and "About" button still says 0.8.0.
Aside of that, isn't this a full setup (I used WPNXM-0.8.1-Lite-Setup-php56-w64.exe)? Because after I reinstalled with it, there were a lot of files missing, including the actual control panel, nginx (only some of the conf files installed), mariadb (only my.ini installed) and php (only some bats and php.ini installed), even though the installer says it would need ~250MB of space...

OK I just used the boxed zip package and that version is working perfectly fine :) Thanks

Ok, so the initial issue is resolved in v0.8.1 of the server control panel.

Aside of that, isn't this a full setup (I used WPNXM-0.8.1-Lite-Setup-php56-w64.exe)?
I need to test the installer. Did you do a fresh install or installed on top of 0.8.0?

Ok, i tested WPNXM-0.8.1-Lite-Setup-php56-w64.exe - the installer works.


Order return dict for correct result parsing in Vue.JS

In RunCard.vue#L376 it's expected that the first part of the return value is always the jid so the links are generated correctly. But the dict isn't ordered so if you've more minions it could happen that the jid is the last entry in the dict.
This code sort the dict in reverse order so jid is always the first part of the dict.

could you please provide an example for when this issue occurs?
Cheers.


Some formatting of README

@justinmorant -- some minor formating. OK to merge?

@otoolep This looks good to me, go ahead and merge.  I don't have permissions to accept the PR.

@justinmorant -- can you drop me an e-mail? Address is on my profile.


Why 2 shaka-player versions are released? 2.5.x and 3.0.x?

Have you read the Tutorials?
Yes
Have you read the FAQ and checked for duplicate open issues?
Yes
What version of Shaka Player are you using?
3.0.4
Please ask your question
When I check the releases, always 2 vesions of player are released. One is 2.5.x and another is 3.0.4. Why there are 2 versions?

We are still maintaining the v2.5.x releases for old users who don't want to upgrade to v3.x yet.  We cherry-pick the important bug fixes to the old branch to keep the old behavior/API.  New users should use the highest version available.

ok thanks


Changes for making debug mode optional while creating Power VS client

This PR contains the changes for making debug mode optional while creating Power VS client

/cc @mkumatag

/retest

/test generate

/assign @mkumatag

@Karthik-K-N can you please squash all 4 commits

/approve


https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Objects/Object-oriented_JS#Using_the_create_method

Request type

[ ] Correction or update

I'm linking you to the section in the title. I just wonder...Why do you explain Object.create() instead of Object.assing(target,source) isn't the second one the one fitting the context?

пт, 21 авг. 2020 г., 6:09 Mister Nobody<EMAIL_ADDRESS>
Request type

Please close this issue, I accidentally submitted it without adding
any details
New documentation
Correction or update

I'm linking you to the section in the title. I just wonder...Why do you
explain Object.create() instead of Object.assing(target,source) isn't the
second one the one fitting the context?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mdn/sprints/issues/3623, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALKRH5Q62AFEQIYWU3ZKTC3SBXQP5ANCNFSM4QG2OOWQ
.


Fix spelling mistakes in descriptions

:+1:

Tests are failing - you need to run rake to rebuild the schemas and propagate the changes.

We try to avoid having merge commits in the history and rebase against master and force push instead, to keep a cleaner history. I admit I can't find this rule in written form, we should probably add it to the git styleguide...

I pushed up the merge and fixup commits to avoid having to force push the branch whilst it's under development/review. I'll rebase before merging if the changes are approved.

Ok, fine by me then!


Create 0.0 Clear Daily Pop-up

To Go infront of Step 1

full ML script in standalone folder is ready for production and tested.
@kastyr1984 please convert the section scripts to OldNox format so they can be manually tweaked as necessary. I've sent you the last 10 lines in old syntax, which once you've converted 8.Exit to old syntax, you'll be able to marry up the lines with the additional bits i've manually adjusted.

Added Monotonous Task Script - Egg Summon - Tasks to hit master branch:
Convert to Old Syntax
Copy & Convert to Memu


iOS端导入后运行导致APPIcon变成默认的flutter图标

iOS端设置了APP图标，导入flutter_bdface_collect: ^1.1.1后运行，图标会变成默认的flutter图标，去掉后又恢复回来，摸不清楚原因
以下是Podfile
# Uncomment this line to define a global platform for your project
platform :ios, '9.0'

install! 'cocoapods', :disable_input_output_paths => true
# CocoaPods analytics sends network stats synchronously affecting flutter build latency.
ENV['COCOAPODS_DISABLE_STATS'] = 'true'

project 'Runner', {
  'Debug' => :debug,
  'Profile' => :release,
  'Release' => :release,
}

def flutter_root
  generated_xcode_build_settings_path = File.expand_path(File.join('..', 'Flutter', 'Generated.xcconfig'), __FILE__)
  unless File.exist?(generated_xcode_build_settings_path)
    raise "#{generated_xcode_build_settings_path} must exist. If you're running pod install manually, make sure flutter pub get is executed first"
  end

  File.foreach(generated_xcode_build_settings_path) do |line|
    matches = line.match(/FLUTTER_ROOT\=(.*)/)
    return matches[1].strip if matches
  end
  raise "FLUTTER_ROOT not found in #{generated_xcode_build_settings_path}. Try deleting Generated.xcconfig, then run flutter pub get"
end

require File.expand_path(File.join('packages', 'flutter_tools', 'bin', 'podhelper'), flutter_root)

flutter_ios_podfile_setup

target 'Runner' do
  use_frameworks!
  use_modular_headers!

  flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))

  pod 'AMapLocation','~>2.8.0'
end

post_install do |installer|
  installer.pods_project.targets.each do |target|
    flutter_additional_ios_build_settings(target)
    target.build_configurations.each do |config|
      config.build_settings['GCC_PREPROCESSOR_DEFINITIONS'] ||= [
        '$(inherited)',

        # dart: PermissionGroup.camera
        'PERMISSION_CAMERA=1',

        # dart: PermissionGroup.photos
        'PERMISSION_PHOTOS=1',

        # dart: [PermissionGroup.location, PermissionGroup.locationAlways, PermissionGroup.locationWhenInUse]
        'PERMISSION_LOCATION=1',
      ]

    end
  end
end

环境如下
[✓] Flutter (Channel stable, 2.8.1, on macOS 12.1 21C52 darwin-x64, locale zh-Hans-CN)
[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 13.2)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.1)

已解决，参考https://www.jianshu.com/p/1e46bf6c48eb
建议手动修改iOS项目AppIcon默认名称

#7


[DEFECT] RampFreq test not being run

Defect Description
Describe the defect
What did you expect to see happen?
The RampFreq test (HERON/tests/integration_tests/mechanics/ramp_freq) should be run.
What did you see instead?
The test is not being run by run_tests.
Do you have a suggested fix for the development team?
The cause is a lack of a tests file in the ramp_freq directory. Such a file needs to be added.
Describe how to Reproduce
Steps to reproduce the behavior:

Run HERON tests

Screenshots and Input Files
Please attach the input file(s) that generate this error.  The simpler the input, the faster we can find the issue.
Platform (please complete the following information):

OS: [e.g. iOS]
Version: [e.g. 22]
Dependencies Installation: [CONDA or PIP]

For Change Control Board: Issue Review
This review should occur before any development is performed as a response to this issue.

[x] 1. Is it tagged with a type: defect or task?
[x] 2. Is it tagged with a priority: critical, normal or minor?
[x] 3. If it will impact requirements or requirements tests, is it tagged with requirements?
[x] 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
[x] 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure
This review should occur when the issue is imminently going to be closed.

[x] 1. If the issue is a defect, is the defect fixed?
[x] 2. If the issue is a defect, is the defect tested for in the regression test system?  (If not explain why not.)
[x] 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
[x] 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
[x] 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

Approved for closure via #385.


patterns for managing transaction restart errors

(Originally noted in https://github.com/yonidavidson/cockroach-ent-example/issues/7)
Cockroach, like all databases which operate at high isolation levels, are liable to encounter serializable restart errors. This is a deep area and there's a lot written about it in the cockroach docs. In general, for go libraries, we recommend use of https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb to handle restarts and transactions. It appears that that library does not compose well with ent's generated client.
In addition to just having a loop, it's worth noting that there's some special handling to make the retries even less prone to starvation: see https://www.cockroachlabs.com/docs/v21.2/advanced-client-side-transaction-retries.
I think that ideally, there'd be a method on client called InTx or WithTx (🚲🏠) which took a closure and then dealt with retries for you. It's possible that we'll want dialect level control on how exactly the dance is managed given the above point.
FWIW, this is not a cockroach-specific consideration. You'll hit this with postgres when run under SERIALIZABLE at actually a higher rate than with cockroach and many other database also run into the same issues. Hopefully I don't have to preach the gospel of strong isolation, but if you need some inspiration: https://blog.acolyer.org/2017/08/07/acidrain-concurrency-related-attacks-on-database-backed-web-applications/.
Happy to collaborate to reach a good solution!

Hi @ajwerner thanks for opening an issue.
If you feel like adding this I can support you in the contribution part.
@a8m, what do you think about the API required for managing transaction retries?

I'm trying to find a solution to this issue and how to implement it. I wonder whether this retry behavior could be implemented as a transaction hook -- in tx.OnCommit catch errors originating from next.Commit and retrying with exponential backoff as appropriate?
Clearly a generic solution that can be set once instead of explicitly applied to every transaction would be preferable.

Exponential backoff is not what is always desirable. In cockroachdb, for example, locks are held across retries in order to avoid starvation. Backing off would induce unnecessary latency and prevent optimal utilization. In systems which are fully optimistic or which drop locks across restarts, backoff with jitter might indeed be necessary to ensure liveness.

Thanks for the heads up. You might want to update the docs then: https://www.cockroachlabs.com/docs/v21.2/transactions#client-side-intervention-example


fix: adapter bump

:tada: This PR is included in version 1.4.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


batsignal: 1.2.0 -> 1.3.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/batsignal/versions.
meta.description for batsignal is: "Lightweight battery daemon written in C"
meta.homepage for batsignal is: "https://github.com/electrickite/batsignal"
meta.changelog for batsignal is: ""
Updates performed

Version update

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
/nix/store/y3y9pf1gz741pylp1bbfiz2l90pqa549-batsignal-1.3.1/bin/batsignal passed the binary check.
1 of 1 passed binary check by having a zero exit code.
1 of 1 passed binary check by having the new version present in output.
found 1.3.1 with grep in /nix/store/y3y9pf1gz741pylp1bbfiz2l90pqa549-batsignal-1.3.1
found 1.3.1 in filename of file in /nix/store/y3y9pf1gz741pylp1bbfiz2l90pqa549-batsignal-1.3.1

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
batsignal
docbookrx

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/y3y9pf1gz741pylp1bbfiz2l90pqa549-batsignal-1.3.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A batsignal https://github.com/r-ryantm/nixpkgs/archive/688235fb07757c7b17e2f58f069f3055351ccebd.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/y3y9pf1gz741pylp1bbfiz2l90pqa549-batsignal-1.3.1
ls -la /nix/store/y3y9pf1gz741pylp1bbfiz2l90pqa549-batsignal-1.3.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

batsignal

Maintainer pings
cc @SlothOfAnarchy for testing.

Result of nixpkgs-review pr 150375 run on x86_64-linux 1

1 package built:

batsignal


Upload token logos from django admin panel

What is needed?
Upload token logos from django admin panel as we are doing with contracts

Closed by https://github.com/gnosis/safe-transaction-service/pull/664


Add support for Gdk

I don't think there is a way to get nuget to detect this correctly, but at least you can make it work by manually editing your .csproj.

I'm mostly into this, a few things:

Can you get rid of the extra SLN file and just put it into Splat-XamarinStudio.sln
Can you ditch MoreSplat and make a Splat.GtkPlus and Splat.MonoMac.OSS?

I actually didn't mean to add all of these commits to this pull request, but I'm glad you are liking it.

Yes, I can do that.
I'm still figuring out how nuget works and have realized that this is the way to go. The GTK/GDK stuff actually only works with GTK#3, so I will name it Splat.GtkSharp3 instead.

I've straightened things out a bit, so now we have 3 new Nuget packages: Splat.MonoMac.OSS, Splat.GtkSharp and Splat.GtkSharp3. I added an Extra/ subdirectory under Splat/bin so that they won't interfere with the existing nuspec.

I should mention that in order to get the gtk-sharp-3 assemblies, you will have to do a little work. See http://stackoverflow.com/a/29184364/1976323.


a security issue

laravel-admin background XSS
first.add xss Roles

next go to operation log page

The following is my system environment

And I think this issue also exists in other versions.And don't know this place.

有漏洞不解决？

有漏洞不解决？


Commits are not boxed in live view

In the live view (in contrast to viewing a log for a specific date), geths commit announcements aren't put in a block.

Yeah... that's a bit... troublesome... still trying to figure out a way to handle "incomplete" commit messages (or test-t messages for that matter).  Especially when other messages can intersperse them.

No worries. Just wanted to mention it so it's known to be known.
Also: Thank you very much for your work on the log archiver and viewer. I am really impressed by the result!

The live view will now apply the boxing of commit and test-t messages, and scrolldown also works correctly now.   Scrollup doesn't do that correctly yet, working on that.


Expected (desired) behavior when storing data in a spreadsheet

For now, when we store data to a spreadsheet, we update the overlapped data but not remove the existing one. What do I mean by that? Suppose I have a todo list that I saved to a spreadsheet, and it became the data source for my app. The next day I remove all the items from the list and save the list again. I expect that all the data were erased from the spreadsheet, but they are not since we have no data to store after deleting everything from the app. We envoke Google Sheets API without providing any data, and the spreadsheet remains untouched.
Another case when I deleted a couple of todos, updated some of the remaining, and saved them. After refreshing the page, I get the mix of updated data and the ones that were "deleted" since, in reality, we delete nothing but only update the existing data that are overlapped with the new one.
Another thing to consider is that there might be other data on a sheet (e.g., for/from another app). So we can't simply, for example, delete the sheet with the old data and create another for the new one.
The described situation is not a problem for apps where data is not deleted but only updates and adds.
@leaverou, @karger
Will you please share your thoughts on the expected (desired) behavior for the GSheets plugin.

Firstly, data should definitely be deleted from the sheet when it's deleted in the app.
However, if the app behaves essentially like pasting cells, it sounds like if you add rows it would overwrite anything that's underneath? E.g. if I have a table of 200 rows, and I specify mv-storage-range="A1:C100", what happens when I add rows?
Is it possible to delete/add cells and have the other cells be shifted up/down? You can do it in the UI, I'm not sure if the API allows it.

Verifying: if the storage is specified as a range of columns, to let me
manage a collection of arbitrarily many items, then when I write, I
should effectively write all the rows.  ie, if the number of items
decreases, I should write blanks into the now-empty rows.
followup: can I specify rectangular ranges, e.g. a1:C100?  and what are
the consequences?  Is this asserting that the collection is not allowed
to have more than 100 items?  This doesn't seem useful on its own, but
perhaps it is important if the mavo is only one part of a larger
spreadsheet context.
On 12/11/2020 3:35 PM, Lea Verou wrote:

Firstly, data should definitely be deleted from the sheet when it's
deleted in the app.
However, if the app behaves essentially like pasting cells, it sounds
like if you /add/ rows it would overwrite anything that's underneath?
E.g. if I have a table of 200 rows, and I specify
|mv-storage-range="A1:C100"|, what happens when I add rows?
Is it possible to delete/add cells and have the other cells be shifted
up/down? You can do it in the UI, I'm not sure if the API allows it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/DmitrySharabin/mavo-gsheets/issues/1#issuecomment-743413677,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIWSXTQKJR55BUVMSZQO3DSUJ7BFANCNFSM4UW6OVZA.

Thank you very much for your thoughts and questions.
I implemented data deletion. So now, the data-saving works as it should.

E.g. if I have a table of 200 rows, and I specify mv-storage-range="A1:C100", what happens when I add rows?

can I specify rectangular ranges, e.g. a1:C100?  and what are the consequences?  Is this asserting that the collection is not allowed to have more than 100 items?

When we work with rectangular ranges like A1:C100, we are limited by this exact range. That means if we try to save data that has 101 rows and/or 4 columns, we won't be able to do that—the Sheets API will block this operation.
Unfortunately, since the write request comes after the clear request, the data in the A1:C100 range will be cleared. Luckily, we have the versions in the Google Sheets so we can recover the needed data. And yes, I know that's not the way it should work. 😔
I'll try to find a way to solve this issue and perform the clear and write requests in one "transaction," so data is written only if both operations succeed. However, I am not sure whether it is possible (I searched for it but couldn't find it yet).

Is it possible to delete/add cells and have the other cells be shifted up/down?

Unfortunately, the API doesn't allow that. We can append data to a spreadsheet by inserting new rows. But the algorithm used in this operation can break data we already have in the spreadsheet and are not used by either mavo app. And not only because the append operation will insert the whole rows.
In apps, we can use adjacent ranges separately, and data in them won't interfere if we specify ranges. But the algorithm used by the append operation tries to find a place where to insert data analyzing the whole sheet, not only the specified range, so the adjacent ranges will be interpreted as they belong to one table. And the insertion point will be found incorrectly. And that may cause data loss.

And one more thing concerning ranges. If we use ranges other than rectangular, we are not limited by sheet bounds: if there are not enough rows/columns to store data, they will be added automatically. So we can start with a sheet with only one cell. 😎

Unfortunately, since the write request comes after the clear request, the data in the A1:C100 range will be cleared. Luckily, we have the versions in the Google Sheets so we can recover the needed data. And yes, I know that's not the way it should work. 😔
I'll try to find a way to solve this issue and perform the clear and write requests in one "transaction," so data is written only if both operations succeed. However, I am not sure whether it is possible (I searched for it but couldn't find it yet).

Not an issue anymore. I believe I could found the solution.

Unfortunately, the API doesn't allow that.

Actually, I'm mistaken. The Google Sheets API allows us to perform almost any action we need. Of course, it might not be very straightforward, but still. I'm going to leave this here as a reference for future use: https://developers.google.com/sheets/api/reference/rest.


Implement a tofino://newtab page that links to the history, starred and saved pages

Part of https://github.com/mozilla/tofino/issues/518


Search release tags without case sensitivity

Code is here

Fixed in fork: https://github.com/edenlabllc/ci-utils/blob/0b4549c73af94f3186c7bd610c6cf5026e40781f/fetch-source-version.sh#L16


Dto如果有枚举的时候就会生成不出来

        "WorkflowInstanceStatusCountDto": {
            "type": "object",
            "properties": {
                "state": {
                    "type": "string",
                    "enum": [
                        "WorkflowExecutionStatus{code=0, desc='submit success'}",
                        "WorkflowExecutionStatus{code=1, desc='running'}",
                        "WorkflowExecutionStatus{code=2, desc='ready pause'}",
                        "WorkflowExecutionStatus{code=3, desc='pause'}",
                        "WorkflowExecutionStatus{code=4, desc='ready stop'}",
                        "WorkflowExecutionStatus{code=5, desc='stop'}",
                        "WorkflowExecutionStatus{code=6, desc='failure'}",
                        "WorkflowExecutionStatus{code=7, desc='success'}",
                        "WorkflowExecutionStatus{code=12, desc='delay execution'}",
                        "WorkflowExecutionStatus{code=14, desc='serial wait'}",
                        "WorkflowExecutionStatus{code=15, desc='ready block'}",
                        "WorkflowExecutionStatus{code=16, desc='block'}",
                        "WorkflowExecutionStatus{code=17, desc='wait to run'}"
                    ]
                },
                "count": {
                    "type": "integer",
                    "format": "int32"
                }
            }
        },

        "WorkflowInstanceStatusCountDto": {
            "type": "object",
            "properties": {
                "state": {
                    "type": "string",
                    "enum": [
                        "WorkflowExecutionStatus{code=0, desc='submit success'}",
                        "WorkflowExecutionStatus{code=1, desc='running'}",
                        "WorkflowExecutionStatus{code=2, desc='ready pause'}",
                        "WorkflowExecutionStatus{code=3, desc='pause'}",
                        "WorkflowExecutionStatus{code=4, desc='ready stop'}",
                        "WorkflowExecutionStatus{code=5, desc='stop'}",
                        "WorkflowExecutionStatus{code=6, desc='failure'}",
                        "WorkflowExecutionStatus{code=7, desc='success'}",
                        "WorkflowExecutionStatus{code=12, desc='delay execution'}",
                        "WorkflowExecutionStatus{code=14, desc='serial wait'}",
                        "WorkflowExecutionStatus{code=15, desc='ready block'}",
                        "WorkflowExecutionStatus{code=16, desc='block'}",
                        "WorkflowExecutionStatus{code=17, desc='wait to run'}"
                    ]
                },
                "count": {
                    "type": "integer",
                    "format": "int32"
                }
            }
        },

在你提供的 enum 部分，枚举值的格式看起来不像是典型的简单字符串集合。在标准的OpenAPI规范中，枚举通常只包含简单的字符串或数字，而不是带有描述性信息的复杂对象。标准的OpenAPI枚举应该如下所示（以字符串为例）：


DEPRECATION WARNING: The InstanceMethods module inside ActiveSupport::Concern will be no longer included automatically.

I get the following message when run rspec:
DEPRECATION WARNING: The InstanceMethods module inside ActiveSupport::Concern will be no longer included automatically. Please define instance methods directly in Address instead. (called from include at /Users/ik/Development/projects/booking/webapp/app/models/address.rb:3)
and line 3 of address.rb is include Mongoid::Spacial::Document
The same issue was fixed in mongoid :
https://github.com/mongoid/mongoid/issues/1525

+1

seems it was fixed in this commit - https://github.com/ryanong/mongoid_spacial/commit/1a4841d94dc8d1e432308e648ee1874fc4748cb8


created a component to show contributers to a github project

This PR fixes #20
Created a custom component to show the list of contributers to a particular github project using github APIs. The project whose list is needed can simply be passed as a prop to the component and it will fetch and render all the data.

@wdevon99  @charithccmc  could you please this PR?

SASS also supports vanilla css styling I guess. However, I will just update it.
Thanks for reviewing @wdevon99

@wdevon99 I have updated the correct syntax, would you please review it now?

Already done in the GitHub APIs PR :)


Errors on julia 1.11 due to changes in AbstractTriangular type parameters

Failed to precompile QuantumStateBase [73ce9c4f-35d1-4161-b9e6-26915895bfed] to "/home/pkgeval/.julia/compiled/v1.11/QuantumStateBase/jl_0DZxw9".
ERROR: LoadError: too many parameters for type AbstractTriangular
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/ArrayLayouts/4IG3b/src/memorylayout.jl:712
 [2] include(mod::Module, _path::String)
   @ Base ./Base.jl:556
 [3] include(x::String)
   @ ArrayLayouts ~/.julia/packages/ArrayLayouts/4IG3b/src/ArrayLayouts.jl:1

Ref https://github.com/JuliaLang/julia/pull/26307

The package is using an outdated version of ArrayLayouts (locally, it appears to be capped to v0.8.18). The recent versions of ArrayLayouts don't face this issue.

Thanks, I should have checked that.


[main] Upgrade to latest dependencies

1.8 release -lionelvillard
/cc knative-sandbox/eventing-writers
/assign knative-sandbox/eventing-writers
Produced by: knative-sandbox/knobots/actions/update-deps

/lgtm
/approve


Control GPO across WAN from HA with Thread router board or Zigbee2MQTT using the Zigbee gateway example.  (TZ-551)

Question
I would like to know if it would be possible to create an example of the Zigbee Gateway using the thread router board talking to Home Assistant. The use case is to control a Zigbee GPO (power point) across a WAN from HA. I currently have something setup using the HA Switch example on the C6 to control a commercial GPO and also using a S3 as a Zigbee2MQTT with a low level interface to the C6 board for on/off control. However I was wondering if it could be achieved with the Thread router board and the Gateway example talking to Home Assistant.
Additional context.
It doesn't neceessarily have to be the Gateway example either. Could even be the Zigbee2MQTT on the Thread router board talking over UART to the H2 to control the GPO. But the current problem I have is that the state is currently not being reported back when I control on or off to the GPO.

AFAIK, the zigpy is required to work with ZHA, we will have a ESP NCP solution with corresponding zigpy-xx to work with home assistant.
zigpy doesn't work on the ESP Thread BR board, but instead you need to implement the protocol between the gateway and home assistant (which was done in zigpy) on your own. Which I think should be possible, if you understand the detailed information of Home Assistant cloud interface.

@djorr5 Let us know if you have any follow up questions on this topic.


Airstage

is it possible to add support for the new app airstage from Fujitsu?
BG

Hi,
Sorry but I think that is a completely different system and that’s not what my unit uses.


amongusdiscord.exe 2.2.0 crashes after entering connect code in Among Us Capture

The bot is running (amongusdiscord.exe) and Among Us Capture is also running (amonguscapture-x64.exe). I use .au new in discord, then enter the provided connect code in Among Us Capture. amongusdiscord.exe immediately crashes after submitting the connect code. The log shows the following:
2020/09/30 11:51:44 Client connection closed:  client namespace disconnect
2020/09/30 11:51:44 meet error: runtime error: invalid memory address or nil pointer dereference

Same issue - it seemed to work once, but every subsequent try seems to crash it.

So the bot seems to be working now - I think it was related to not having all the emojis on the server.


Gradle TestKit adds Groovy to the beginning of the test class path

If you write a Plugin in any language and then want to test it with Spock 2.x in the Groovy 3 variant, I'd say this simple build script should be fine as a start:
plugins {
    groovy
}

repositories {
    mavenCentral()
}

dependencies {
    testImplementation("org.spockframework:spock-core:2.0-M4-groovy-3.0")
    testImplementation(gradleTestKit())
}

But no, it is not.
gradleTestKit() seems to add Groovy of the current distribution to the beginning of the test class path.
In case of Gradle 6.8.3 that is Groovy 2.5.12.
This will make compileTestGroovy task fail as Spock verifies it is compatible with the used Groovy version.
Is this intentional and you simply cannot use a newer Groovy version for your tests if you use the TestKit which forces you to 2.5 suddenly? (suddenly because you maybe wrote unit tests first with Groovy 3 and then add TestKit as a dependency)
Or is this more a bug?
With 7.0-milestone-3 it is the other way around.
The Groovy 3 variant works fine, but if I use the Groovy 2 variant, it complains, as in Gradle 7.0-m-3 Groovy is already updated to version 3.0.7.

Maybe you could try the redistribution of the Gradle API and TestKit jar from https://github.com/gradle-plugins/gradle-api. They are my best estimate to what a Gradle API/TestKit JARs as external dependencies should look like. It still have the dependency on Groovy/Kotlin pointing to the version packaged within the Gradle distribution but they are normal external dependencies which can be excluded or upgraded to the right version. For example, if you target Gradle API 6.8.3 you can use: dev.gradleplugins:gradle-api:6.8.3 and/or dev.gradleplugins:gradle-test-kit:6.8.3. The major downside to this approach is having to download an additional ~110mb. I'm open to fix any issue the redistributed Jar may have if you fine any.
If you don't want to use the these JARs, you could probably try to remove self-resolving dependency, but I believe test-kit includes directly the gradleApi so you would have to do some more evolved manipulation.


Add a PublicUrl to ApiResource .

I need to give some public information about my API to developers.
I think with having a property (e.g PublicUrl,...) I can do this.
Thanks

Can you elaborate?

Please look at google developer console. In there there are many APIs that we can add them to client .
The Apis have many information when we click on one of them .
With having some fields on ApiResource  I can link the developer to an internal or external page to get mode info about the api (e.g api price , api usage , ...) .
sorry for my bad english :)

I think this issue would provide you what you need: #2249. If so, then I'd close this as a dup.

This is duplicate of #2249
I close this and follow that.


drop-in directory

For installing, you say to cd to drop-in directory, but there's no drop-in directory?

Me, too


(test): add fake for GCE Metadata Server

This PR establishes a basic fake for a GCE Metadata server. At the moment, nothing currently uses this fake. The intention is to use this in VM testing to provide simulation of GCE environment for testing purposes. The initial use of this will be for e2e telemetry tests on VMs.
The Makefile and Dockerfile are provided to allow easy creation of new images when required. It is NOT expected that these will need to be used at any frequency.
/cc @justinwei2 @howardjohn
[ X ] Policies and Telemetry
[ X ] Test and Release

@douglas-reid: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
ce7e76b76cc9b527abcc6ef7cfcb941b6c9c54d7
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@douglas-reid: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
cb31791c3f10136835467c263d42de833955a308
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@douglas-reid: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
cb31791c3f10136835467c263d42de833955a308
link
/test unit-tests_istio

integ-multicluster-k8s-tests_istio
cb31791c3f10136835467c263d42de833955a308
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test unit-tests_istio
/test integ-multicluster-k8s-tests_istio

@douglas-reid: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-galley-k8s-tests_istio
0ce9ad20a44c231c8834930bde4f41e92c32ef43
link
/test integ-galley-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-galley-k8s-tests_istio


could not find native static library 'crypto', perhaps an -L flag is missing?

This error I'm receving when I tried to compile quiche tool given in the link https://github.com/cloudflare/quiche
The full error I'm getting is given below
Caused by:
process didn't exit successfully: rustc --crate-name quiche --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --crate-type staticlib --crate-type cdylib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="boringssl-vendored"' --cfg 'feature="default"' -C metadata=ec649e4481e8d95d --out-dir /root/quiche/target/debug/deps -C incremental=/root/quiche/target/debug/incremental -L dependency=/root/quiche/target/debug/deps --extern lazy_static=/root/quiche/target/debug/deps/liblazy_static-3e523cea53aa78b2.rlib --extern libc=/root/quiche/target/debug/deps/liblibc-3d02aff0893bcd1d.rlib --extern libm=/root/quiche/target/debug/deps/liblibm-770d78106c2380e5.rlib --extern log=/root/quiche/target/debug/deps/liblog-29d81ded01531039.rlib --extern ring=/root/quiche/target/debug/deps/libring-45d6332ced1f3119.rlib -L native=/root/quiche/deps/boringssl/build/crypto -L native=/root/quiche/deps/boringssl/build/ssl -l static=crypto -l static=ssl -L native=/root/quiche/target/debug/build/ring-cf06706f198ab4f5/out (exit code: 1)
I used cargo to build, I don't why it fails to fetch borginssl in /deps folder. Can anyone help me to diagonize it?

why did you closed this issue? Had you solved this problem?How to do it?


How can i send file from local file or url ?

hi
How can i send file from local file or url ?
my code

            $data = [
                'caption' => 'caption file',
                'chat_id' => $chat_id,
                'file'  => $this->telegram->getUploadPath() . "/" . $file_name
                // 'file' => 'https://ballooncrm.com/assets/img/img_backgroud_footer.png'
            ];

            return Request::sendDocument($data);

First off, you were using file instead of document as the key to set the file to be sent.
When sending a file from a local path, use Request::encodeFile(...);:
return Request::sendDocument([
    'caption'  => 'caption file',
    'chat_id'  => $chat_id,
    'document' => Request::encodeFile($this->telegram->getUploadPath() . '/' . $file_name),
]);

Take a look at the official Telegram docs about sending files here:
https://core.telegram.org/bots/api#sending-files
You'll see that only certain file types can be sent with sendDocument. To send any type, either just download it to your server first and then send it the local way with Request::encodeFile(...):
$file_url  = 'https://ballooncrm.com/assets/img/img_backgroud_footer.png';
$file_name = 'balloon.png';
$file_path = $this->getTelegram()->getUploadPath() . '/' . $file_name;

// First download the file locally.
file_put_contents($file_path, fopen($file_url, 'rb'));

return Request::sendDocument([
    'caption'  => 'caption file',
    'chat_id'  => $chat_id,
    'document' => Request::encodeFile($file_path),
]);

or just stream it from the URL directly:
$file_url = 'https://ballooncrm.com/assets/img/img_backgroud_footer.png';

return Request::sendDocument([
    'caption'  => 'caption file',
    'chat_id'  => $this->getMessage()->getChat()->getId(),
    'document' => fopen($file_url, 'rb'),
]);

@irvatoDev Did you manage to get it working for you? Can we close off here?

i have file id, what i am doing ?

@enghelab If you have the file ID ($file_id), you can use it directly, like this:
return Request::sendDocument([
    'caption'  => 'caption file',
    'chat_id'  => $this->getMessage()->getChat()->getId(),
    'document' => $file_id,
]);


take advantage of docker for install

I just released a docker image for sqitch:pgsql.
I wanted to let you know: https://registry.hub.docker.com/u/docteurklein/sqitch/
Now using sqitch is as simple as this one liner:
docker run --rm docteurklein/sqitch:pgsql

Oh sweet! I need to try Docker…

Added it to the home page in 933905a. Thanks!


moto.mock_dynamodb2 treats not equals as equals in filter expressions

when constructing a not equals filter expression like this:
Attr('fieldname').ne('some value')

moto appears to be treating as eq (equals)

How to duplicate

See test script below

What you expect to happen

The script should print out 3 scans of the table with sample data.

Non filtered scan -- all 3 records
filter with providerName eq aws -- 1 record, where providerName eq aws
filter with providerName ne aws -- 2 records, where providerName ne aws

What actually happens

Script prints out 3 scans of the table with sample data

Non filtered scan -- all 3 records
filter with providerName eq aws -- 1 record, where providerName eq aws
filter with providerName ne aws -- 1 record, where providerName eq aws

This demonstrates the reported bug

Version Info. I've tested on the following

python 2.7.15 with moto 1.1.25 and boto3 1.7.40 installed via pip
python 2.7.15 with moto 1.3.3 and boto3 1.7.66 installed via pip
python 3.6.5 with moto 1.3.3 and boto3 1.7.66 installed via pip

test script
import boto3
from boto3.dynamodb.conditions import Attr
from moto import mock_dynamodb2
import os
from pprint import pprint

# set aws env vars
os.environ['AWS_DEFAULT_REGION'] = 'us-east-1'

# start mock
mock = mock_dynamodb2()
mock.start()

# vars and objects
table_object = boto3.resource('dynamodb')
table_name = 'test_table'
data = [
    {
        'id': 1,
        'providerName': 'aws'
    }, {
        'id': 2,
        'providerName': 'azure'
    }, {
        'id': 3,
        'providerName': 'google'
    }
]

# create table
create_table = table_object.create_table(
    TableName=table_name,
    KeySchema=[
        {
            'AttributeName': 'id',
            'KeyType': 'HASH'
        }
    ],
    AttributeDefinitions=[
        {
            'AttributeName': 'id',
            'AttributeType': 'N'
        }
    ],
    ProvisionedThroughput={
        'ReadCapacityUnits': 1,
        'WriteCapacityUnits': 5
    }
)

# instantiate boto/moto table object
table = table_object.Table(table_name)

# batch write data to table
with table.batch_writer() as batch:
    for item in data:
        batch.put_item(Item=item)

# print all data
print('\n\n\nALL ITEMS')
pprint(table.scan())

# filter providerName eq aws
filter = Attr('providerName').eq('aws')
print('\n\n\nPROVIDER NAME EQ aws')
pprint(table.scan(FilterExpression=filter))

# filter providerName ne aws
filter = Attr('providerName').ne('aws')
print('\n\n\nPROVIDER NAME NE aws')
pprint(table.scan(FilterExpression=filter))

running into this as well.  Kinda breaking the tests

I don't think we've implemented this yet. Setting as a feature request. Thanks!

To clarify, this isn't an enhancement, it's a bug.  It is implemented but treats not_equals as equals.

Closing this, as the associated PR has been merged


Add Visual Studio Code extension link

Looks good to me. Feel free to put the repo in the zig-lang organization if you want.


[4316] Add backfill for hesa_updated_at on trainees which handles leading zeros on hesa_id

Context
Trello ticket: https://trello.com/c/UW954uIW/4316-s-backfill-the-hesaupdatedat-field-from-the-hesastudents-table-to-the-trainees-table
The original backfill that we merged for this piece of work failed to backfill some of the hesa_updated_at for some trainees. This is because some trainee hesa_ids have leading zeros in the trainees table, but not in the hesa_students table.
In this ticket we format the trainee.hesa_id to remove leading zeros and then match to the hesa_students table.
The scoped trainees is about 31568. I'll run on DTTP import to see how many we manage to update and some of these are not present in the hesa_students table.
Changes proposed in this pull request

Add backfill to target trainees with leading zeros in the hesa id

Guidance to review

I'll run the backfill on dttp import to see how it performs
Check my logic in format_hesa_id(trainees) as it is not my intention to update the DB field
Check nil handling is covered

Important business

[ ] Does this PR introduce any PII fields that need to be overwritten or deleted in db/scripts/sanitise.sql?
[ ] Does this PR change the database schema? If so, have you updated the config/analytics.yml file and considered whether you need to send 'import_entity' events?

NB: Please notify the #twd_data_insights team and ask for a review if new fields are being added to analytics.yml

 [S] Backfill the hesa_updated_at field from the hesa_students table to the trainees table


Masking passed to the backend RNN calls.

This has been discussed in a number of places, #1282 #1300 #1206 #1224.
@fchollet proposed this:

What are your thoughts on the following:

handling masking via an explicit mask array being passed around
rnn returning the last valid output for any masked step (in much the same way that it currently returns as states the last valid states). For instance, for a timeseries [1., 2., 3., masked, masked], an accumulator RNN would return [1., 3., 6., 6., 6.].

This turned out to be much simpler than I originally expected, due to
the nicely componentized design of the new RNN backend. Nice job!
This currently works with the Theano backend, and I've included new tests that check with both left- and right-padding.
There's still an issue with TensorFlow, so don't merge yet. I wanted to get some feedback on the API and the Theano approach.

        Args:
          target_list: A list of byte arrays corresponding to names of tensors
            or operations to be run to, but not fetched.
          fetch_list: A list of byte arrays corresponding to names of tensors to
            be fetched and operations to be run.
          feed_dict: A dictionary that maps tensor names (as byte arrays) to
            numpy ndarrays.

        Returns:
          A list of numpy ndarrays, corresponding to the elements of
          `fetch_list`.  If the ith element of `fetch_list` contains the
          name of an operation, the first Tensor output of that operation
          will be returned for that element.
        """
      try:
        # Ensure any changes to the graph are reflected in the runtime.
        with self._extend_lock:
          if self._graph.version > self._current_version:
            graph_def = self._graph.as_graph_def(
                from_version=self._current_version)

            try:
              status = tf_session.TF_NewStatus()
              tf_session.TF_ExtendGraph(
                  self._session, graph_def.SerializeToString(), status)
              if tf_session.TF_GetCode(status) != 0:
                raise RuntimeError(compat.as_text(tf_session.TF_Message(status)))
              self._opened = True
            finally:
              tf_session.TF_DeleteStatus(status)

            self._current_version = self._graph.version

        return tf_session.TF_Run(self._session, feed_dict, fetch_list,
>                                target_list)
E                                SystemError: <built-in function delete_Status> returned a result with an error set

So [1,2,3,pad0,pad0] gives hidden  states [1,3,6,6,6].. now i think we could should multiply this with the mask and produce the final output [1,3,6,pad0,pad0].. (I recommend this even if the mask is also available). This would be helpful and make sense for advanced users who extract features from mid layers.. we dont want our pad values to be dependent on the last input. [which is 6 in this case]..let it be always zero.

@wxs @fchollet TensorFlow is a pain now.. whenever I try to write something that is not very straightforward its all like if theano then this elif tensorflow then that..TensorFlow is from Google and they have a cool website. But its still a baby.

Ah also I had only been running the test_recurrent.py tests, look like some other stuff broke.

So [1,2,3,pad0,pad0] gives hidden states [1,3,6,6,6].. now i think we could should multiply this with the mask and produce the final output [1,3,6,pad0,pad0].. (I recommend this even if the mask is also available). This would be helpful and make sense for advanced users who extract features from mid layers.. we dont want our pad values to be dependent on the last input. [which is 6 in this case]..let it be always zero.

This would be straightforward to do, I would remove this line: output = broadcasting_select(mask_t, output, states[0]). However I think it makes more sense to keep it this way. Otherwise it becomes more complicated when you have right-padding and return_sequences=False, you'd need to specially extract an old output.
If advanced users need access to the middle layers, they can just also access the mask. Or, if you use left-padding you'll get all 0s anyway.

OK I've fixed the other Theano test errors. The only thing left to fix is TensorFlow, and I think we're pretty close. If anyone has any ideas about the above error let me know, otherwise I should be able to keep looking at this soon.

OK some investigation into the issue. It appears that sometimes TensorFlow is having trouble inferring dimensions in places where it seems like it should be able to. If in tensorflow_backend.py I modify it thus:
    inputs = tf.transpose(inputs, (1, 0, 2))
    input_list = tf.unpack(inputs)
    if mask is None:
        print("AAAA")
        mask = ones_like(tf.slice(inputs, [0,0,0],[-1, -1,1]))
        print("Input shape[0] %s" % inputs.get_shape()[0].value)
        print("Mask shape[0] %s" % mask.get_shape()[0].value)
    else:
        print("BBBB")
        mask = tf.transpose(mask, (1, 0, 2))

then for some tests, e.g. test_temporal_classification of test_tasks.py. I get the following output:
Input shape[0] 3
Mask shape[0] None

even though it should match the length of the first dimension of input. This happens in
However if I make the following simple little standalone tensorflow script it does correctly infer the first dimension of mask:
import tensorflow as tf
inputs = tf.constant(3, shape=[2,3,4])
mask = tf.ones_like(tf.slice(inputs, [0,0,0],[-1, -1,1]))

print("Input shape[0] %s" % inputs.get_shape()[0].value)
print("Mask shape[0] %s" % mask.get_shape()[0].value)

>> Input shape[0] 2
>> Mask shape[0] 2

This feels like a TensorFlow bug to me, since it should clearly be able to infer the mask's shape in the example where it fails. Still investigating whether I can force it via set_shape() or something. Any ideas welcome.

OK I've solved that issue with set_shape() but if someone has a better idea I'm open to it. Continuing to investigate other cryptic issues.

Another point of discussion. There's this interesting design that now separates the output variable from the list of state variables. However given this design requirement:

rnn returning the last valid output for any masked step (in much the same way that it currently returns as states the last valid states). For instance, for a timeseries [1., 2., 3., masked, masked], an accumulator RNN would return [1., 3., 6., 6., 6.]."

I need to be able to access the preivous timestep's output. I've been running on the assumption that the first variable of the states matches the output, (see this
output = T.switch(mask, output, states[0]) # output previous output if masked.

Because that's the case with all the recurrent layers as currently built, however I think that's a poor choice on my part because you could easily design a recurrent layer that didn't follow this pattern. In fact, I ran up against this with the TimeDistributedDense layer which happens to be implemented as a recurrent layer but passes no state between scan calls, so I had to make this alternate line for that case:
output = T.switch(mask, output, zeros_like(output))

My question for readers of this PR (@fchollet). What is the desired initial output, that is if the first timestep is masked, what should its output be? I'm inclined to say all 0s (which is the special case I already built) or I could make it a user-parameter similar to the initial_states parameter. Either way I'm going to have to modify the signature of the backend rnn call because it doesn't currently know the output shape until after the first call to scan so I can't provide an initializer.

What is the desired initial output, that is if the first timestep is masked, what should its output be? I'm inclined to say all 0s

All zeros is a very reasonable default.

I've been running on the assumption that the first variable of the states matches the output

While generally the case, we shouldn't rely on it. In the Theano case, the previous output would be part of the variables passed to the step function in scan, while in TF, we have explicit access to the latest state.

Either way I'm going to have to modify the signature of the backend rnn call

We should avoid modifying the rnn API. You could simply run the step function on the first input then multiply the result by 0., giving you an all-zero output of the correct shape.

We should avoid modifying the rnn API. You could simply run the step function on the first input then multiply the result by 0., giving you an all-zero output of the correct shape.

I don't think this is a possibility. If I want scan to give me access to the previous output, I need to provide it a non-None outputs_info. Unfortunately from the Theano docs:

the initial state should have the same shape as the output and should not involve a downcast of the data type of the output

So I need to pass in an array of the correct (unfortunately currently unknown) size to the outputs_info of scan. This is an easy change to the backend API, though, and this PR already changes the rnn function to accept a mask so existing calls to rnn need to be changed either way.

Further, though I guess it's an edge-case, when using a stateful RNN you probably don't expect an all-0 initial masked output, but rather the output from the previous batch, no? I haven't been following the stateful design too closely.

OK I've implemented in the way that modifies the rnn call. This is not necessary for the TensorFlow backend, but unfortunately unless someone can find a different way to use scan I don't think we can get away with having it be inferred by the Theano backend. Take a look at the approach.

The TF error is:
No OpKernel was registered to support Op 'Transpose' with these attrs
TF does have an Op for Transpose with integer tensors. So it isn't clear to me where this error comes from.

Hey sorry just got back from a vacation. Looking at your feedback now.

The TF error is:
No OpKernel was registered to support Op 'Transpose' with these attrs

How did you find this? I couldn't get it to give me a useful error message.

From the Travis build.
E         tensorflow.python.framework.errors.InvalidArgumentError: No OpKernel was registered to support Op 'Transpose' with these attrs
E         	 [[Node: transpose_801 = Transpose[T=DT_BOOL](ExpandDims_68, transpose_801/perm)]]
E         Caused by op 'transpose_801', defined at:
[ blah blah blah]
E           File "/home/travis/build/fchollet/keras/keras/backend/tensorflow_backend.py", line 404, in rnn
E             mask = tf.transpose(mask, (1, 0, 2))

Wait, I hadn't read that message very carefully. It actually complains about the absence of a DT_BOOL transpose op. I thought the mask was uint8. It's quite likely that there is indeed no TF transpose op for bool tensors (there is in general very poor bool tensor support).

OK I'll experiment with using the mask as a uint8 until we get to the switch statements.
On my Mac I don't get the same output as the Travis build sadly, rather a more generic error message. Is the Travis machine running linux?

Yes. If you're not on Linux, then it might be a difference in your TF build.

Great: well that fixed it on my machine. Hopefully the same for the Travis build, I've pushed a fix.

So the current status is: this works and now passes tests both in TensorFlow and Theano. The remaining point of contention is the API change where the output_dim parameter must be specified. It's certainly an ugly change, since there's no conceptual reason why it should be required, but I do not currently have any ideas on how to bypass this requirement due to Theano's scan API requiring you to explicitly specify the initial output.

Ok, thanks for the update! Have you tried obtaining the output size by evaluating the step function on a dummy array before calling scan? Pretty ugly, but it should work.

Have you tried obtaining the output size by evaluating the step function on a dummy array before calling scan? Pretty ugly, but it should work.

I have, but can't get it to work. The dummy array needs to be of the correct shape, and if I try to evaluate the shape of the input array to build the dummy, I get a disconnected input error. I also tried passing in a (1,1) array with broadcasting, but not every operation in an arbitrary step function supports broadcasting. For instance a dot failed in SimpleRNN.

This has been merged: https://github.com/fchollet/keras/commit/ac773ed243154c5408937122bb67f5ec754b29d5
Still unhappy about the rnn API, but it seems like we don't have any alternative for the time being.

@fchollet

Still unhappy about the rnn API

Can you tell us what you have in mind? What would be the ideal RNN api. Just exactly what you want, strictly from a UX perspective, without worrying how it would be implemented?

@farizrahman4u I know that the main concern was this modification: https://github.com/fchollet/keras/pull/1310/files#diff-ff634bb5c5441d7052449f20018872b8L406
Basically I needed to explicitly pass in the output dimension to the rnn call despite that being redundant, in order to be compatible with Theano's scan API.
This doesn't affect users of built-in RNN layers, just people building custom RNN layers that need to call the backend directly.

@wxs. Oh! I am noticing the output_dim parameter now..Its very bad (and annoying) UX to be honest. (no offense).  Can't believe @fchollet merged this! I think  its based on the assumption that no one uses the rnn api directly but a lot of people do. I will come up with some idea to fix this ASAP.

#1494  -(not completed)

Changes Unknown when pulling 2f300573ce8664eb8b3e14aaa6884475e9db0360 on wxs:backend-masking into ** on fchollet:master**.


Figure out the situation with register_notifications_protocol

The NetworkService::register_notifications_protocol method is a hack, as explained here.
The problem with this method is that we might have started already connecting to other nodes.
At the moment, if two nodes connect before register_notifications_protocol has been called, and if then we try to send a message on the given protocol, it will be sent on the legacy substream as a fallback.
This is however getting in the way of removing the legacy substream. cc https://github.com/paritytech/substrate/issues/5670
The proper fix is to refactor Substrate to pass the list of protocols in the NetworkConfiguration, but that's a lot of work.

Update: the legacy substream is no longer in use. If two nodes connect before register_notifications_protocol has been called, messages are simply lost.
https://github.com/paritytech/substrate/pull/6833 was done in order to prevent this situation from happening.


Can't POST Ingress to API using curl, but kubectl works fine.

Trying to execute:
curl -X POST -d @ingress.yaml  -H 'Content-type:application/yaml' http://<IP_ADDRESS>:8080/apis/extensions/v1beta1/namespaces/somenamespace/ingresses
ingress.yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echomap
  namespace: somenamespace
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /foo
        backend:
          serviceName: echoheaders-x
          servicePort: 80
  - host: bar.baz.com
    http:
      paths:
      - path: /bar
        backend:
          serviceName: echoheaders-y
          servicePort: 80
      - path: /foo
        backend:
          serviceName: echoheaders-x
          servicePort: 80

But the call fails with:
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "the object provided is unrecognized (must be of type Ingress): yaml: mapping values are not allowed in this context",
  "reason": "BadRequest",
  "code": 400
}

Even though when executing kubectl create -f ./ingress.yaml, everything works just fine.

Seems to only be an issue with YAML. JSON works fine.

@brandong954 please open an issue in the main Kubernetes repo.
This could be related to https://github.com/kubernetes/kubernetes/issues/22904

I have encountered this problem. Has anyone been solved yet ？

@willstudy One workaround is to convert your definition file to JSON.

Same here;
json works, yaml not
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.1", GitCommit:"82450d03cb057bab0950214ef122b67c83fb11df", GitTreeState:"clean", BuildDate:"2016-12-14T00:57:05Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"darwin/amd64"}

is there any update on this? Why does kubectl yaml work and the curl doesn't work?

This still seems to be an issue.  I tested it on
kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.1", GitCommit:"82450d03cb057bab0950214ef122b67c83fb11df", GitTreeState:"clean", BuildDate:"2016-12-14T00:57:05Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.4", GitCommit:"$Format:%H$", GitTreeState:"not a git tree", BuildDate:"2017-03-15T16:47:16Z", GoVersion:"go1.7", Compiler:"gc", Platform:"linux/amd64"}
curl http://<IP_ADDRESS>:8080/api/v1/namespaces/default/pods     -XPOST -H'Content-Type: application/yaml'<EMAIL_ADDRESS>{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "the object provided is unrecognized (must be of type Pod): yaml: mapping values are not allowed in this context (2d2d2d6b696e643a20506f6461706956657273696f6e3a2076316d657461 ...)",
"reason": "BadRequest",
"code": 400
}

For me the problem was with curl if you use -d flag it removes new lines ('\n') from the data. Use --data-binay instead:
Working example:
curl -X POST --data-binary @nginx.pod.yaml -H "Content-Type: application/yaml"  http://localhost:8001/api/v1/namespaces/default/pods


Throw error

Got This Error

It says cannot find module cron. Either it can't find the module in the package file or there was some problem with the npm install. What were the steps that you followed?


Running multiple canvases in on window

right now you need to edit a line in the ccpwgl_int and one in the ccpwgl file to run multiple canvases in one scene.
to get the benefits of a shared resManager you wouldnt want to use the solution i use, the problem is though that the Tw2Device is an instance instead of a constructor because a ton of other classes use the device without a reference but through using the exported instance.
anyone interested in a "fixed" version?

There are a few constructors that can have their dependencies to the instantiated device object removed by replacing their device references with GL and CCP enums.
As you can't create textures and buffers on one context/device and use them on another, any constructor that was tied to a specific device would need to have it's device defined as one of it's properties - which would need to be set by Tw2ObjectReader and/or Tw2ResMan and/or EveSOF or by it's parent object.
However, regardless of how it's done you'd still need to load the same resources for each device/context you have, so a single resource manager between the two may not be any better than what you are doing now.

i just want to point out that with a tiny mod to the dist ccpwgl_int or a "hack" aka extra script being loaded before ccpwgl_int you can easily spawn multiple canvases on one page since you can generate multiple ccpwgl_int contexts

@Robbilie whats the extra script?

window.define = function (a, ccpwgl_int) {
	window.ccpwgl_int = ccpwgl_int;
};
window.define.amd = {};

Your solution is the easiest "fix", I can't see any other way of solving this presently.
This issue can be closed.


Python: TypeError: KernelFunction.from_prompt() missing 2 required positional arguments: 'function_name' and 'plugin_name'

https://github.com/microsoft/semantic-kernel/blob/45dc8c229342cef19ae56b33bc4d893b9fa55177/python/semantic_kernel/kernel.py#L372
This line when calling kernel.invoke_prompt() with a prompt string errors out with the above error.

Thanks for reporting this, @silarsis. A fix is incoming.