Introduce WAF in guard-gate

Currently we don't have any sort of web application WAF in guard-gate and without this we miss some features:

injection prevention
virtual patching
payload inspection

we could implement WAF with @corazawaf as it is also in Go. I would be up for a PR.

Hi jcchavezs, Welcome
The overall direction of identifying functionality that WAFs typically use and considering them to Guard is a positive one that we will gladly endorse.
However, we need to recognize two main aspects:

Guard is not a WAF and does not try to replace a WAF - this does not mean that functionality common in WAFs has no place in Guard, it just means that each function needs to be reviewed and considered face value  - do we need it in front of every Pod, do we gain from placing it in each Pod vs placing it in at the ingress, etc.
Guard is very much tuned into improving security operations - This means that when we consider adding a function, we ask if it can be useful without any configuration/rules being set by DevOps. It is ok, if under certain conditions (when there is a CVE for example), rules will be required to improve security - but we do aim to offer "out-of-the-box" security even without human-created rules. To do that, we often look into options for creating the rules with ML.

Please try to analyze the functionality you wish to add in view of these two aspects. Maybe best if we create a document to start discussing this - I do ask that we separate each functionality that we want to add and consider these aspects first, then understand the planned design and only then move to PR.  I created an empty document for this (will add the formal Knative Proposal structure later).
cc: @evankanderson, @nainaz

+1 to minimal configuration, or to making it easy to layer WAF into the ingress path before the queue-proxy.  It looks like the Gateway API may be reaching a sufficiently-supported stage that looking at the policy extension options in that API might make sense as an alternative to embedding a WAF and WAF configuration inside guard.
In particular, because of where it lives, the path for steering complex configuration like virtual patching into guard is not entirely clear.  (guard already does some payload inspection, so that might be a good place to start?)

+1 to minimal configuration, or to making it easy to layer WAF into the ingress path before the queue-proxy. It looks like the Gateway API may be reaching a sufficiently-supported stage that looking at the policy extension options in that API might make sense as an alternative to embedding a WAF and WAF configuration inside guard.
In particular, because of where it lives, the path for steering complex configuration like virtual patching into guard is not entirely clear. (guard already does some payload inspection, so that might be a good place to start?)

Let's shed more light on this comment - Guard does implement a per-service configuration which we call Guardian. Guardian is a CRD Object include the service "micro-rules". These micro-rules can be learned with ML but users also have the ability to adapt them manually.
What @evankanderson seek to highlight is that the boot sequence of Guard is presently limited such that we do not have a good way to pass too many config parameters on boot (this is the result of some limitations on how Guard and Queue-Proxy interact).
virtual patching can be configured into a Guardian or instead, we could have a Guardian indication that a virtual patch exists and that we should go fetch it from some other location.
In principle, I believe placing virtual patching in Guard can be a nice fit. Note that for most microservices, we will have no patch  - this is in-line with Guard's "out-of-the-box" protection.  However, it does empower the users to add a patch as a last resort when other defenses fail.

Thanks for the careful feedback.
First of all, most of the WAF solutions offer coreruleset support which means we might not need configuration as we can leverage CRS under a low paranoia level or even better careful chose a default subset of it that can be leverage in every sidecar. I still believe per sidecar makes sense as an attacker can aim to exploit vulnerabilities from a compromised host to the entire cluster. Virtual patching can be achieved by CRS by the way.
As for injection prevention we could leverage a small validator for injection using https://github.com/corazawaf/libinjection-go inside the guard. This will not require config and it is pretty standard.
As for Gateway API, main issue here is that there is no (and I think there won't be) a way to uniformly extend the Gateway. Different implementations might chose their own plugin system and that is not only a huge friction but sometimes even if the implementation API allows it, the implementation might not be as performant as we would like to.

@jcchavezs
Would you be able to describe each WAF feature you seek to add separately in https://docs.google.com/document/d/1Q2UnKJ99YG_LNSQTc2PTAmM1HGH0Ctvc-bc9a43p3-8/edit#,
Maybe best to start by describing one feature first.  I can suggest "virtual patching" since from where I stand it looks like a good match to Guard to cover more sophisticated filtering that can't be achieved by Guard's Microrules.
For each feature, try to analyze (I will help with this analysis based on what you will write):

Why it would make sense to have it in a per-pod gate
How can it work out of the box and provide basic common protection
What additional configurations will it add for DevOps (or DevSecOps) and will they need to be configured (examples will be useful)
What is the interplay it has with what Guard already does (what does it cover that the existing technology does not) - I will defiantly help you with this one :)
The design we should have (e.g. should it block before or after other mechanisms)
If you rely on existing code or library point to it, please.

I just filled the document, I am up for feedback! https://docs.google.com/document/d/1Q2UnKJ99YG_LNSQTc2PTAmM1HGH0Ctvc-bc9a43p3-8/edit#


⚠ Bofrost.si has degraded performance

In 9c11350, Bofrost.si (https://www.bofrost.si) experienced degraded performance:

HTTP code: 200
Response time: 2585 ms

Resolved: Bofrost.si performance has improved in bb2b90d after 6 minutes.


Power ratings rely on summary file line ordering that penalizes newer models

Some of the results of your new power rankings are quite counterintuitive. For example:

Your baseline and the COVIDhub-baseline appear as the 6th and 7th best.
Some newer models (e.g., LNQ-ens1, JCB-PRM, karlen-pypm, SteveMcConnell-CovidComplete) that have demonstrated pretty high accuracy even over months appear much lower. Three of those mentioned here actually appear below the median in your power ranking.

As I read the code, this is because for weeks in which a model doesn't have a value in a given column (i.e., a week), your row ordering method for the summary files evaluates them as having the worst of all performance in that column (week). So models that are newer are penalized for not submitting in the weeks before they were developed. Longevity is rewarded over accuracy.
Quite interestingly, your new code contains a feature to ensure that models that have stopped submitted forecasts are not penalized in the same way for weeks without values in your summary files (i.e., since they've stopped submitting).
A fair comparison would treat these two situations on an equal basis, i.e., not penalizing more recently developed models. (All models should be subject to a minimum threshold for number of forecasts, of course).
I trust this is an oversight. The modeling community would benefit from a more fair comparison. Would you please investigate this and consider revising your code and posted rankings?
Thanks.

Thanks for your feedback, John. The main purpose of this repository is to compute the forecast errors for all models and present them in a table format. The ordering of the models is secondary - you are welcome to come up with alternate methods to order them as you see fit. I understand you use these errors to compute your own rankings where you do not factor in longevity. And that is fine.
To address your first point, there is no question that the LNQ, CMU, and JCB models have performed well since they first started making forecasts in July, and the data supports that. But the evaluation is from April to the present, and unfortunately those models do not have forecasts from April-July. Hence, the ordering takes that into account. Please also note that the longer a model makes forecasts, the less a "late entry" matters. For example, the Oliver Wyman model did not begin making forecasts until June but is now one of the top models. Your notion of optimal ranking methodology may also change over the next few months if even newer models appear or if you choose to no longer make forecasts.
There is no ranking system that is going to be perfectly fair. The methodology and rationale for the ordering currently used in this repo is specified in the README:

A byproduct of this method is that we are rewarding models for submitting forecasts and submitting their forecasts early. A model is much more useful if it began making forecasts in April than in July. When there are very few models in the early stages of the pandemic, each model's forecasts carry much more weight and usefulness. We believe this most accurately balances the time/complexity trade-off: you can make models more complex/accurate at the expense of development time.

The methodology can change in the future, and I will be sure to notate that in the README.

I share concerns similar to John's, FWIW. You're clear that the "score" is the number on the right, but people assume that evaluations are presented in ranked order, and the order you're presenting the models sends a mixed message vs. the scores.
I've already shared my thoughts about using MAE as the error measure. Almost any other error measure would be more meaningful, including log difference, log difference squared, Pearson fit criteria, RMSE, BRE, pred(25) -- almost anything.
One other thought for your consideration is that you might consider including forecasts for periods in addition to the 4-week forecasts. In terms of lead time needed to shift PPE, ventilators, medical staff, and other equipment to the areas that need it, 2 week and 3 week forecasts provide the lead time to do that, and accurate forecasts for those periods seem as important as 4-week forecasts. They are arguably more important, since the 4-week forecasts will be supplanted by updated 2 and 3 week forecasts before the target date is reached.
(You might find it interesting that for national forecasts, your forecasts were most accurate for 1 week, then 2 weeks, then 3 weeks, then 4 weeks. John's are most accurate at 2 weeks, then weeks 1, 3, 4. The Ensemble model is most accurate at 2 then 3 then 4 then 1. My model is most accurate at 4 then 2 then 3 then 1. This is with error measured as the geometric mean of log difference.)

Thank you for your feedback.
Regarding the usage of MAE, to my understanding it is a fairly standard error metric. In fact, the Ensemble pre-print uses MAE in its evaluations. But I agree with you in that there is no perfect metric.
As mentioned in the README:

There are more advanced techniques for scoring forecasts that take advantage of quantile estimates, but we will leave that as a future extension.

I don't have any knowledge of what MAE might validly be used for, but it is not standard for evaluating forecasts.
You might check out Ch 14 in J. Scott Armstrong's book, Principles of Forecasting. His first heading under the "Error Measures" section is "Ensure that error measures are not affected by scale." That rules out MAE.
A good paper is "A better measure of relative prediction accuracy for model selection and model estimation," by Chris Tofallis. Here's a link:
https://www.researchgate.net/publication/271773969_A_better_measure_of_relative_prediction_accuracy_for_model_selection_and_model_estimation
Since my background is software, there are good discussions of forecast evaluation (aka estimation evaluation) in these books:
Software Engineering Metrics and Models, Conte, Dunsmore, and Shen - see Section 3.5 on Model Evaluation
Estimating Software Intensive Systems, Richard Stutzke - see Appendix C, Measures of Estimation Accuracy.
I hope this helps.


Feature - Debug Output

From the wiki:
Could you create a verbose mode which will allow the user to get a high
level output of what is going on.
Maybe a debug level would be helpful as well to show a full debug output.

I am having issues on my Linux box where the cmd program is failing silently
with no outputs created. I have no idea what the underlying issue is. With a
verbose and/or debug output I could make more
progress narrowing in on the issue.

This is an excellent feature request and have been meaning to do this for quite a while. However, with the current design, it is quite difficult to add this. I'm working on a new UI that will have a different backend as well. I will take this basic feature into consideration while developing this new UI.

Thanks for looking at this.

This will be in the next build.

--verbose option was added in the latest CMD build. Also, the error log gets more information now from the GUI.

how do i access debugging on runtime?

You mean the developer tools? Enable the "Show Toolbar" option under the "Window Settings" tab. Then click the little three lined box in the upper right hand corner.

How do we do this now that Show Toolbar has been removed from the Windows Settings tab?  I added "toolbar":true to my manifest but still not seeing it.
Thanks.

@karneaud The sdk version had the debug options, the regular version does not. The toolbar no longer exists in either.

You must use the "SDK build" in the download settings. This build of NW.js will allow you to debug your app and should not be used for production. You can access the console using the F12 key or ⌘+⌥+i on Mac as specified here

Thanks Joey- worked like a charm.    Thank you for keeping this project going and being so quick to respond.


Error: ENOENT no such file or directory as /polls/bot_1.json

Expected behavior
Website starts
Actual behavior
Website doesn't start
Steps to reproduce the behavior
root@iskin:~# cd ../var/www/html
root@iskin:/var/www/html# nodejs index.js
Other

Error Stack
Bot (bot_1) has been logged-in.
[!] All bots online.
[!] Website server is online.
[!] Socket server is online.
fs.js:101
throw err;  // Forgot a callback but don't know where? Use NODE_DEBUG=fs
^
Error: ENOENT: no such file or directory, open './polls/bot_1.json'
at Error (native)

Cant you look at past/closed issues? Create a polls folder. Then it will populate it.

CLOSE!


style: run ruff checks on ci

Hey @kszucs! I'd be interested in helping with this one.
I did a preliminary ruff check, and I'm getting the following totals (some of these are coming from the tests, too):

Rule
Count

BLE001
4

D417
2

E402
4

F401
1

F403
4

F405
12

F811
4

F821
1

PLR1714
2

RUF012
3

RUF100
1

SIM110
2

Verbose Output
koerce/init.py:5:1: F403 from ._internal import * used; unable to detect undefined names
koerce/init.py:8:17: F405 Deferred may be undefined, or defined from star imports
koerce/init.py:10:19: F405 Var may be undefined, or defined from star imports
koerce/init.py:14:16: F405 Capture may be undefined, or defined from star imports
koerce/init.py:53:20: F405 pattern may be undefined, or defined from star imports
koerce/init.py:54:20: F405 deferred may be undefined, or defined from star imports
koerce/init.py:62:16: F405 Replace may be undefined, or defined from star imports
koerce/init.py:74:5: D417 Missing argument description in the docstring for koerce: allow_coercion
koerce/init.py:75:10: F405 Pattern may be undefined, or defined from star imports
koerce/init.py:75:26: F405 Any may be undefined, or defined from star imports
koerce/init.py:75:40: F405 Context may be undefined, or defined from star imports
koerce/init.py:76:6: F405 Any may be undefined, or defined from star imports
koerce/init.py:107:11: F405 pattern may be undefined, or defined from star imports
koerce/init.py:110:12: F405 MatchError may be undefined, or defined from star imports
koerce/_internal.py:3:1: F403 from .annots import * used; unable to detect undefined names
koerce/_internal.py:4:1: F403 from .builders import * used; unable to detect undefined names
koerce/_internal.py:5:1: F403 from .patterns import * used; unable to detect undefined names
koerce/annots.py:743:19: RUF100 Unused noqa directive (unused: SIM105)
koerce/annots.py:841:33: F821 Undefined name Self
koerce/patterns.py:806:16: PLR1714 Consider merging multiple comparisons: lowered in ("true", "1"). Use a set if the elements are hashable.
koerce/patterns.py:808:18: PLR1714 Consider merging multiple comparisons: lowered in ("false", "0"). Use a set if the elements are hashable.
koerce/patterns.py:904:16: BLE001 Do not catch blind exception: Exception
koerce/patterns.py:936:16: BLE001 Do not catch blind exception: Exception
koerce/patterns.py:977:16: BLE001 Do not catch blind exception: Exception
koerce/patterns.py:1027:16: BLE001 Do not catch blind exception: Exception
koerce/patterns.py:2558:5: D417 Missing argument descriptions in the docstring for pattern: allow_coercion, self_qualname
koerce/tests/test_annots.py:705:9: F811 Redefinition of unused test from line 701
koerce/tests/test_annots.py:722:9: F811 Redefinition of unused test from line 718
koerce/tests/test_annots.py:999:9: SIM110 Use return all(a == b for a, b in zip(self, other)) instead of for loop
koerce/tests/test_annots.py:1051:9: SIM110 Use return all(self[key] == other[key] for key in self) instead of for loop
koerce/tests/test_annots.py:1244:11: F811 Redefinition of unused Op2 from line 1241
koerce/tests/test_annots.py:2183:31: RUF012 Mutable class attributes should be annotated with typing.ClassVar
koerce/tests/test_builders.py:209:5: F811 Redefinition of unused resolve from line 28
koerce/tests/test_y.py:13:1: E402 Module level import not at top of file
koerce/tests/test_y.py:14:1: E402 Module level import not at top of file
koerce/tests/test_y.py:15:1: E402 Module level import not at top of file
koerce/tests/test_y.py:17:1: E402 Module level import not at top of file
koerce/tests/test_y.py:171:27: RUF012 Mutable class attributes should be annotated with typing.ClassVar
koerce/tests/test_y.py:178:27: RUF012 Mutable class attributes should be annotated with typing.ClassVar
koerce/utils.py:7:25: F401 typing.ClassVar imported but unused

Would you like to add any of these rules to the ruff ignore list, or should we mark them as #noqa?

We should also add a build running the ruff checks.


Create requirements.txt

Thanks for the project.
To make installation easier, I've added a requirements.txt, which is a common way to describe dependencies.
With that, users can simply run pip install -r requirements.txt and they are all set. 😊

Thanks, for the suggestions. I didn't think anyone was even going to find this repository.

😆 People will.


The OOB prediction in the rust rf function

In R, the prediction method with "prob" argument of randomForest package returns the rate of votes of the base trees. I am not that fimilar to Rust. I try to read the source code of the random forest code in biosphere. It seems the same as R did?
See the following question in stackexchange.
random-forest-probability
The Method 1 is what the R did. Is the rf function in Rust works as the Method 2？

Yes, the random forests used in changeforest predict probabilities according to method 2.

Thanks for your reply! It is a spired work! I try the probability estimation including the two methods today. The function in R-package randomForest uses method 1 and ranger uses method 2. Here is the sample code.
library(ranger)
library(tictoc)
library(ggplot2)
library(tidyr)
library(dplyr)
library(randomForest)
simulate_data <- function(n){
  X <- data.frame(matrix(runif(n*10), ncol = 10))
  Y <- data.frame(Y = rbinom(n, size = 1, prob = apply(X^3*2, 1, sum) %>%
                               pnorm(mean = 5)
  ) %>% as.factor())
  dplyr::bind_cols(X, Y)
}

train_data <- simulate_data(5000)
test_data <- simulate_data(5000)

tic("ranger")
model_ranger <- ranger(Y ~., data = train_data,
                        num.trees = 100,
                        mtry = floor(sqrt(10)),
                        write.forest = T,
                        min.node.size = 100,
                        probability = T
)
toc()

tic("randomForest")
model_randomForest <- randomForest(Y ~., data = train_data,
                                    ntree = 100,
                                    mtry = floor(sqrt(10)),
                                    nodesize = 100
)
toc()

pred_ranger <- predict(model_ranger, test_data)$predictions[,2]

# rfUtilities::probability.calibration(Y, pred_ranger, regularization = TRUE)

pred_rf <- predict(model_randomForest, test_data, type = "prob")[,2]
Y = as.numeric(test_data$Y) - 1
pred_rf_calibrated <- rfUtilities::probability.calibration(Y, pred_rf)

prob_real <- apply(test_data[,1:10]^3*2, 1, sum) %>% pnorm(mean = 5)

print(c(mean((pred_rf - prob_real)^2),
        mean((pred_ranger - prob_real)^2),
        mean((pred_rf_calibrated - prob_real)^2)))

data.frame(prob_real, pred_ranger, pred_rf, pred_rf_calibrated) %>%
  tidyr::gather(method, prob, -prob_real) %>%
  ggplot(aes(x = prob, y = prob_real)) + geom_point(size = 0.1) + facet_wrap(~method)

Also I try the probability calibration algorithm Isotonic Regression, it seems that in this setting, all the probabilities estimator could be biased. But overall, it is enough to detect the signals once changes occur.

You can use the method 2 aggregation with any random forest library for 2-class classification by treating the classification problem as a regression problem of $Y \in \{0.0, 1.0\}$. It is true that random forests (or most classifiers) will struggle to produce calibrated probability estimates. However, there is evidence suggesting that method 2 produces better calibrated estimates, which is why they are used in changeforest.

You can use the method 2 aggregation with any random forest library for 2-class classification by treating the classification problem as a regression problem of Y∈{0.0,1.0}. It is true that random forests (or most classifiers) will struggle to produce calibrated probability estimates. However, there is evidence suggesting that method 2 produces better calibrated estimates, which is why they are used in changeforest.

Quite agree. A approximated class probability estimation (with nearly monotone property) shall be enough for CP detection.


ocm-lib facade implementation

How to categorize this PR?

/area oci
/kind task
/priority 3
What this PR does / why we need it:
This PR is based on #755.
It provides adapters to implement the introduced facade for working with the ocm with the ocm-lib. Ultimately, this shall enable the landscaper to also work with ocm component descriptor format instead of the legacy cnudie component descriptor format.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Release note:

- support for ocm component descriptors

@achimweigel separated this huge pull request into two pull request - one dealing primarily with the dependencies introduced in this pull request and the non-active coding #837 and another one dealing with the actual changes to the core functionality #844.
This separation allows for a more comfortable and secure review and handling of those changes to the core functionality.


Policy-request handler in server

#18

Created a mature prototype. Can be found in the forked repo


.aspx and .asax files don't get published

I'm deploying as follows:
msbuild /t:Restore,Rebuild /p:DeployOnBuild=true /p:PublishProfile=Live.pubxml
/p:Password="(my password)" /p:Configuration=Live .\MyProject\MyProject.csproj

This does build a new DLL and upload that and its dependencies to the server. However, it does not replace any aspx files on the server.
I notice VS infers the build action for aspx as None, and I'm not seeing a reason at a glance, in my csproj. Some snippets:
<Project Sdk="MSBuild.SDK.SystemWeb/4.0.49">
  <PropertyGroup>
    <TargetFramework>net472</TargetFramework>
    <UseIISExpress>true</UseIISExpress>
    <IISExpressSSLPort>44304</IISExpressSSLPort>
    <IISExpressAnonymousAuthentication />
    <IISExpressWindowsAuthentication />
    <IISExpressUseClassicPipelineMode />
    <UseGlobalApplicationHostFile />
    <Use64BitIISExpress />
    <LangVersion>9.0</LangVersion>
    <AssemblyTitle>MyProject</AssemblyTitle>
  </PropertyGroup>
  <ItemGroup>
    [.. a few package references]
  </ItemGroup>
  <ItemGroup>
    [.. a few linked files to add to compilation]
  </ItemGroup>
  <ItemGroup>
    <PackageReference Update="Microsoft.Net.Compilers.Toolset" Version="3.11.0" />
  </ItemGroup>
</Project>

I'm guessing asmx and ascx files would be affected as well.
So I think the default item includes should have:
  <ItemGroup>
    <Content Include="*.asax" />
    <Content Include="*.ascx" />
    <Content Include="*.asmx" />
    <Content Include="*.aspx" />
  </ItemGroup>

It has been a while since I did webforms development (I mainly use/maintain MVC and Razor these days).
Are there any other types that should be included? I seem to remember master pages (.master?).
I believe these should be recursive so they include at any depth of folder, not just the root so probably something like
  <ItemGroup>
    <Content Include="**\*.asax" />
    <Content Include="**\*.ascx" />
    <Content Include="**\*.asmx" />
    <Content Include="**\*.aspx" />
    <Content Include="**\*.master" />
  </ItemGroup>

I'll look at adding these as default includes, perhaps behind a flag so that they can be disabled if required.

Good point — .master, too. And yes, it should be recursive (technically, the rules are more complex; e.g. App_Code should not contain markup).

I believe .ashx should be also included

Possibly also resx, axd, config.

@chucker .resx and web.config files were already handled. And .axd files do not actually exist - the URLs are just routed to a handler class by IIS.

@CZEMacLeod Thanks for the great work on this project.  I updated to this version and noticed an issue after performing a publish.
In my project I end up with "bin\Release\Publish"  as the output folder, however if turn on EnableWebFormsDefaultItems, once I publish it see's that published folder as content. Then each publish adds more content upon itself and eventually errors.
I have worked around this in some of my other globs by adding Exclude="$(DefaultItemExcludes)".
I recommend changing from:
<Content Include="**\*.asax" />
<Content Include="**\*.ascx" />
<Content Include="**\*.ashx" />
<Content Include="**\*.asmx" />
<Content Include="**\*.aspx" />
<Content Include="**\*.master" />

to
<Content Include="**\*.asax" Exclude="$(DefaultItemExcludes)" />
<Content Include="**\*.ascx" Exclude="$(DefaultItemExcludes)" />
<Content Include="**\*.ashx" Exclude="$(DefaultItemExcludes)" />
<Content Include="**\*.asmx" Exclude="$(DefaultItemExcludes)" />
<Content Include="**\*.aspx" Exclude="$(DefaultItemExcludes)" />
<Content Include="**\*.master" Exclude="$(DefaultItemExcludes)" />

@mcnallys The props file actually adds all the item types to DefaultItemExcludes so using that to exclude won't work.
However, we can setup excludes for the Content entries
  <!-- Exclude WebForms items from default items -->
  <PropertyGroup Condition="'$(EnableWebFormsDefaultItems)'=='true'">
    <DefaultItemExcludes>$(DefaultItemExcludes);**\*.asax;*.asax</DefaultItemExcludes>
    <DefaultItemExcludes>$(DefaultItemExcludes);**\*.ascx;*.ascx</DefaultItemExcludes>
    <DefaultItemExcludes>$(DefaultItemExcludes);**\*.ashx;*.ashx</DefaultItemExcludes>
    <DefaultItemExcludes>$(DefaultItemExcludes);**\*.asmx;*.asmx</DefaultItemExcludes>
    <DefaultItemExcludes>$(DefaultItemExcludes);**\*.aspx;*.aspx</DefaultItemExcludes>
    <DefaultItemExcludes>$(DefaultItemExcludes);**\*.master;*.master</DefaultItemExcludes>
    <DefaultWebFormsItemExcludes>$(BaseOutputPath)\**;$(BaseIntermediateOutputPath)\**</DefaultWebFormsItemExcludes>
  </PropertyGroup>

  <!-- Include WebForms items as content -->
  <ItemGroup Condition="'$(EnableWebFormsDefaultItems)'=='true'">
    <Content Include="**\*.asax" Exclude="$(DefaultWebFormsItemExcludes)" />
    <Content Include="**\*.ascx" Exclude="$(DefaultWebFormsItemExcludes)" />
    <Content Include="**\*.ashx" Exclude="$(DefaultWebFormsItemExcludes)" />
    <Content Include="**\*.asmx" Exclude="$(DefaultWebFormsItemExcludes)" />
    <Content Include="**\*.aspx" Exclude="$(DefaultWebFormsItemExcludes)" />
    <Content Include="**\*.master" Exclude="$(DefaultWebFormsItemExcludes)" />
  </ItemGroup>

If you try this (but change EnableWebFormsDefaultItems to EnableWebFormsDefaultItems2) and it resolves the issue for you, I can make the update and publish.

@mcnallys I've created a new issue #25 for this problem.


[Bug] include_all_form_fields causes maximum size allowed in big forms

Bug report
What I did
I have 50 fields in the create operation, one of them is select2_from_ajax with include_all_form_fields.
What I expected to happen
I expect to get items from the Ajax request.
What happened
The response is "414, Request-URI Too Large, The request URL is over the maximum size allowed!".
What I've already tried to fix it
As I see, filling the form with long texts causes the error because it include_all_form_fields, so I suggest adding an option like include_dependencies_form_fields to include only the related fields.
Is it a bug in the latest version of Backpack?
After I run composer update backpack/crud the bug... is it still there?
Backpack, Laravel, PHP, DB version
When I run php artisan backpack:version the output is:
BACKPACK VERSION:
5.1.0@9e1de794137ceb1ade1fa35e3b9366f97285efb3

Making method=post may solve the issue. but include_dependencies_form_fields still a better solution.

That's exactly right @mamprogr - the best solution here, I think, is to use POST. In hindsight, we should have seen this coming, and should have made POST the default, when we instruct people to create their own API. We thought about changing the default in v5 but... there were already enough breaking changes.
We'll reconsider changing the default to POST in v6 too, or perhaps default to the fetchOperation, which uses POST too.
I don't think there's anything we can do here in a non-breaking way, so I'm going to close this. If you think I'm wrong, please let me know and we'll reopen.
Cheers!

Actually, I think there IS something we can do... we can change the docs, starting now, so that at least new people use POST, not GET. 😅👀 A dirty solution but... it might just work, for a lot of people. What do you think @pxpm , am I missing something here?

Hello @tabacitu
I totally agree, we can document POST now but keep GET as default until v6.
What do you think is a good place to document it ? Probably here: https://backpackforlaravel.com/docs/5.x/crud-how-to#how-to-add-a-select2-field-that-depends-on-another-field ?
Let me know.

@promatik this should be another easy one, in the docs.


In Android Studio IDE

import de.fayard.refreshVersions.RefreshVersionsSetup in settings.gralde is recognized as an error but there is no problem of compilation
All the dependencies that we write with "_" are in warning and we suggest to replace by the version
This is only the display the compilation works very well but here I wanted to put it back

Here's how to tell lint to stop warning you: https://github.com/jmfayard/refreshVersions/issues/270#issuecomment-709602733


Refine peer-to-hole-punch <-> client allocation logic

Right now, every dcutr-capable peer gets served to every client if it has not hole-punched that peer within the last 10min. This means if we had dozens of clients at some point, we would hole punch that one dcutr-capable peer dozens of times.
The selection logic is captured in the SQL statement:
SELECT p.multi_hash, ma.maddr
FROM connection_events ce
         INNER JOIN connection_events_x_multi_addresses cexma ON ce.id = cexma.connection_event_id
         INNER JOIN multi_addresses ma ON cexma.multi_address_id = ma.id
         INNER JOIN peers p ON ce.remote_id = p.id
WHERE ce.listens_on_relay_multi_address = true
  AND ce.supports_dcutr = true
  AND ce.opened_at > NOW() - '10min'::INTERVAL
  AND ma.is_relay = true
  AND NOT EXISTS(
        SELECT
        FROM hole_punch_results hpr
                 INNER JOIN hole_punch_results_x_multi_addresses hprxma ON hpr.id = hprxma.hole_punch_result_id
        WHERE hpr.remote_id =  ce.remote_id
          AND hpr.client_id IN (%s)
          AND hprxma.multi_address_id = ma.id
          AND hprxma.relationship = 'INITIAL'
          AND hpr.created_at > NOW() - '10min'::INTERVAL
    )
LIMIT 1

While this is quite a beast, the logic could be a little more elaborate.
Improvement suggestions:

Don't DoS a remote peer by serving it to all clients. Limit the hole punches to a single remote peer to, let's say, 10 attempts in 10mins - regardless of which clients have hole punched that peer.
Don't serve QUIC multi addresses to the rust client as it does not support those anyways.
Serve all relay multi-addresses to the clients instead of just a single one. This will make the cause-effect analysis a little bit more difficult but would yield more realistic success rate data.

Don't DoS a remote peer by serving it to all clients. Limit the hole punches to a single remote peer to, let's say, 10 attempts in 10mins - regardless of which clients have hole punched that peer.

Fixed that 👍


Problem Using Different Effects on Different Items on Same Canvas

Is it possible for different meshes in the same scene to have different effects applied to them?  In this code sandbox, only the effect fo the last rendered item is applied:
https://codesandbox.io/s/react-post-processing-single-item-3dm54?file=/src/index.js

Hi, at the moment it's not possible. If you use multiple EffectComposers only one of them will work.
I don't know the fix yet.

Updated your example, looks like the issue is still relevant. Dunno the best solution

in postprocessing yes, it's possible to select items, in react-pp, atm no, effects are meant for the entire scene. the problem is that selecting specific items is harder to express declaratively and we need to find a good model. @gsimone has begun merging the first effect that can select (afaik), godrays, and it's using regular react-refs, which might just be the solution for everything.


Can't pull single entry with 3.19

Hey, I'm getting this error when I try to pull a specific entry
Using: let query = Stack.ContentType(contentTypeUid).Entry(entryUid)
Error fetching entries: { message: 'Cannot call a class as a function' }
I can still get all the entries if only providing contentId.
I can confirm that it works when I'm on 3.17.1

I confirm, it doesn't work for me either

We have the same issue, had to downgrade

Yes same, even on a bare CRA

Hey , thanks for raising this. Could any of you please provide more clarity on the query? The fetch call seems to be working on my side. Here's how I did it:
import contentstack from 'contentstack';
import 'dotenv/config';

const stack = contentstack.Stack({
  api_key: process.env.API_KEY,
  delivery_token: process.env.DELIVERY_TOKEN,
  environment: process.env.ENVIRONMENT,
})

const result = await stack
    .ContentType("author")
    .Entry("bltf01a781af0dda942")
    .toJSON()
    .fetch();

console.log({ result });

@nadeem-cs
Here's how we're currenlty calling it.
     const Query = this._contentStack
      .ContentType('author')
      .Entry('bltf01a781af0dda942')
      .includeContentType()
      .includeEmbeddedItems()
      .language(locale)
      .includeFallback()
      .includeReference('some_sub_type');
    const content = (await Query.fetch()).toJSON();

@c0pp3rt0p the toJSON method should be called before fetch call. So your query would become
const Query = this._contentStack
    .ContentType('author')
    .Entry('bltf01a781af0dda942')
    .includeContentType()
    .includeEmbeddedItems()
    .language(locale)
    .includeFallback()
    .includeReference('some_sub_type');
const content = await Query.toJSON().fetch();

Try this and let me know if that works.

@nadeem-cs Can you update the QuickStart guide? It does not designate that in the instructions here. The example it gives is
const Stack = Contentstack.Stack({ "api_key": "api_key", "delivery_token": "delivery_token", "environment": "environment" });
const entry = Stack.ContentType('blog').Entry("entry_uid");
const result = await entry.fetch()

@nadeem-cs That fixes my issues.  Calling toJSON() prior to fetch() corrects the problem for my case.  Thank you!

I had the same problem after upgrading from a previously working version, and calling toJSON() before .fetch() was the resolution that allowed me to use the latest package.

Closing the issue as it has been resolved


Update to ACL 23.05.1, add ACL reorders

Raising PR on behalf of @davsva01 for reverted PR https://github.com/tensorflow/tensorflow/pull/61123

@penpornk

Please help resolve conflicts / rebase as well. :)


Implement proposal from #553 when consensus reached

Should this be transferred to openssl/project?

It's at least unclear what this is about.

@kroeckx sorry, this was to separate out the implementation task from the investigation task here https://github.com/openssl/project/issues/553


cant input transaction after reaching payment type

Describe the bug
input price, selected payment time (renamed to eg. credit card), when press credit card, it appear to have error and cannot be logged
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Hey @angyuxi , can you provide some screenshots in regard to this issue.

hi bruce
[image0.png]
Sent from my iPhone
On 3 May 2024, at 5:51 PM, Bruce Wang @.***> wrote:

Hey @angyuxihttps://github.com/angyuxi , can you provide some screenshots in regard to this issue.
—
Reply to this email directly, view it on GitHubhttps://github.com/brucewzj99/tele-tracker-v2/issues/42#issuecomment-2092672852, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BIH5N4J5W5MJZOTEAADVQQTZANMYBAVCNFSM6AAAAABHFAQLJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSGY3TEOBVGI.
You are receiving this because you were mentioned.Message ID: @.***>

Hi @angyuxi, im not able to see the screenshot, could you email it to me at<EMAIL_ADDRESS>if not perhaps you want to try /cancel and retry adding the entry. Thanks


Issue#260

issue #260
Hi heronshoes,
I try to update on readme.md, and readme.ja.md. But, I'm not sure these are the correct url, please check it out. Thanks!

Hi @viktoriussuwandi,
Thank you for the PR! Unfortunately links were not correct, but I have fixed them and will commit it. Thank you for your interest in the Issue in RedAmber!

Closes #260 .

Appreciate your corrections @heronshoes !


v.0.4.7 to Production

See commit details for more info.

Hello @mikkokotila! Thanks for submitting the PR.

In the file setup.py, following are the PEP8 issues :

Line 68:80: E501 line too long (85 > 79 characters)
Line 69:80: E501 line too long (84 > 79 characters)
Line 74:80: E501 line too long (81 > 79 characters)

In the file talos/__init__.py, following are the PEP8 issues :

Line 19:1: W191 indentation contains tabs
Line 19:2: E117 over-indented
Line 21:1: W191 indentation contains tabs
Line 21:2: E117 over-indented
Line 22:1: W191 indentation contains tabs
Line 22:1: W293 blank line contains whitespace
Line 23:1: E402 module level import not at top of file

In the file talos/commands/evaluate.py, following are the PEP8 issues :

Line 17:52: W291 trailing whitespace
Line 28:22: E251 unexpected spaces around keyword / parameter equals
Line 28:24: E251 unexpected spaces around keyword / parameter equals
Line 29:28: W291 trailing whitespace
Line 41:80: E501 line too long (80 > 79 characters)
Line 45:80: E501 line too long (82 > 79 characters)
Line 46:80: E501 line too long (81 > 79 characters)
Line 46:82: W291 trailing whitespace
Line 53:80: E501 line too long (86 > 79 characters)
Line 55:35: W291 trailing whitespace
Line 75:41: E203 whitespace before ','
Line 76:1: W293 blank line contains whitespace
Line 79:41: E203 whitespace before ','
Line 80:1: W293 blank line contains whitespace
Line 82:61: W291 trailing whitespace
Line 83:41: E203 whitespace before ','
Line 83:80: E501 line too long (81 > 79 characters)
Line 88:1: W293 blank line contains whitespace

In the file talos/scan/scan_finish.py, following are the PEP8 issues :

Line 51:1: W293 blank line contains whitespace

In the file talos/scan/scan_round.py, following are the PEP8 issues :

Line 34:80: E501 line too long (86 > 79 characters)
Line 35:80: E501 line too long (91 > 79 characters)
Line 38:80: E501 line too long (130 > 79 characters)
Line 49:80: E501 line too long (94 > 79 characters)


[PE-D][Tester D] Phone number length not limited

Currently there is no limit to the phone number length, despite in real life, there should be some limit (15-20 characters max).

Labels: severity.Low type.FeatureFlaw
original: shayaansultan/ped#10

Not a bug


Enhancement: Add link to the concurrency limit page

Concurrency limit pages have been added in Prefect and Cloud. This PR adds a link to those pages from the limits table.

Actually - there merge conflicts and changes in this that I need to resolve before we merge. Do need to review yet!


Inline values provider isn't called when a variable is set

from https://github.com/microsoft/vscode/issues/105690#issuecomment-804710058

So the gist of the issue is: Inline Values are not refreshed when a variable is set

yes, correct.
I think we never did this.

I know @testforstephen has tried this out and confirmed it is good now -> verified

yes, it's fixed in insider.


failed to save input

unit tZNNJUyXiVHeUEoidPU9tO1GtQ/Xejc1KNWET1k6qSo=  and unit Vx1J841Nfy1aQ90mgs9zenkNECYayCVs4M8sXpyw10s= conflict in save inputs
call = "ws.find_and_handle_joints_that_are_ready(&mut db, None)"
err = SqliteFailure(Error { code: ConstraintViolation, extended_code: 2067 }, Some("UNIQUE constraint failed: inputs.src_unit, inputs.src_message_index, inputs.src_output_index, inputs.is_unique"))

@joyious
Vx1J841Nfy1aQ90mgs9zenkNECYayCVs4M8sXpyw10s= is a double spend unit, and we need to wait the validate message implementation #178


Fix warnings

Fixed everything VS complained about.

Oops, getting some socket errors I haven't seen before. Sit tight.


Dataset _init_from_sample - allow np.float32

Description
I'm building a Dataset from files on disk using the Sequence interface, for a machine with very limited memory.
in _init_from_np2d here:
https://github.com/microsoft/LightGBM/blob/01568cf59a412de7823b373fe13eb36db9cffe53/python-package/lightgbm/basic.py#L1597
the np array can be either float64 or float32 which is nice for keeping data smaller with float32 if possible.
However in _init_from_sample here:
https://github.com/microsoft/LightGBM/blob/01568cf59a412de7823b373fe13eb36db9cffe53/python-package/lightgbm/basic.py#L1280
it seems np.double (== float64 on most machines I think?) is enforced.
Not sure if this is intentional and np.double is required, or if this could be changed to float64 | float32 like above?

@jaymegordo Hey thanks a lot for using LightGBM and for raising this question!
np.double is intentional in that case because sample_data is then passed into LGBM_DatasetCreateFromSampledColumn() C API function which accepts only float64 data

double **sample_data
https://lightgbm.readthedocs.io/en/latest/C-API.html#c.LGBM_DatasetCreateFromSampledColumn

https://github.com/microsoft/LightGBM/blob/01568cf59a412de7823b373fe13eb36db9cffe53/python-package/lightgbm/basic.py#L1285-L1287
https://github.com/microsoft/LightGBM/blob/01568cf59a412de7823b373fe13eb36db9cffe53/python-package/lightgbm/basic.py#L1291-L1292
https://github.com/microsoft/LightGBM/blob/01568cf59a412de7823b373fe13eb36db9cffe53/python-package/lightgbm/basic.py#L1300-L1301
I believe your feature request is reasonable and we can unify creating Dataset from sampled column with other similar functions for Dataset creation by adding new argument data_type.
Hey @cyfdecyf could you please help with this?

@StrikerRUS thanks, would be really appreciated!
FYI I just changed the following lines:
# if sample_data[i].dtype != np.double:
if sample_data[i].dtype not in (np.float32, np.float64):

and
# sample_col_ptr = (ctypes.POINTER(ctypes.c_double) * ncol)()
sample_col_ptr = (ctypes.POINTER(ctypes.c_float) * ncol)()

And it looks like I can create the dataset with np.float32 data with no errors, but not really sure if that may cause any other issues somewhere else.

I believe your feature request is reasonable and we can unify creating Dataset from sampled column with other similar functions for Dataset creation by adding new argument data_type and allowing data to be either float32 or float64.

Just took a look. The difficulty is in DatasetLoader::ConstructFromSampleData which currently accepts only double.
https://github.com/microsoft/LightGBM/blob/83a41dabec3c97ae99a4820721e23d7422e7144e/src/c_api.cpp#L992
https://github.com/microsoft/LightGBM/blob/83a41dabec3c97ae99a4820721e23d7422e7144e/src/io/dataset_loader.cpp#L643-L645

```python
# sample_col_ptr = (ctypes.POINTER(ctypes.c_double) * ncol)()
sample_col_ptr = (ctypes.POINTER(ctypes.c_float) * ncol)()

And it looks like I can create the dataset with np.float32 data with no errors, but not really sure if that may cause any other issues somewhere else.

Please refer to my previous comment. I just took a quick look, I think this is problematic because the underlying DatasetLoader::ConstructFromSampleData is expecting double**.

I'm not sure if there's anyone else can pick up this. If not, I can do it after the main work for v4.0.0 is done.

@shiyu1994 Sounds cool, thanks!

A good trick around this is to store the dataset in float32 and convert it to float64 while fetching the data:
class HDFSequence(lgb.Sequence):
    def __init__(self, hdf_dataset):
        self.data = hdf_dataset
        self.batch_size = 2048

    def __getitem__(self, idx):
        return self.data[idx].astype('float64')

    def __len__(self):
        return len(self.data)

I'm not sure how this impacts performance or training time, but it at least trains.


Error in "ksync create" on windows

when i am running:
ksync create --selector=app=app $(pwd)\ksync /code
i am getting this error:
FATA[0000] remote path must be absolute
i thinks because i am running on windows ksync does not recognize /code as a valid path...
so to overcome this i ran this command:
ksync create --selector=app=app $(pwd)\ksync c:\code  and it worked!
but now i needed to change the remotepath key manually in the ksync.yaml file to:
...
localpath: C:\ksync
remotepath: **/code** (from c:\code)
...

after the change i saw syncthing logged the change i made in the ksync.yaml file and it's also create a log every time i make a change in the C:\ksync directory, but it doesn't sync the /code directory...
i also tried to do ksync get , but still the folders haven't synced.
any idea how to make the folder to sync, maybe i need to modify the remotepath key differently?

@gadavivi You'll likely be finding a few of these, we largely rely on the standard lib for cross platform functionality.
Have you tried specifying the paths as non-interpolated strings?
ksync create -lapp=some-app 'local\path\' '/remote/path'
You can just edit the yaml directly, but you won't be getting any validation. You'll also need to restart the syncthing process (stop and rerun watch) to have it pick up the changes.

if i am specifying non-interpolated strings i am getting FATA[0000] remote path must be absolute
i tryed to changed the yaml file as below:
`apikey: ksync
context: ""
log-level: info
namespace: dok
port: 40322
spec:

name: loved-tapir
containername: ""
pod: ""
selector: app=stock-con
namespace: dok
localpath: C:\Users\gadavivi\ksync
remotepath: /code
reload: true
localreadonly: false
remotereadonly: false
syncthing-port: 8384`

i restarted the syncthing process after changing the .yaml file, but still the sync is not happing (i am not getting an error, everything is running, but the sync not happening)

the error is here
in a windows machine the filepath.IsAbs act differently...
but i still have no idea why after changing the yaml file manually, the sync doesn't happen...

@gadavivi Try the steps I outlined above.

i did a try to with an ubuntu machine (trough virtualbox) against a kubernetes cluster in aws and everything worked fine, so for sure there is some problems with windows :(

@gadavivi That's unfortunate. Unfortunately, we haven't been able to do user testing (or any specific testing) for Windows since none of us use it, so you're a bit of guinea pig on this one, sorry about that.
I'll try and get a Windows box going and track down what's going on. For whatever reason, the spec isn't getting processed into syncthing's config.
ping @grampelberg

Ok thanks!
by the way  do you have a guess where might be the problem? (i will try to dig in on my spare time)

@gadavivi I can't look at the code to verify this, but my guess that assumptions were made in writing the config.xml file that syncthing uses (i.e. it uses some string paths that don't work in Windows).
It could be in the path writing itself, or the serialization in grpc. I suspect the latter, but again this is just a guess. That code is probably in pkg/ksync/syncthing/server.go.

ok,
i did some digging and i found the problem (but i don't know where it exactly in the code):
because i am running on windows the go library that deals with paths translate paths to this\is\my\path, but when running on linux the go paths library  are translate correctly to this/is/my/path.
this cause ksync to do a wrong mapping in the syncthing container when running on windows.
I found this when in the ksync example i logged to the syncthing container and saw this:
/ # ls
\var\lib\docker\overlay2\ec73f389284c39ee852cd9045cd1ffda802e62a9f07c2a4fa1fe6fea78068558\merged\code  root
bin                                                                                run
dev                                                                                sbin
etc                                                                                srv
home                                                                                syncthing
lib                                                                                sys
media                                                                                tmp
mnt                                                                                usr
proc                                                                                var

as you can see because ksync uses the windows mapping, the syncthing container got a bad interpenetration (see the first path above from the ls command) to the folder that it need to sync (it also created the folder because of course it doesn't exits).
so because that folder does not really map to the correct /code directory in the desired container, my files didn't synced.
P.S: When i created a file in this bad directory, my local machine got sync immediately :), so when this bad path interpenetration will get fix, everything is supposed to work.
may you point where this problem in the code?

@gadavivi As expected. This is the problem I outlined earlier. You can see (from our CI config) that we compile for each platform/arch in turn. When go builds for a specific arch, it substitutes any different libraries in at compile time based on the GOOS. This means that in the case of windows, the compiler uses the windows version of the path library. That means that all paths evaluated by the client-side ksync binary will use this library.
So the problem is this: When using a different client platform from that of the cluster both the remote path and the local path get evaluated by the same client binary. We'll maybe need to figure out a way to differentiate.
It's possible we should just do the remote evaluation on the remote, since I don't think you could run Windows in a cluster remotely.

@gadavivi Can you test cb560ca to see if that solved the create problem for you?
Windows x64 binary ksync

The create work , but the sync doesn't...
i still have the same problem, that in the synchthing container the config.xml doesn't contain a valid linux path...
i added the xml file from the syncthing container (you will see that the path element is from a windows format)
config.xml.txt

Damn. It seems they fixed the "bug" I was hoping to exploit. On to the next idea.

@gadavivi From the same run, did the spec show the correct paths for local and remote?

@gadavivi Ok, try this. Sorry to keep making you try things, but I don't have a way to test Windows binaries :(
https://738-105306631-gh.circle-artifacts.com/0/binaries/ksync_windows_amd64.exe

it's ok, i am more then happy to help, also really appreciate your help :)
by the way, the above binary is good, now the sync and the create command are working :)

Great! 👍 Thanks for testing. I'll get this merged in.

I tried with ksync version 0.3.5, but I'm getting a similar problem.
With single quotes around local path:
$ ksync create --name my-app --namespace default -l app=my-app --reload=false --force C:\Users\pieter\minikube\sources\my-app /var/www/app
time="2019-03-12T17:30:01+01:00" level=fatal msg="local path must be absolute"

Without single quotes around local path:
$ ksync create --name my-app --namespace default -l app=my-app --reload=false --force 'C:\Users\pieter\minikube\sources\my-app' /var/www/app
time="2019-03-12T17:30:12+01:00" level=fatal msg="remote path must be absolute"

Is there some kind of regression?

@pietervogelaar Did you paste those backwards?


customize redis role to our needs

This removes the bennojoy.redis submodule and replaces it with a tailored version which only handles ubuntu.
The only addition is to add a task to add the ppa:chris-lea/redis-server apt repo so that we get the latest version of redis.
@NoahCarnahan
cc @TylerSheffels

@snopoke Looks great, unmergeable now though.

Is using an "unofficial" ppa a security concern?

Potentially I guess though this one in particular is used and referenced
quite a lot.
On 29 July 2015 at 17:09, Noah Carnahan<EMAIL_ADDRESS>wrote:

Is using an "unofficial" ppa a security concern?
—
Reply to this email directly or view it on GitHub
https://github.com/dimagi/commcarehq-ansible/pull/168#issuecomment-125983301
.

--
Simon Kelly
Senior Engineer | Dimagi South Africa

Sounds wrong to say this, but, we already use a couple unofficial PPA's. Don't know if that makes it any better or us more comfortable. It's probably doable to find the md5 on the redis site and test the download as well.


Add bots blocked by Tumblr

I was looking at the robots.txt on my Tumblr-hosted blog: https://blog.corbin.io/robots.txt
There are a few bots Tumblr is blocking that aren't in this list:
# Common Crawl's crawler
User-agent: CCBot
Disallow: /

# SentiBot's crawler
User-agent: sentibot
Disallow: /

# ImageSift's AI crawler
User-agent: ImagesiftBot
Disallow: /

# TurnitinBot crawler
User-agent: TurnitinBot
Disallow: /

Not sure if all of these should be blocked or just some of them.

Hey! Thanks for calling this out! We've got CCBot included and I added ImageSiftBot. I don't believe sentibot or TurnitinBot are strictly related to AI so I'll hold off for now (I believe the former is sentiment analysis and the latter is for plagiarism detection).

If I'm right, the sentibot belongs to https://sentione.com/ and that looks a bit like AI.


Do not search elements from inactive lists

Overview
Do not include results from inactive lists in search.
Connects #396
Demo

Testing Instructions

Check out this branch and server
Follow the instructions in #396 to try and recreate the bug

[x] Ensure you see only one facility, not two

Thanks for taking a look!


Extend this directory for Data Structures in Java Language

I really like the initiative of this repo. I want to extend it furthermore so that students proficient in the java language can use this.
this is what ill do..

[X] Create a new directory with name "Data structures in Java"
[X] Add implementation of LinkedList in it (java file)
[X] Add implementation of BinaryTree in it (java file)
[X] Create another directory in it where people can add top DSA questions in java

If my idea sounds good, then kindly assign me to this and add hacktoberfest tags to it.

@KrishJani permission granted proceed as mentioned

I would like to add some algorithms implemented in Java. Some of the algorithms I will be adding are:-

Binary Search (For ascending as well as descending sorted array)
Linear Search
Selection Sort
Bubble Sort

If you would like me to work on this then please assign me to this issue as well


Automically is spelled wrong ("automaticly") in export dataset modal

My actions before raising this issue

[x] Read/searched the docs
[x] Searched past issues

Expected Behaviour

When exporting a dataset, a modal should appear that states "Download will start automatically as soon as the dataset is ready."
Current Behaviour

When exporting a dataset, a modal should appear that states "Download will start automaticly as soon as the dataset is ready."
"Automatically" is spelled wrong in the current state.
Possible Solution

Change "automaticly" to "automatically" in cvat-ui/src/components/export-dataset/export-dataset-modal.tsx.
Steps to Reproduce (for bugs)

Create a task with a dataset
Click "Open" on the task
Open the "Actions" menu
Click "Export task dataset"
Click "Ok"
A modal will appear in the top-right corner of the screen with the misspelling

Context

Obviously, this doesn't have a large impact. I am trying to fix a spelling error that I see.
Your Environment

The live website, cvat.org.
Next steps
You may join our Gitter channel for community support.

Closed by the PR above


Update diffusers to 0.26.0 and fix imports/names

What type of PR is this? (check all applicable)

[ ] Refactor
[ ] Feature
[X] Bug Fix
[X] Optimization
[ ] Documentation Update
[ ] Community Node Submission

Have you discussed this change with the InvokeAI team?

[ ] Yes
[X] No, because I don't think this is needed

Have you updated all relevant documentation?

[ ] Yes
[ ] No

Description
A new version of diffusers has just released yesterday with some major refractions. I tried to catch all imports, but it is possible that I missed some. The old imports will be deprecated in version 0.29.0, so I tried to find them all and change them while I am at it.
If you like, please look if I missed some somewhere.
There seem to be some scripts that might use the old imports. But as I said, for now, there is nothing breaking.
Fixed the hotfix.py for a name change (fix?) introduced upstream here
Everything seems to be running fine for me. Invoke is generating images! However, I would like you guys to test it first because of some refactoring changes that I might have missed.
Thank you very much and kind regards.
Related Tickets & Documents

Related Issue #
Closes #

QA Instructions, Screenshots, Recordings

Merge Plan

Added/updated tests?

[ ] Yes
[ ] No : please replace this line with details on why tests
have not been included

[optional] Are there any post deployment tasks we need to perform?

Can you fix style checks? Thanks for this update!

Can you fix style checks? Thanks for this update!

How and which one? I don't know what's wrong :/

Fixed 👍

Works on Mac!


std.getopt: two little features to aid rdmd

As outlined in https://github.com/D-Programming-Language/tools/pull/140 which aims to fix https://issues.dlang.org/show_bug.cgi?id=13290:

endOfOptionsPred: stop processing on a predicate (rdmd: stop on source and executable file names)
GetoptResult.stoppedAt: distinguishing unrecognized options from non-options (rdmd: dmd args vs program args)

handles the argument, of course;
can signal if the argument is to be removed from the array; and
can stop argument processing right there.

That sounds almost like a foreach loop, how about this.
foreach (arg, ref keep; getopt(args, "v|verbose", &verbose))
{
    if (arg == "--") break;
    keep = false;
}

Another problem with getopt is that it doesn't support help text.
Something like this might work.
getopt(args,
    opt("v|verbose", &verbose, "print verbose output")
);

Then getopt could be used to generate a help menu.

That sounds almost like a foreach loop, how about this.

I had thought that way too:
NilsBossung commented on 3 Sep 2014:

It would be possible to expand on the idea of getOneOpt, separating the option specificiation from the popping, maybe turn it into a range. But it would still be out of line with the rest of getopt which works with callbacks.

Another problem with getopt is that it doesn't support help text.

It does actually.

How is that? Looks like you still need to duplicate each option in the usage message by hand.

How is that? Looks like you still need to duplicate each option in the usage message by hand.

import std.getopt;
void main(string[] args)
{
    bool verbose;
    auto r = getopt(args, "v|verbose", "print verbose output", &verbose);
    if(r.helpWanted) defaultGetoptPrinter("foo", r.options);
}

Run it with --help and it prints:
foo
-v --verbose print verbose output
-h    --help This help information.

See also: 'Synopsis' of http://dlang.org/phobos-prerelease/std_getopt#.getopt


Introduce new roles/teams in kedaorg

Currently, we are only the maintainers who have the approval/merge permissions and we are the only included in codeowners. This doesn't scale, and we should introduce an intermediate level (reviewer, approver, contributor, whatever) between reviewing PRs without any permission and merging permissions.
My proposal is to include a new team with approval permissions on repositories (one group per repo on those repos where it makes sense) and include them as part of codeowners in order to notify them automatically when there are new PRs.
The permissions that I'd add to this new team are:

codeowners -> To be notified on PRs
issue triagge -> We need help triaging issues and it can be useful for the project
pr approval (not merge) -> Having more folks whose approval can be usable for merging can improve the project. e.g: zbynek creates a PR but Tom or me can't review it. Recurrent contributors have enough context and knowledge to review it and approve it, and zbynek could merge it once it's approved.

This could be something in the middle to not have to jump from "nothing" to maintainer. This can be also useful for us as probably new maintainers will be found on those teams (or eventually project scoped maintainers, idk)
If you also think that it's useful, I'm willing to draft a PR defining the new roles/teams, the scopes, expectations and the way to be invited to them.
@tomkerkhove  @zroubalik ?

Do you think that drafting the team definition can be useful to discuss the edges over it can be useful? As this has to be reflected on this repo for transparency, I guess that I can draft the description in a PR

Yes we should, we also need this for Credly

I'll try to draft something this week

These are  the CNCF roles which are also what they offer for Credly, I'd align with this given this is fairly standard in CNCF:

Okey, for the moment I think that I'll draft just "contributor" to cover the reviewer that I proposed, but we can iterate from it in the future to add the others


Another IB login page change?

Describe the bug
I am suspicious that IB has changed their login page again, breaking compatibility with what IBeam is expecting.
The system was working OK on 2023-06-16, and failing on 2023-06-19.  I'm seeing a message that makes me think IBeam can no longer recognize / load the login form.
2023-06-20 13:47:39,965|I| No active sessions, logging in...
2023-06-20 13:47:39,965|I| Loading auth webpage at https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on
2023-06-20 13:48:03,363|W| Cannot determine the version of IBKR website, assuming version 1
2023-06-20 13:48:23,773|E| Timeout reached when waiting for authentication. The website seems to not be loaded correctly. Consider increasing IBEAM_PAGE_LOAD_TIMEOUT.
Website URL: https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on
IBEAM_PAGE_LOAD_TIMEOUT: 15
Exception:
  File "/usr/local/lib/python3.7/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 80, in _worker
    work_item.run()
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/venv/lib/python3.7/site-packages/apscheduler/executors/base.py", line 125, in run_job
    retval = job.func(*job.args, **job.kwargs)
  File "/srv/ibeam/ibeam/src/gateway_client.py", line 360, in _maintenance
    success, shutdown = self.start_and_authenticate(request_retries=var.REQUEST_RETRIES)
  File "/srv/ibeam/ibeam/src/gateway_client.py", line 332, in start_and_authenticate
    success, shutdown = self.try_authenticating(request_retries=request_retries)
  File "/srv/ibeam/ibeam/src/gateway_client.py", line 256, in try_authenticating
    success, shutdown = self._authenticate()
  File "/srv/ibeam/ibeam/src/gateway_client.py", line 233, in _authenticate
    two_fa_handler=self.two_fa_handler)
  File "/srv/ibeam/ibeam/src/authenticate.py", line 261, in authenticate_gateway
    WebDriverWait(driver, 15).until(user_name_present)
  File "/opt/venv/lib/python3.7/site-packages/selenium/webdriver/support/wait.py", line 95, in until
    raise TimeoutException(message, screen, stacktrace)

To Reproduce
I restarted IBeam and the logs indicated the above issue.
I then tried visiting the port 5000 page directly and pulled a login page down, which I filled in an submitted successfully.
After that IBeam indicated it was successfully logged in.
Expected behavior
I would have expected IBeam to navigate the login page.
Environment
IBeam version: v0.4.5
Docker image or standalone: Docker image
Python version (standalone users only):
OS: Linux
Additional context
I notice that the login page now has a large javascript component, I'm unclear on whether or not it had this before:


cached token prices and apr volume api calls

Description
How has this been tested?
Types of changes

[ ] Technical Debt (non-breaking change which removes unused code/assets)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] The related ClickUp task has been linked to this PR
[ ] The person creating the pull request is listed in "Assignees"
[ ] Change requires updated documentation

Screenshots or Loom Video (optional):

Task linked: CU-2r6c6y9 API Cache farm volume and APR calls

Task linked: CU-2h1a5xe UI: Farm page must call apr and volume only once


[Feature] Void fields displaying

Reported by vkotkas on 2009-10-20 15:14 UTC
void fields are not displayed in the properties window. Having a void field in a specification is useful for synthesis guidance. One may want to make a void field being a goal through a user interface.

Commented by pgrigorenko on 2009-10-20 20:35 UTC

Commented by pgrigorenko on 2009-10-20 20:38 UTC
it is displayed and could be marked as a goal, see screenshot

Updated by pgrigorenko on 2009-10-20 20:38 UTC

labels:  --> User Interface Improvements
status: open --> closed


Build C-Ares with Bazel on Windows

With this PR, gRPC can be built with Bazel on Windows without --define grpc_no_ares=true flag.
config_windows/ares_config.h is generated by CMake.

Ping @apolcyn

getting to this in the next day

this mostly lgtm, but one thought is this would have us using the generated ares_config.h when building on Windows under bazel, and using the HAVE_CONFIG_H ares macro to do so, which would be inconsistent with how we're not using a pre-generated ares_config.h header when using the Windows cmake build, though. I'm thinking it would make sense to supply the custom ares_config.h files to cmake to match up with this (perhaps in a separate PR though). cc @jtattermusch

using the generated ares_config.h when building on Windows under bazel, and using the HAVE_CONFIG_H ares macro to do so, which would be inconsistent with how we're not using a pre-generated ares_config.h header when using the Windows cmake build,

Not sure where the inconsistency comes from. The other 5 config_*/ares_config.h files were also generated by CMake. Until C-Ares is updated to a newer version again, CMake build and Bazel build on Windows should be identical (in worst case, only as bad as other 5 platforms).

Not sure where the inconsistency comes from. The other 5 config_*/ares_config.h files were also generated by CMake. Until C-Ares is updated to a newer version again, CMake build and Bazel build on Windows should be identical (in worst case, only as bad as other 5 platforms).

Sorry, nevermind that :) you're right

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences


[Docs] Update grammar/capitalization on the landing page for the docs site

This should read:

"A Set of React Components that Implements Google's Material Design"

Read it without the prepositional phrase "of React Components" to hear it better

Maybe the capitalization can be adjusted, as well. It is not a title, so using a normal sentence capitalization scheme would be preferable to me.

"A set of React components that implements Google's Material Design"

But it's not the set that implements Material Design, it's the components that implement it, so the "s" is correct. :thinking:
That said, the title case should be changed, it looks weird…

The subject of the sentence is the word set. If we want to reword the sentence, that is fine, but it is incorrect as written.

Even though it sounds odd, I believe @m2mathew is technically correct on the grammar.
I was going to suggest dropping "A set" to satisfy leMaik, so:

"React components that implement Google's Material Design"

then noticed @oliviertassinari had added the legacy tag, so went to see what we had on v1-beta. Lo and behold (!):

"React Components that Implement Google's Material Design."

(We still need to fix the casing there.)

I will update the legacy real quick. That @oliviertassinari...so smart!

v0.x and v1.x have been fixed :)


Discontinued Course - Machine Learning

Another course is being deprecated per the screenshot below.

The link in the email pointing to a description leads to this website.

The link in the curriculum now goes to this course:
https://www.coursera.org/learn/machine-learning
This is the first of 3 new courses by Ng. There's a quick overview of the differences from Class Central: https://www.classcentral.com/report/coursera-sunsets-machine-learning/
I'm going to close this issue, as there is not an actionable change proposed.
I will point out for those who want to propose a replacement for this course, CS2013 recommends that students learn about search...
• Problem spaces (states, goals and operators), problem solving by search
• Factored representation (factoring state into variables)
• Uninformed search (breadth-first, depth-first, depth-first with iterative deepening)
• Heuristics and informed search (hill-climbing, generic best-first, A*)
• Space and time efficiency of search
• Two-player games (introduction to minimax search)
• Constraint satisfaction (backtracking and local search methods)
...before learning about the general categories of learning (supervised, unsupervised) and general problems in learning (overfitting).
A course covering these issues well would be a strong candidate to replace Machine Learning in the core curriculum.

This course looks like it may be more narrowly tailored to the introductory ML expectations of CS2013: https://www.coursera.org/learn/practical-machine-learning


bug(Datepicker): Top left calender button has unclear purpose

Using the DatePicker for selected years and months. Typically I guess the user wants to use the month and year, and never the day.
So by using the startView property to multi-year you will see the month and year in the select field:

Once you click on the icon the pop-up opens you are able to change the date.
Current outcome:
Here comes the bug. When you click on the button on the left top:

You will be able to select a day:

Why would you want to enable the user to select a day in a month and year calendar?
Expected outcome:
I was expecting instead that you would see the month that was selected. And if you'd click on it you would be sent to that year showing all the months.
When setting the startView to year the button entirely does not need to be there. Or am I misinterpreting this whole thing?

Since you can use the arrow keys you can disable the calendar control button for the year view:
::ng-deep .date-picker .mat-calendar-period-button { display: none; }

But for the month/year datepicker this won't be a good solution since you'd like to be able to switch between the month and year view.

Right now this is working as designed, at least. We don't officially support only month/year selection; the startView property only controls which view is the first thing rendered, but users can always go back to the individual day selection.
The feature to only have month/year selection is tracked in #14202

This worked for me:
https://stackblitz.com/edit/angular-z2zfjx-rwkbsd?file=app%2Fdatepicker-custom-header-example.ts
Just change line 91 from
this.calendar.currentView == "month" ? "multi-year" : "month";
to
this.calendar.currentView == "year" ? "multi-year" : "year";

Using the DatePicker for selected years and months. Typically I guess the user wants to use the month and year, and never the day.
So by using the startView property to multi-year you will see the month and year in the select field:

Once you click on the icon the pop-up opens you are able to change the date.
Current outcome:
Here comes the bug. When you click on the button on the left top:

You will be able to select a day:

Why would you want to enable the user to select a day in a month and year calendar?
Expected outcome:
I was expecting instead that you would see the month that was selected. And if you'd click on it you would be sent to that year showing all the months.
When setting the startView to year the button entirely does not need to be there. Or am I misinterpreting this whole thing?

This worked for me:
https://stackblitz.com/edit/angular-z2zfjx-rwkbsd?file=app%2Fdatepicker-custom-header-example.ts
Just change line 91 from
this.calendar.currentView == "month" ? "multi-year" : "month";
to
this.calendar.currentView == "year" ? "multi-year" : "year";


Hard line breaks don't work

Unable to use Hard Line Breaks
 works fine
\n Does not work

Perhaps you've already found this workaround, but my issue was that the markdown came in as part of a JSON response to I had to replace the newlines using this .replaceAll("\\n", "\n") at which point vue-markdown treated everything like I expected.


Run the integration tests in headless Chrome

Puppeteer recently added headless support for running Chrome
extensions[1], so let's make use of that. Also, let's take care to
properly wait for the extension's background page to load.
1 - https://github.com/puppeteer/puppeteer/pull/8260
Reviewer: @jonathanKingston
CC: @kdzwinel, @ladamski
Steps to test this PR:
Verify that integration tests are still passing both on CI and locally.
Automated tests:

[ ] Unit tests
[x] Integration tests

Reviewer Checklist:

[ ] Ensure the PR solves the problem
[ ] Review every line of code
[ ] Ensure the PR does no harm by testing the changes thoroughly
[ ] Get help if you're uncomfortable with any of the above!
[ ] Determine if there are any quick wins that improve the implementation

PR Author Checklist:

[ ] Get advice or leverage existing code
[ ] Agree on technical approach with reviewer (if the changes are nuanced)
[ ] Ensure that there is a testing strategy (and documented non-automated tests)
[ ] Ensure there is a documented monitoring strategy (if necessary)
[ ] Consider systems implications

(See also my suggested changes to the Puppeteer docs: https://github.com/puppeteer/puppeteer/pull/8321.)


IngressDNSRecord not receiving FederatedIngress IP addresses

Following this guide: https://github.com/kubernetes-sigs/kubefed/blob/master/docs/ingressdns-with-externaldns.md
This step is failing:

The Ingress DNS Controller watches for IngressDNSRecord resources and updates the status of the resource with the IP address of matching target Ingress resource(s).

I expect the IngressDNSRecord to have ip addresses in status.dns[].loadBalancer, instead it shows loadBalancer: {}
There's a repeated error in kubefed-controller-manager but I'm not sure its related, it's been throwing all day.
sigs.k8s.io/kubefed/pkg/controller/util/federated_informer.go:432: watch of <nil> ended with: too old resource version: 60737 (62023)

I ran this command to create the IngressDNSRecord
$ cat <<EOF | kubectl create -f -
apiVersion: multiclusterdns.kubefed.k8s.io/v1alpha1
kind: IngressDNSRecord
metadata:
  name: test-ingress-dns-record
spec:
  hosts:
  - test.arc.io
  recordTTL: 300
EOF

The resulting IngressDNSRecord
apiVersion: multiclusterdns.kubefed.k8s.io/v1alpha1
kind: IngressDNSRecord
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"multiclusterdns.kubefed.k8s.io/v1alpha1","kind":"IngressDNSRecord","metadata":{"annotations":{},"name":"test-ingress-dns-record","namespace":"default"},"spec":{"hosts":["test.arc.io"],"recordTTL":300}}
  creationTimestamp: "2019-06-03T22:49:57Z"
  generation: 1
  name: test-ingress-dns-record
  namespace: default
  resourceVersion: "58319"
  selfLink: /apis/multiclusterdns.kubefed.k8s.io/v1alpha1/namespaces/default/ingressdnsrecords/test-ingress-dns-record
  uid: e910749e-8651-11e9-b225-42010a8a0123
spec:
  hosts:
  - test.arc.io
  recordTTL: 300
status:
  dns:
  - cluster: arc-us-east4
    loadBalancer: {}
  - cluster: arc-us-west1
    loadBalancer: {}

FederatedIngress
apiVersion: types.kubefed.k8s.io/v1beta1
kind: FederatedIngress
metadata:
  creationTimestamp: "2019-06-03T22:48:59Z"
  finalizers:
  - kubefed.k8s.io/sync-controller
  generation: 1
  name: ingress-nginx
  namespace: default
  resourceVersion: "58301"
  selfLink: /apis/types.kubefed.k8s.io/v1beta1/namespaces/default/federatedingresses/ingress-nginx
  uid: c69f2c3a-8651-11e9-bcd7-42010a8a0124
spec:
  placement:
    clusterSelector:
      matchLabels: {}
  template:
    metadata:
      annotations:
        kubernetes.io/ingress.class: nginx
        nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    spec:
      rules:
      - host: test.arc.io
        http:
          paths:
          - backend:
              serviceName: hello-kubernetes
              servicePort: 80
      tls:
      - hosts:
        - test.arc.io
        secretName: letsencrypt-secret
status:
  clusters:
  - name: arc-us-west1
  - name: arc-us-east4
  conditions:
  - lastProbeTime: "2019-06-03T22:49:55Z"
    lastTransitionTime: "2019-06-03T22:48:59Z"
    status: "True"
    type: Propagation

Ingress 1 ip
$ kubectl get ing --context arc-us-west1

NAME            HOSTS         ADDRESS         PORTS     AGE
ingress-nginx   test.arc.io   <IP_ADDRESS>   80, 443   19m

Ingress 2 ip
$ kubectl get ing --context arc-us-east4

NAME            HOSTS         ADDRESS         PORTS     AGE
ingress-nginx   test.arc.io   <IP_ADDRESS>   80, 443   11m

Let me know if you need more info.

/triage support

I have re-run the examples and found they are working as expected. From your issue report i could point that the issue is related to the error in the log
sigs.k8s.io/kubefed/pkg/controller/util/federated_informer.go:432: watch of <nil> ended with: too old resource version: 60737 (62023)

which basically means that the watch on the resource in member cluster is not working.
Please give us additional information about.

what is your kubefed codebase? (release)
what is the k8s version of member cluster. Particularly, what is the GVK of ingress resource in member cluster. The informer we are using is using the type as in here

I used helm to install the chart version kubefed-0.1.0-rc2. helm list only displays the chart version, not the app version.
The k8s version of the member cluster is 1.13.6-gke.0

For the GVK, is that the apiVersion + kind? If so, the GVK of the ingress resource is extensions/v1beta1/Ingress

Does "of matching target Ingress resource(s)" imply that the names have to be the same?

Yes, that's correct and sorry for the confusion created by the doc. Shall update the user guide to clear this.

Thanks for the clarification.


chore: remove pre-commit from dev dependency

:tada: This PR is included in version 5.10.0 :tada:
The release is available on:

v5.10.0
GitHub release

Your semantic-release bot :package::rocket:


[Multi Configs] Use directories to differentiate between subsets/configurations

Currently to define several subsets/configurations of your dataset, you need to use a dataset script.
However it would be nice to have a no-code way to to this.
For example we could specify different configurations of a dataset (for example, if a dataset contains different languages) with one directory per configuration.
These structures are not supported right now, but would be nice to have:
my_dataset_repository/
├── README.md
├── en/
│   ├── train.csv
│   └── test.csv
└── fr/
    ├── train.csv
    └── test.csv

Or with one directory per split:
my_dataset_repository/
├── README.md
├── en/
│   ├── train/
│   │   ├── shard_0.csv
│   │   └── shard_1.csv
│   └── test/
│       ├── shard_0.csv
│       └── shard_1.csv
└── fr/
    ├── train/
    │   ├── shard_0.csv
    │   └── shard_1.csv
    └── test/
        ├── shard_0.csv
        └── shard_1.csv

cc @stevhliu @albertvillanova

I want to be able to create folders in a model.

How to set new split names, instead of train/test/validation? For example, I have a local dataset, consists of several subsets, named "A", "B", and "C". How can I create a huggingface dataset, with splits A/B/C ?
The document in https://huggingface.co/docs/datasets/dataset_script only tells me how to create datasets with subsets that is hosted on another server. How to do it if my datasets are local?

The document in https://huggingface.co/docs/datasets/dataset_script only tells me how to create datasets with subsets that is hosted on another server. How to do it if my datasets are local?

It works the same - you just need to use local paths instead of URLs


Numpy Version

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): windows 10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: no
TensorFlow installed from (source or binary): from source
TensorFlow version: 2.5, 2.6, etc
Python version: 3.9

Where can I find the nupmy version requirements for a Tensorflow build?
Please write requirements for numpy as you write for cuda.

@Expert73
Could you please refer the similar issue #31249,also refer this documentation ,let us  know if it helps.Thanks

Before doing a build from tensorflow source where can I find out the REQUIRED version of numpy?

I would select the appropriate tag (version) in code view then check tensorflow/tools/pip_package/setup.py file for required packages.

I would select the appropriate tag (version) in code view

what do you mean? how to do it?

@Expert73
TensorFlow NumPy is built on top of TensorFlow and hence interoperates seamlessly with TensorFlow. Please refer this documentation and let us know if it helps.Thanks


Aurora Pipeline Fails when NCEDC is down

Parkfield tests fail on gitactions runner because the data and metadata cannot be called when NCEDC is suffering an outage.
First observed on 17 Mar, 2022.
Since NCEDC is not a stakeholder at this point, we cannot expect them to be concerned about this issue.
We could:

make a copy of the data on an IRIS hosted server so that these tests can query IRIS instead of NCEDC.
Alternatively, we could revisit mth5_test_data repo, tidy it up and place test data there.  Note that make_mth5 is already tested by mth5, i.e. the access / transmission of data and metadata is being tested outside aurora.

If revisiting mth5_test_data, the Pooch package maybe of interest:

This happened again on April 23 at 0900 Pacific time.
The error is:
Traceback (most recent call last):
    streams = dataset_config.get_data_via_fdsn_client(data_source="NCEDC")
  File "/home/kkappler/software/irismt/aurora/aurora/sandbox/io_helpers/fdsn_dataset_config.py", line 78, in get_data_via_fdsn_client
    self.endtime,
  File "/home/kkappler/anaconda2/envs/py37/lib/python3.7/site-packages/obspy/clients/fdsn/client.py", line 830, in get_waveforms
    raise ValueError(msg)
ValueError: The current client does not have a dataselect service.

I have attached the hz data from PKD for the time interval that we use for the tests ...
ex, ey, hx, hy are already archived at IRIS.
hz_pkd.csv
@timronan Can you or Laura look at adding this hz data to the IRIS archive?  Then we can set up the tests to use IRIS (or try NCEDC and catch exception use IRIS).

python make_parkfield_mth5.py creates the mth5 file locallly, with both the data and the metadata (from NCEDC).
This file could actually be used as a source of data and metadata that we could push to IRIS, see issue 99 in mth5:
https://github.com/kujaku11/mth5/issues/99

Here's a new one, Sept 2, 2022:
from obspy.clients.fdsn import Client Client(base_url="NCEDC")

Client(base_url="NCEDC")
Traceback (most recent call last):
File "/home/kkappler/software/pycharm-community-2019.1.1/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_exec2.py", line 3, in Exec
exec(exp, global_vars, local_vars)
File "", line 1, in
File "/home/kkappler/anaconda2/envs/py38/lib/python3.8/site-packages/obspy/clients/fdsn/client.py", line 276, in init
self._discover_services()
File "/home/kkappler/anaconda2/envs/py38/lib/python3.8/site-packages/obspy/clients/fdsn/client.py", line 1531, in _discover_services
wadl_parser = WADLParser(wadl)
File "/home/kkappler/anaconda2/envs/py38/lib/python3.8/site-packages/obspy/clients/fdsn/wadl_parser.py", line 28, in init
doc = etree.parse(io.BytesIO(wadl_string)).getroot()
File "src/lxml/etree.pyx", line 3536, in lxml.etree.parse
File "src/lxml/parser.pxi", line 1893, in lxml.etree._parseDocument
File "src/lxml/parser.pxi", line 1913, in lxml.etree._parseMemoryDocument
File "src/lxml/parser.pxi", line 1800, in lxml.etree._parseDoc
File "src/lxml/parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
File "", line 1
lxml.etree.XMLSyntaxError: Space required after the Public Identifier, line 1, column 50

The lxml error is due to NCEDC changing their urls.  See Issue 3134 https://github.com/obspy/obspy/issues/3134

Here is a new one again Dec 2022
Symptoms:

Python 3.6, 3.7 fail due to no inventory returned by NCEDC
Python 3.8, 3.9 Fails in run_ts_obj.from_obspy_stream(streams_dict[station_id], run_metadata)
at the end of the method when calling self.validate_metadata()
message is:
mt_metadata.base.metadata.run.add_channel - ERROR: component cannot be empty

Note the mth5.timeseries.run_ts.RunTS calls self.validate_metadata() twice.  The first time through it passes, but not the second.
The first time through is in the set_dataset method of RunTS.  There is a check of the condition:
self.run_metadata.id not in self.station_metadata.runs.keys()
which is False, because self.run_metadata.id = '0' and self.station_metadata.runs.keys() = ['0',], so the
self.station_metadata.runs[0].update(self.run_metadata) is ignored.
After set_data() a check is made:
if run_metadata is not None:
            self.run_metadata.update(run_metadata)

This metadata update is what triggers the failure, because after the metadata update:
self.run_metadata.id = '001'
and
self.station_metadata.runs.keys() = ['0',]
i.e. the run_metadata.id changed, but the station_metadata.runs.keys did not.   Because of this inconsistency, the next time self.validate_metadata() executes, the condition
self.run_metadata.id not in self.station_metadata.runs.keys() returns True, which triggers
self.station_metadata.runs[0].update(self.run_metadata)
I followed the trail for awhile, and the error occurs when an auxiliary channel is encountered...
@kujaku11  do we want to force component on auxiliary channels?  Also, we might need to track down why there is an aux channel at all here.
if channel_obj.component is None:
        if not isinstance(channel_obj, Auxiliary):  # Adding this condition seems to fix the 3.8/3.9 issue
                msg = "component cannot be empty"
                self.logger.error(msg)
                raise ValueError(msg)

Regarding the second flavor of failure, ... this might be related to the obspy version.
Note that obspy v1.2.2 has python2 code in it.
A long-awaited python3-only version of obspy (v1.3) was released in 2022, and updated to 1.3.1 in October 2022.  This requires python >= 3.7.
So we should probably require the same.
Only a month after v1.3.1 was released, out popped v1.4, November 2022.  This version requires python>=3.8.  It is not clear the value of maintaining v3.7 compatibility.  In any case, to fix the v3.7 issue, one need only replace the kwarg:
data_source="NCEDC"
with
data_source='https://service.ncedc.org/'
in make_parkfield_mth5
This argument is passed as base_url to obspy Client
To reproduce the error:
from obspy.clients.fdsn import Client
client = Client(base_url="NCEDC", force_redirect=True)

but replacing with
from obspy.clients.fdsn import Client
client = Client(base_url="https://service.ncedc.org/", force_redirect=True)

works.
This is discussed  in comment by alexhutko.
It has to do with hardcoded url lookup tables, and the fact that NCEDC is only available via https not http.  This may get fixed in obspy, but if we want to support py37 we can just use the explcit url (for now).

Now that these tests are working again, there are a couple of things that can be done to simplify the parkfield tests:

We don't really need to support the creation of separate PKD and SAO, and PKDSAO h5 files

[ ] Replace all references to h5 files with pkd_sao_test_00.h5
[ ] A single method called  ensure_data_exists() can be placed in /test_utils/parkfield/make_parkfield_mth5.py, and all the try/except stuff that is replicated in several methods can be placed in that one spot

I pushed an h5 of the combined PKD and SAO data to mth5_test_data.
in mth5_test_data/mth5/parkfield/pkd_sao_test_00.h5
It should be possible from this file to extract the metadata and the data-streams and archive these somewhere at IRIS.


Add binary GCD

Adds binary GCD to the final step of our gcd_vartime.
The speedup is real but marginal.
 If the two are comparable then we could perhaps get rid of some code. Sadly that is not the case (at all), given our second operand is a Word (aka u64) I do not think the performance difference is due to any inefficiency we can address easily. The difference is one to two orders of magnitude.

Not for this PR, but I wonder if we can use binary GCD to calculate the Jacobi symbol too. It should have more performance impact since it's called more often during Lucas test, although probably still not too much.

Also, again not for this PR, but there's a note in MillerRabin::test():
    pub fn test(&self, base: &T) -> Primality {
        // TODO: it may be faster to first check that gcd(base, candidate) == 1,
        // otherwise we can return `Composite` right away.

Worth checking out since GCD is available in crypto-bigint now.

@fjarri Is this good to go?


update TRANSLATORS

Codecov Report

Merging #4406 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4406      +/-   ##
==========================================
- Coverage   20.06%   20.05%   -0.01%
==========================================
  Files         153      153
  Lines       30769    30769
==========================================
- Hits         6174     6171       -3
- Misses      23652    23654       +2
- Partials      943      944       +1

Impacted Files
Coverage Δ

modules/process/manager.go
69.56% <0%> (-4.35%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e445cc...96f597a. Read the comment docs.


Fix build workflow

After migrating to kubebuilder v4, we need to update the wofkflow

@db-operator/admins It should be ready


How many GPUs are used for training?

How many GPUs are used for training? I used 8 gpus, and the results are not good.

Hi,
We have used two GPUs for training the teacher, and just one for the student.
Let me know if this solves the problems you are dealing with.
A.

I tried to using two GPUs, but came across the following error and got struck:
CUDA_VISIBLE_DEVICES=0,1 python ./tools/train_v2v.py mars --backbone resnet50 --num_train_images 8 --p 8 --k 4 --exp_name base_mars_resnet50 --first_milestone 100 --step_milestone 100
=> MARS loaded
Dataset statistics:
subset   | # ids | # tracklets | # images
train    |   625 |     8298 |   509914
query    |   626 |     1980 |   114493
gallery  |   622 |     9330 |   543216
total    |  1251 |    19608 |  1167623
number of images per tracklet: 2 ~ 920, average 59.5
EXP_NAME:  base_mars_resnet50
Epoch: 0 - triplet 1.54 - class 6.44
Epoch: 5 - triplet 0.34 - class 4.44
Epoch: 10 - triplet 0.16 - class 2.76
Epoch: 15 - triplet 0.12 - class 1.69
Epoch: 20 - triplet 0.08 - class 1.07
Epoch: 25 - triplet 0.04 - class 0.64
Epoch: 30 - triplet 0.04 - class 0.50
Epoch: 35 - triplet 0.03 - class 0.33
^[[A^[[A^[[AEpoch: 40 - triplet 0.02 - class 0.25
Epoch: 45 - triplet 0.03 - class 0.21
I2V
top1:78.18% top5:89.19% top10:91.97% mAP:67.63%
V2V
top1:85.56% top5:94.14% top10:96.16% mAP:75.78%
Traceback (most recent call last):
File "./tools/train_v2v.py", line 125, in
main()
File "./tools/train_v2v.py", line 84, in main
saver.save_net(net.module, f'chk_{e // args.save_epoch_interval}')
File "/home/pichao.wang/code/VKD/utils/saver.py", line 78, in save_net
raise ValueError('PREVENT OVERWRITE WEIGHTS')
ValueError: PREVENT OVERWRITE WEIGHTS

I will delete the logs and try again.

Thanks. And two GPUs working for teacher network. And the results are:
I2V
top1:81.67% top5:92.12% top10:93.64% mAP:73.68%
V2V
top1:88.28% top5:96.31% top10:97.12% mAP:81.05%

Oh, good, the results you have achieved are comparable with the ones discussed in our paper.


backport: bitcoin#15906, #16503, #16535, #16561, #16598, #16643, #16670, #16691, #16726, #16740

Issue being fixed or feature implemented
Follow-up missing backports from v18-v19. Some of them have no dash-relevant code or functionally, but still backported related docs/fixes/refactorings from them.
What was done?
this PR is splitted from PR https://github.com/dashpay/dash/pull/5199 due to large size of PR

bitcoin#15906
bitcoin#16535
bitcoin#16503
bitcoin#16598
bitcoin#16561
bitcoin#16643
bitcoin#16670
bitcoin#16691
bitcoin#16740
bitcoin#16726

How Has This Been Tested?
Run unit/functional tests
Breaking Changes
No breaking changes
Checklist:

[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[x] I have assigned this pull request to a milestone

Sooooo the GitHub UI is still messed up and shows like 137 commits, but that's not true, so merging anyway. I'm not worried about this messing stuff up.


how to call oracle function in mybatis

Hello
I want to call Oracle function using Mybatis i tried Different way  but did not get result.
please solve my issue.
<select id="getNo" resultType="String" parameterType="map" statementType="CALLABLE">
'begin'
     #{retval, mode=OUT, jdbcType=VARCHAR}  = CALL pc_sys.f_get_no
     (
      #{notyp, mode=IN, jdbcType=VARCHAR},
     #{ymdDate, mode=IN, jdbcType=DATE}
     );
end;
 </select>

Error Contents :
begin      ?  = CALL pc_sys.f_get_no      (       ?,      ?      ); end;
java.sql.SQLException: ORA-06550: line 2, column 10:
PLS-00103: Encountered the symbol "=" when expecting one of the following:
:= . ( @ % ; indicator
; bad SQL grammar []; nested exception is java.sql.SQLException: ORA-06550: line 2, column 10:
PLS-00103: Encountered the symbol "=" when expecting one of the following:
:= . ( @ % ; indicator

Hi!
you need to place CALL command before begin statement
for example:
<select id="getNo" resultType="String" parameterType="map" statementType="CALLABLE"> 'call begin'      #{retval, mode=OUT, jdbcType=VARCHAR}  = pc_sys.f_get_no      (       #{notyp, mode=IN, jdbcType=VARCHAR},      #{ymdDate, mode=IN, jdbcType=DATE}      ); end  </select>

We use GitHub Issues only to track bugs and feature requests.
Please post your question to the mailing list.
Thank you!