Pyspark DecisionTreeRegressionModel bundle does not include all attributes

Issue Description
Pyspark DecisionTreeRegressionModel loses values in attributes after packaging and loading them.
Minimal Reproducible Example
mleap version: 0.23.1
pyspark version: 3.3.0
Python version: 3.10.6
import pyspark
import mleap
import mleap.pyspark
from mleap.pyspark.spark_support import SimpleSparkSerializer

from pyspark.sql import SparkSession
from pyspark.ml.feature import VectorAssembler
from pyspark.ml.regression import DecisionTreeRegressor, DecisionTreeRegressionModel

# Step 1: Create a Spark session
spark = SparkSession.builder\
         .config('spark.jars.packages', 'ml.combust.mleap:mleap-spark_2.12:0.23.1') \
        .getOrCreate()

# Step 2: Prepare Data
data = [(1.0, 2.0, 3.0), (2.0, 3.0, 4.0), (3.0, 4.0, 5.0)]
columns = ["feature1", "feature2", "label"]
df = spark.createDataFrame(data, columns)

# Step 3: Feature Vector Assembly
assembler = VectorAssembler(inputCols=["feature1", "feature2"], outputCol="features")
df = assembler.transform(df)

# Step 4: Split Data
(trainingData, testData) = df.randomSplit([0.8, 0.2], seed=1234)

# Step 5: Create and Train Decision Tree Model
dt = DecisionTreeRegressor(featuresCol="features", labelCol="label")
model = dt.fit(trainingData)

# Step 6: Make Predictions
predictions = model.transform(testData)

If we take a look to the created model, we can see that nodes have different attributes.
print(model._to_java().rootNode().toString())
print(model._java_obj.rootNode().toString())

InternalNode(prediction = 4.0, impurity = 0.6666666666666666, split = org.apache.spark.ml.tree.ContinuousSplit@3ff80000)
InternalNode(prediction = 4.0, impurity = 0.6666666666666666, split = org.apache.spark.ml.tree.ContinuousSplit@3ff80000)

If I save and load the model the results are:
model_path = f"{os.getcwd()}/tree_regressor.zip"
model.serializeToBundle(f"jar:file:{model_path}", predictions)
print(f"Model Saved as MLeap bundle at: {model_path}")

loaded_model = DecisionTreeRegressionModel.deserializeFromBundle(f"jar:file:{model_path}")

print(loaded_model._to_java().rootNode().toString())
print(loaded_model._java_obj.rootNode().toString())
print(loaded_model._to_java().rootNode().impurityStats())

InternalNode(prediction = 0.0, impurity = 0.0, split = org.apache.spark.ml.tree.ContinuousSplit@3ff80000)
InternalNode(prediction = 0.0, impurity = 0.0, split = org.apache.spark.ml.tree.ContinuousSplit@3ff80000)
None

Step 1: Create a Spark session
spark = SparkSession.builder
.config('spark.jars.packages', 'ml.combust.mleap:mleap-spark_2.12:0.19.0')
.getOrCreate()

Your example is using mleap 0.19.0.  Does this go away if you use the latest version?  Also note that v0.23.1 is tested against Spark 3.4.  I'd suspect it still works with Spark 3.3, but untested/unsupported.

Hello @jsleight,
You're right I used the jar of the v0.19.0 by mistake.
I have tested using the correct jar:

spark = SparkSession.builder
.config('spark.jars.packages', 'ml.combust.mleap:mleap-spark_2.12:0.23.1')
.getOrCreate()

And the results remain the same

Looks like the op isn't serializing the impurities right now.
I'd be happy to accept a PR to change that behavior though.

Hello @jsleight
The impurities are important for explainability for example.
Shap libraries use them to calculate shap values.

Yeah for sure.  But I'd argue that serializing to mleap is for inference tasks.  To do evaluation and introspection you could just
pipeline.save(path)
pipeline.load(path)

using spark's built in functions.  Then serializeToBundle  when you're ready to productionize the model.

Hello @jsleight
I have no knowledge of Scala but I think I understood how objects are serialized internally.
What do you think about the possibility of an additional parameter in serializeToBundle and deserializeFromBundle that allows us to send a Map with:
Key: Canonical Name of the class that you want to Serialize in a special way.
Value: Custom Ops to apply to that class
And then in the BundleRegistry check if a class is in the new map or if it not, use the defaults
With this perhaps users could create their own ops and add and change attributes.

Ah, in mleap you can do that exact process by altering the ops registry.  We use it for xgboost in order to allow xgboost models to be serialized in different ways depending how you want to serve them.  See this readme and associated xgboost-runtime code as an example.
Using this process, your approach would be to:

Create a custom Op
Specify the new op in the resources.conf file


Why L2 norm attack samples the same window over one-batch's images?

In L-inf norm attack, square attack randomly samples different window positions for each image inside a mini-batch.
However, in L2 norm attack, square attack uses the same window position for different images of a mini-batch.
As shown in https://github.com/max-andr/square-attack/blob/master/attack.py#L156
Why?

It doesn't really matter since each image leads to a separate optimization problem. We can share randomness (i.e. in this case: window position) over multiple optimization problems that are solved in parallel, that's totally fine.

@max-andr Does it mean that L-inf norm attack can also sample the same window over different images inside a mini-batch, instead of sampling separately?

Yes, that would be totally fine for the Linf attack as well.
However, note that we decided to have a separate loop for Linf attack over each image:
https://github.com/max-andr/square-attack/blob/master/attack.py#L226
since it can happen (especially at later stages of the optimization process when the squares are small) that we sample an update which doesn't change the perturbation, and thus we would waste a query for some images. To prevent this, we iterate over each image separately and resample the color of the square-shaped update until we ensure that it leads to a new perturbation. Since we anyway had to implement a loop here, we also sample separately the window position for each image. But this is anyway not an important part.

@max-andr How to set epsilon in CIFAR-10 dataset (32x32x3)? Is the number of 5 too large?

For L2-norm, eps=0.5 is a reasonable choice, e.g., see RobustBench: a standardized adversarial robustness benchmark: a standardly trained model has 0% robust accuracy under eps=0.5 perturbations.


Add Initial Mappings

Add the initial mappings and update the masterthermconnect to a version that fixes a few issues

Hi @johny-mnemonic ,
this integration is under active development right now. We've progressed immensely in the past few weeks. You'll see more sensors soon, and controls are on the radar but that will take longer.
If you're interested in helping us test, can you share what API version you are using (2022+ or the older one?)

@SeBsZ yeah, I noticed it is brand new :-)
I just got PV system installed and would like to gain more control over my heat-pump, so I wanted to call MasterTherm how far they are with promised new API.
And while searching for info I found this new integration :-)
I have BAI22 from 2016, so the old API.
I am a sysadmin and I have some basic knowledge of python and linux, so I can at least help with testing and debugging.

You think there is yet another API coming? Or could it be the 2022 one that is in my heat pump (www.mastertherm.online).

@SeBsZ - Good question mastertherm.online was registered 10th July 2020 so I suspect this one it also has the latest Open ID auth so we should be good :)
@johny-mnemonic:  Lastest is sitting under branch issue16, once I get time to add the heating/ cooling circuites and correct a few minor issues should be good.  Its a bit slow right now as trying to make sure i pick the right entities for each of the types so a bit of trial and error.
It will be read only first version then will look at the updating of certain parts. I know what to do it will mainly be testing/ feedback and use cases.

You think there is yet another API coming? Or could it be the 2022 one that is in my heat pump (www.mastertherm.online).

I was trying to get some means of connecting my home automation to the heat-pump from MasterTherm in 2017/2018 and they were promising new API to me. But after asking for more than a year ("Are we there yet?") and still receiving negative answers, I gave up😀
So most probably it is the one registered in 2020 and brought to life in 2022.
I can ask them again, how to get my heat-pump to the new version if you think new API is better than the old one.

@sHedC is there a way to list available entities from the API, or you need to know upfront which to poll?

@johny-mnemonic this integration will work for both, so no need to upgrade to a new API. They are quite similar anyway, not quite the new API they promised.
There is not really a way to list all available entities from the API. @sHedC, with a little help from me, was able to figure out what all the data means and how to process it so that we can show it in HA.

@johny-mnemonic - have a look at python-masterthermconnect, if you check out the const.py under the masterthermconnect folder you can see all the mappings we have been working on.
You are right there is not much difference, all those in there are to be exposed just working which entities to put each of them under.
Thanks to @SeBsZ for the effort in debugging and  mapping the registers.

@johny-mnemonic - the latest version has sensors and is for testing, would appreciate getting feedback, at the moment everything is read only and only HC0 to HC2 are supported, if you have more I will enabled them.
Of interest

Naming makes sense
changeable binary entities are switches (they can be changed but change back)
Measures are showing correctly in the history

If you can raise a new issue if you notice anything we can discuss in that.  NOTE it does not yet do updates version 1 will be read only then we will work on a new version for simple updates.


nixos/xautolock: improve doc of time parameer

Specify that the time parameter expresses minutes.
Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Thanks! When I did set this up I didn't know which unit it was. So this is handy :-)


Publish on pypi

The readme says it can be installed from pypi, but it hasnt been published on pypi.
https://github.com/bitmazk/flake8-snippets/blob/master/README.rst#installation

@jayvdb It has =)
https://pypi.python.org/pypi/flake8-snippets

Sorry about that.  Their search is failing:
https://pypi.python.org/pypi?%3Aaction=search&term=flake8-snippets&submit=search
There were no matches.

No problems =)


Authentication with Socket.IO - Unclear instructions, incorrect/incomplete solution

Describe your problem and how to reproduce it:
Working on the Advanced Express problems with a server I control.
In the Authentication with Socket.IO section, you are instructed to:

Be sure to add the key and store to the session middleware mounted on the app. This is necessary to tell SocketIO which session to relate to.

However, it is completely unexplained what this is supposed to mean or where you would do this.
I couldn't initially figure out what this even meant so I would get an error like this on the socket connection:

failed connection to socket.io: No session found

I googled around a bit and eventually figured it out.  My solution was to change the key definition in the passportSocketIo.authorize block to match the default connect.sid and add the store to my session definition.
When I went to check against the provided GIST/solution, it was incorrect and incomplete.
Here is the relevant bit of code:
app.use(session({
  secret: process.env.SESSION_SECRET,
  resave: true,
  saveUninitialized: true,
  cookie: { secure: false },
  // Missing entries for the key and store
}));

and also:
io.use(
  passportSocketIo.authorize({
    cookieParser: cookieParser,
    key: 'express.sid',   // This is not the default cookie name, which is connect.sid
    secret: process.env.SESSION_SECRET,
    store: store,
    success: onAuthorizeSuccess,
    fail: onAuthorizeFail
  })
);

More clarity needs to be added to the instructions and the GIST needs to be corrected to match.
Add a Link to the page with the problem:
https://www.freecodecamp.org/learn/quality-assurance/advanced-node-and-express/authentication-with-socket-io

Also, this provided code:
console.log(`user ${socket.request.user.name} connected`);

is wrong.
It should be:
console.log(`user ${socket.request.user.username} connected`);

Thanks, for this, @SaintPeter . The missing key in the Gist is my fault, and I will fix it on my side just for parity. @scissorsneedfoodtoo could you update the camperbot side?

However, it is completely unexplained what this is supposed to mean or where you would do this.

I am not sure where the confusion is...there is only one 'session middleware mounted on the app'. How would you want this explained? (I did not want any more code for a camper to copy-pasta)
Someone else with more experience will have to clarify here:

key: 'express.sid',   // This is not the default cookie name, which is connect.sid

As far as I am aware, this does not matter, provided the two match. I assumed 'express.sid' made sense, because it is an express app. Kris, do you have any insight?

Maybe something along the lines of:

Be sure to add the key with the cookie name to match the socket.io key  as well as the store reference to the options
for the session middleware, near where we set saveUninitialized: true . This is necessary to tell SocketIO which session to relate to.

I don't recall hearing the terminology "mounted on the app" previously and I found it confusing.  We're really talking about changing the options on the session.
I know what you mean about the cut and paste.  Not a whole lot of learning going on here, TBH.  Not sure if it is possible to get the average camper to be able to figure these things out - it's pretty complex.

Fair enough. Thank you, for these suggestions. I would be happy with that wording change.

Thanks for bringing this to our attention @SaintPeter, and for all of your helpful suggestions.
I went ahead and updated that Gist and the ones after to the following:
app.use(session({
  secret: process.env.SESSION_SECRET,
  resave: true,
  saveUninitialized: true,
  cookie: { secure: false },
  key: 'express.sid',
  store: store
}));

About the name of the key, I could go either way. Changing it to the default value seems fine, but leaving them as 'express.sid' also works.
We don't really explain what they are or why they work in the current lessons, so it might be best to leave them as is for now. Then if we go back and break this project down into more steps, we could start with the default value, then have a step where learners rename the keys and go into the fact that they should match.

We don't really explain what they are or why they work in the current lessons, so it might be best to leave them as is for now. Then if we go back and break this project down into more steps, we could start with the default value, then have a step where learners rename the keys and go into the fact that they should match.

I like this idea.  It could be the default key name initially, then by way of explanation we have them change it to match?  I'm not sure what a reason would be for changing the key name, though.  I do wonder how likely it is be that you would need to change it.  It's just as likely that express would randomly change it from connect.sid just to break things.

@SaintPeter, that's a good point. I also don't know why you would change it from the default value unless you really had to for some reason. It could go either way, where we start at the default value and just point out that you could change it in certain situations. Or actually have them change it, though it would be rather arbitrary.
Hopefully Express doesn't go and change connect.sid in some later version and cause bugs if learners update Express before starting!

Hey, guys!
I just finished this project and was stuck with same problem for about an hour.
The instructions is still unclear.
Also, as @SaintPeter mentioned in this message https://github.com/freeCodeCamp/freeCodeCamp/issues/39595#issuecomment-693634106 there is inconsistency between logic of registering user using our app registration form with field username and registration via GitHub OAuth where user (by guide) should be stored in database with field name.
Do we have any agreement (proposals) on how this should be fixed?

Hi @NikIvan, thanks for your feedback.
I think a good first step would be to update the wording for this challenge to the following, which is based on @SaintPeter's suggestion above:

Be sure to add the key with the cookie name to match the Socket.IO key as well as the store reference to the options
for the session middleware, near where we set saveUninitialized: true. This is necessary to tell Socket.IO which session to relate to.

If you're interested in implementing this fix, please check out our documentation here: https://contribute.freecodecamp.org/

I have opened PR.
Also, in my humble opinion, Advanced Node and Express has at least one more thing to improve.
There's no explanation of why do we add store as MongoStore and why do we need it.  I think it would be cool to extract this part into separate lesson and explain difference (pros and cons) between in-memory (by default) storage of session versus storing sessions in database.
Possible location of this lesson: right after Set up Passport.
And probably different issue should be opened with proposition to fix this as well as an issue about
socket.request.user.name

vs
socket.request.user.username

Hi @NikIvan, thanks for those suggestions and for your PR. We'll review it as soon as we can.
At the moment, we're trying not to add any lessons to the curriculum unless absolutely necessary. I could see there being a quick explanation of those points in "Set Up Passport", but it's probably not necessary to add another lesson to go into the pros and cons of storing the session in-memory vs in the DB.
About the other issue you mentioned about the inconsistency of user names, I can see what you mean. I believe it was done this way originally to prevent possible conflicts from storing the username from both registration methods as user.username, though I could be wrong.
@ShaunSHamilton, what do you think? Would it be easy to make that change? Or would it require reworking the boilerplate and a handful of the challenges?

Not sure if we should continue conversation about user names issue in this topic but just to clarify:
With current logic if we try to make conversation between GitHub registered users and signup form registered users - one group of users would have same name undefined in chat because on client side we relate to either name or username.
From my understanding there are 3 ways to fix this:

add mapping username -> name, name -> name client side
add mapping server side before sending to client
fix logic of storing users in DB

it's probably not necessary to add another lesson to go into the pros and cons of storing the session in-memory vs in the DB.

I agree with this. It would be wonderful, if we could go into full detail about all the ins-and-outs of Node/Express + Socketio. However, there is little bandwidth for changing the current lessons to do such, and I would discourage large amounts of effort being put into re-working the current version of the curriculum.
The name/username issue is being tracked here: https://github.com/freeCodeCamp/freeCodeCamp/issues/39599

@ShaunSHamilton, what do you think? Would it be easy to make that change? Or would it require reworking the boilerplate and a handful of the challenges?

As far as I can remember, there should be no need to change the boilerplate. It would require reworking a handful of challenges and Gists.

Hi all, I am having an issue with the code that @SaintPeter mentioned above:
console.log('user ' + socket.request.user.name + ' connected');
When I include this code in function onAuthorizeSuccess(data, accept) in my server.js file, I get a throw error that says "socket is not defined at Object.onAuthorizeSuccess [as success] (/home/runner/boilerplate-advancednode/server.js:44:25)....."
When I delete that piece of code in function onAuthorizeSuccess(data, accept) in my server.js file, my app runs smoothly, with no throw errors.  Any idea what is going on?
I can't quite seem to debug this.


Implement num_traits::Float for Complex<f64>

It would be nice if Complex<f64> implements num_traits::Float. In my code I have a lot of functions that can either take a float, a complex number or a dual number. Ideally I'd like to write a function f<T: Float>(x: T) -> T.
The dual number library dual_num already implements  num_traits::Float. Could you do the same for Complex<f64>?
Many thanks!

I see this issue is similar to https://github.com/rust-num/num-complex/issues/2.


Invalid file path error when providing pattern via css-typed.config.js on Windows

When not providing the pattern argument from the command line, and instead using the default css-typed.config.js
export default {
	pattern: "path/to/**/*.module.css",
};

it throws the following error:

node:internal/modules/esm/load:227
throw new ERR_UNSUPPORTED_ESM_URL_SCHEME(parsed, schemes);
^
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:227:11)
at defaultLoad (node:internal/modules/esm/load:109:3)
at ModuleLoader.load (node:internal/modules/esm/loader:570:13)
at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:443:56)
at new ModuleJob (node:internal/modules/esm/module_job:76:27)
at #createModuleJob (node:internal/modules/esm/loader:456:17)
at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:266:34)
at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:247:17)
at async onImport.tracePromise.proto (node:internal/modules/esm/loader:482:25) {
code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

Workaround:
pass the pattern via CLI

Thank you for the issue! I will look at this and confirm I am using proper cross-platform libraries.

(Self-note) Look to add config test on Windows agent in GitHub actions build.


Beta API

Would be great if beta API (apis/extensions/v1beta1) could be integrated as well. Components such as Ingress are useful and currently to be found under this beta endpoint.

@backender thanks for submitting this ticket! shouldn't be too hard to add, but will need to grab the swagger json and gen the data types for v1beta1. Is v1beta1 and entire description of the API or is it an addition to the api (i.e. lives alongside v1)? If the latter is true, then the code-gen might need some refactoring.

Seems to me like an entire description: https://github.com/kubernetes/kubernetes/blob/master/api/swagger-spec/extensions_v1beta1.json


QUESTION: Chinese garbled code problem

Note that the issue tracker is NOT the place for general support.
I deployed a model, but encountered the problem of garbled Chinese, what is the reason?
such as:

this is my model.json
{
    "version":1,
    "context_length":2048,
    "model_name":"Llama2-Chinese-7b-Chat",
    "model_lang":[
        "zh"
    ],
    "model_ability":[
        "chat"
    ],
    "model_description":"This is a custom model description.",
    "model_specs":[
        {
            "model_format":"pytorch",
            "model_size_in_billions":7,
            "quantizations":[
                "4-bit",
                "8-bit",
                "none"
            ],
            "model_id":"FlagAlpha/Llama2-Chinese-7b-Chat",
            "model_uri":"file:///home/mingzhi/xInference/FlagAlpha/Llama2-Chinese-7b-Chat"
        }
    ],
    "prompt_style":{
        "style_name":"LLAMA2",
        "system_prompt":"<s>[INST] <<SYS>>\nYou are a helpful AI assistant.\n<</SYS>>\n\n",
        "roles":[
            "[INST]",
            "[/INST]"
        ],
        "intra_message_sep":" ",
        "inter_message_sep":" </s><s>",
        "stop_token_ids":[
            2
        ],
        "stop":[
            "</s>"
        ]
    }
}

this is huggingface url:
https://huggingface.co/FlagAlpha/Llama2-Chinese-7b-Chat

请问您有Llama2-Chinese-7b-Chat的vocab.txt文件的文件吗


Add Azure capabilities based on its new CIS Benchmark

New CIS_Microsoft_Azure_Foundations_Benchmark_v1.0.0.pdf has been released, what if we add its checks to Prowler? A bit of code refactorizing has to be done before hand for sure...
Some ideas for v2:
prowler aws [options] and prowler az|azure [options]
also keeping same extras and check groups for each vendor.

Clean list of 80 checks:
1 Identity and Access Management
11 Ensure that multi-factor authentication is enabled for all privileged users (Scored)
12 Ensure that multi-factor authentication is enabled for all non-privileged users (Scored)
13 Ensure that there are no guest users (Scored)
14 Ensure that 'Allow users to remember multi-factor authentication on devices they trust' is 'Disabled' (Scored)
15 Ensure that 'Number of methods required to reset' is set to '2' (Scored)
16 Ensure that 'Number of days before users are asked to re-confirm their authentication information' is not set to '0' (Scored)
17 Ensure that 'Notify users on password resets?' is set to 'Yes' (Scored)
18 Ensure that 'Notify all admins when other admins reset their password?' is set to 'Yes' (Scored)
19 Ensure that 'Users can consent to apps accessing company data on their behalf' is set to 'No' (Scored)
110 Ensure that 'Users can add gallery apps to their Access Panel' is set to 'No' (Scored)
111 Ensure that 'Users can register applications' is set to 'No' (Scored)
112 Ensure that 'Guest users permissions are limited' is set to 'Yes' (Scored)
113 Ensure that 'Members can invite' is set to 'No' (Scored) 37 114 Ensure that 'Guests can invite' is set to 'No' (Scored)
115 Ensure that 'Restrict access to Azure AD administration portal' is set to 'Yes' (Scored)
116 Ensure that 'Self-service group management enabled' is set to 'No' (Scored)
117 Ensure that 'Users can create security groups' is set to 'No' (Scored)
118 Ensure that 'Users who can manage security groups' is set to 'None' (Scored)
119 Ensure that 'Users can create Office 365 groups' is set to 'No' (Scored)
120 Ensure that 'Users who can manage Office 365 groups' is set to 'None' (Scored)
121 Ensure that 'Enable "All Users" group' is set to 'Yes' (Scored)
122 Ensure that 'Require Multi-Factor Auth to join devices' is set to 'Yes' (Scored)
123 Ensure that no custom subscription owner roles are created (Scored)

2 Security Center
21 Ensure that standard pricing tier is selected (Scored)
22 Ensure that 'Automatic provisioning of monitoring agent' is set to 'On' (Scored)
23 Ensure that 'System updates' is set to 'On' (Scored)
24 Ensure that 'Security Configurations' is set to 'On' (Scored)
25 Ensure that 'Endpoint protection' is set to 'On' (Scored)
26 Ensure that 'Disk encryption' is set to 'On' (Scored)
27 Ensure that 'Network security groups' is set to 'On' (Scored)
28 Ensure that 'Web application firewall' is set to 'On' (Scored)
29 Ensure that 'Next generation firewall' is set to 'On' (Scored)
210 Ensure that 'Vulnerability assessment' is set to 'On' (Scored)
211 Ensure that 'Storage Encryption' is set to 'On' (Scored)
212 Ensure that 'JIT Network Access' is set to 'On' (Scored)
213 Ensure that 'Adaptive Application Controls' is set to 'On' (Scored)
214 Ensure that 'SQL auditing & Threat detection' is set to 'On' (Scored)
215 Ensure that 'SQL Encryption' is set to 'On' (Scored)
216 Ensure that 'Security contact emails' is set (Scored)
217 Ensure that security contact 'Phone number' is set (Scored)
218 Ensure that 'Send me emails about alerts' is set to 'On' (Scored)
219 Ensure that 'Send email also to subscription owners' is set to 'On' (Scored)

3 Storage Accounts 115 31 Ensure that 'Secure transfer required' is set to 'Enabled' (Scored)
32 Ensure that 'Storage service encryption' is set to Enabled for Blob Service (Scored)
33 Ensure that storage account access keys are periodically regenerated (Not Scored)
34 Ensure that shared access signature tokens expire within an hour (Not Scored)
35 Ensure that shared access signature tokens are allowed only over https (Scored)
36 Ensure that 'Storage service encryption' is set to Enabled for File Service (Scored)
37 Ensure that 'Public access level' is set to Private for blob containers (Scored)

4 SQL Services
41 SQL Servers
411 Ensure that 'Auditing' is set to 'On' (Scored)
412 Ensure that 'Threat Detection' is set to 'On' (Scored)
413 Ensure that 'Threat Detection types' is set to 'All' (Scored)
414 Ensure that 'Send alerts to' is set (Scored)
415 Ensure that 'Email service and co-administrators' is 'Enabled' (Scored)
416 Ensure that 'Auditing' Retention is 'greater than 90 days' (Scored)
417 Ensure that 'Threat Detection' Retention is 'greater than 90 days' (Scored)
418 Ensure that Azure Active Directory Admin is configured (Scored)
42 SQL Databases
421 Ensure that 'Auditing' is set to 'On' (Scored)
422 Ensure that 'Threat Detection' is set to 'On' (Scored)
423 Ensure that 'Threat Detection types' is set to 'All' (Scored)
424 Ensure that 'Send alerts to' is set (Scored)
425 Ensure that 'Email service and co-administrators' is 'Enabled' (Scored)
426 Ensure that 'Data encryption' is set to 'On' (Scored)
427 Ensure that 'Auditing' Retention is 'greater than 90 days' (Scored)
428 Ensure that 'Threat' Retention is 'greater than 90 days' (Scored)

5 Logging and Monitoring
51 Ensure that a Log Profile exists (Scored)
52 Ensure that Activity Log Retention is set 365 days or greater (Scored)
53 Ensure that Activity Log Alert exists for Create Policy Assignment (Scored)
54 Ensure that Activity Log Alert exists for Create or Update Network Security Group (Scored)
55 Ensure that Activity Log Alert exists for Delete Network Security Group (Scored)
56 Ensure that Activity Log Alert exists for Create or Update Network Security Group Rule (Scored)
57 Ensure that Activity Log Alert exists for Delete Network Security Group Rule (Scored)
58 Ensure that Activity Log Alert exists for Create or Update Security Solution (Scored)
59 Ensure that Activity Log Alert exists for Delete Security Solution (Scored)
510 Ensure that Activity Log Alert exists for Create or Update SQL Server Firewall Rule (Scored)
511 Ensure that Activity Log Alert exists for Delete SQL Server Firewall Rule (Scored)
512 Ensure that Activity Log Alert exists for Update Security Policy (Scored)
513 Ensure that logging for Azure KeyVault is 'Enabled' (Scored)

6 Networking
61 Ensure that RDP access is restricted from the internet (Scored)
62 Ensure that SSH access is restricted from the internet (Scored)
63 Ensure that SQL server access is restricted from the internet (Scored)
64 Ensure that Network Security Group Flow Log retention period is 'greater than 90 days' (Scored)
65 Ensure that Network Watcher is 'Enabled' (Scored)

7 Virtual Machines
71 Ensure that VM agent is installed (Scored)
72 Ensure that 'OS disk' are encrypted (Scored)
73 Ensure that 'Data disks' are encrypted (Scored)
74 Ensure that only approved extensions are installed (Not Scored)
75 Ensure that the latest OS Patches for all Virtual Machines are applied (Not Scored)
76 Ensure that the endpoint protection for all Virtual Machines is installed (Not Scored)

8 Other Security Considerations
81 Ensure that the expiry date is set on all Keys (Scored)
82 Ensure that the expiry date is set on all Secrets (Scored)
83 Ensure that Resource Locks are set for mission critical Azure resources (Not Scored)

Hi @toniblyx
When is this going to be added in prowler? Looking forward to it.

Hello @gaurang8492, thanks for asking. There is not short time plan for this at the moment. You can look at https://github.com/nccgroup/azucar/ and see if it works fine for your needs.

Thanks for your comment @kopit2. No actual plans to do so at short term. We will see how it goes the year.


How to reset value for phone input

I would like to reset the value of the PhoneInput. Does anyone know how to do it?
Really appreciated.

same problem)
I want to create a controlled component

A hacky workaround, but if you provide a key prop to the component you can change it to force a new component to be created/mounted. It doesn't strictly reset the instance you have, but it gives the appearance of resetting.

this.phone.state.inputValue = "";
this.phone.state.formattedNumber = "";

This resets the phone number

If you want to maintain the current dialling code on the input field, the following works for me:

I grab the ISO code from the input
Set the iso2 value to undefined
Call selectCountry with the saved ISO code

const ISOCode = this.phoneInput.getISOCode()
this.phoneInput.setState({ iso2: undefined }, () => {
  this.phoneInput.selectCountry(ISOCode)
})

If you are working with functional component you can use:
phoneInput.current?.setState({number: 0})

If you are working with functional component you can use: phoneInput.current?.setState({number: 0})

setState does not seem to be defined in the type


Request: add a search bar to every page

It woul be great if there was a basic search bar through all the supported licenses. I prefer to view summaries of licenses on your website, so it's helpful to be able to jump to the page for a specific license.
Implementation options:

A basic search bar which loads a search result page is minimum
Ajax live search (i.e., autocomplete/search suggestions) would be a bonus, where you can jump right to the resulting page right from the autocomplete/search suggestions.

Bookmark https://choosealicense.com/appendix/ and c-f (or whatever triggers in-page search in your browser) is one way to jump to the page for any specific license on the site.
But that doesn't search license text or SPDX ID or any other nicknames, which your idea for a search bar could.
Any recommendation for a tiny open source client side search library to use?
Low priority for me to get to, but I'd welcome a prototype implementation in a pull request.

Bookmark https://choosealicense.com/appendix/ and c-f (or whatever triggers in-page search in your browser) is one way to jump to the page for any specific license on the site.
But that doesn't search license text or SPDX ID or any other nicknames, which your idea for a search bar could.
Any recommendation for a tiny open source client side search library to use?
Low priority for me to get to, but I'd welcome a prototype implementation in a pull request.


DEX Circuit Breaker

Feature Definition

to disable all transaction types from Liquidity module

Control

DEXCircuitBreaker

True : disable all transaction types from Liquidity module
False : enable all transaction types from Liquidity module

Authority

DEXCircuitBreakerHandle : the address which has ability to change DEXCircuitBreaker
if null : no address can change DEXCircuitBreaker
how to change? : via a transaction --> existing tx type or new tx type ?

TXCircuitBreaker : change the status of circuit breaker between True/False

Change Via Parameter Governance

DEXCircuitBreaker and DEXCircuitBreakerHandle can be changed via governance

Further Plan

Generalize Circuit Breaker to any arbitrary list of tx types

LGTM


Traceback (most recent call last):

Traceback (most recent call last):
File "/home/panpan/Desktop/unet-master/main.py", line 15, in
model = unet()
File "/home/panpan/Desktop/unet-master/model.py", line 35, in unet
merge6 = merge([drop4,up6], mode = 'concat', concat_axis = 3)
TypeError: 'module' object is not callable
您好，我刚接触这块，基础较弱；
我不知道为什么会有这个错误，找了很多关于这个的错误，也是不可以。您能告诉我怎么修改么

@PanPan0210  try with trainUnet.ipynb . How are you running your program?

I have the same error.

@ajithvallabai the python notebook throws the same error! any imports I am missing out?

@PanPan0210  @Pmac23  i am able to execute . check your path for files and imports .

Hi I got same problem. while running model = unet().
**merge6 = merge([drop4, up6], mode='concat', concat_axis=3)

TypeError: 'module' object is not callable**
I am using keras 2.2.0 with tensorflow 1.5.0.

@sagardatascientists  could you use all the merges like below code
 merge6 = concatenate([drop4,up6],axis=3)
coudl you provide your python version ? .

I have modified the code for concatenate() and transpose but i didnt check the accuracy rate
from up6 layer to conv 10 you need to provide the following code
up6 = Conv2DTranspose(512, (2,2),strides=(2,2), activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(drop5)
merge6 = concatenate([drop4,up6],axis=3)
conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)
conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)
up7 = Conv2DTranspose(256,(2,2),strides=(2,2), activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)
merge7 =concatenate([conv3,up7],axis=3)
conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7)
conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)
up8 = Conv2DTranspose(128,(2,2),strides=(2,2), activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)
merge8 =concatenate([conv2,up8],axis=3)
conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8)
conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)
up9 = Conv2DTranspose(64,(2,2),strides=(2,2), activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)
merge9 =concatenate([conv1,up9],axis=3)
conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9)
conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)
conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)
conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)

Hello Ajit,
Yes, I already did that and it works fine.  I used Python 3.6 version.
Thank you for your reply!
On Wed, Jun 20, 2018 at 7:28 PM, Ajith<EMAIL_ADDRESS>wrote:

@sagardatascientists https://github.com/sagardatascientists could you
use all the merges like below code
merge6 = concatenate([drop4,up6],axis=3)
coudl you provide your python version ? .
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zhixuhao/unet/issues/45#issuecomment-398651926, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AQ2e3EGxWEJBsaxS2ZmNqgYXWW-wZaiQks5t-fmqgaJpZM4Ul1jo
.

--
With Regards,
*Sagar Soni | Machine Learning Engineer *
+22 109 7206

can you please tell me what changes i need to do code while i am import data by me without fetching fro site.
because i am unable to fetch it from kaggle site in this code i have kaggle ID and password

I got the same problem and I solved it! It’s because my Jupyter uses python 2, while my Keras is using python 3. When I run everything in python3, it worked well.

Hi,
I am getting the below message while running Unet. I am using pyhon 3 and  heras - 2.1.6-tf version.Please help me.
I changed below two lines or similar lines on code
up6 = Conv2DTranspose(512, (2,2),strides=(2,2), activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(drop5)
merge6 = concatenate([drop4,up6],axis=3)
But getting the same error message.
TypeError                                 Traceback (most recent call last)
 in ()
8 myGene = trainGenerator(2,'./data/membrane/train','image','label', data_gen_args, save_to_dir = None)
9
---> 10 model = unet()
11 model_checkpoint = ModelCheckpoint('unet_membrane.hdf5', monitor = 'loss',verbose = 1, save_best_only = True)
12 model.fit_generator(myGene,steps_per_epoch = 300, epochs = 1, callbacks = [model_checkpoint])
/project/bioinformatics/Rajaram_lab/s183574/myCopy/unet-master/model.py in unet(pretrained_weights, input_size)
32
33     #up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))
---> 34     #merge6 = merge([drop4,up6], mode = 'concat', concat_axis = 3)
35     up6 = Conv2DTranspose(512, (2,2),strides = (2,2), activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(drop5)
36     merge6 = concatenate([drop4, up6],axis = 3)
TypeError: 'module' object is not callable

Hi,
I am getting the below message while running Unet. I am using pyhon 3.6 and keras - 2.1.6-tf version.
Please help me.
I changed below two lines or similar lines on code
up6 = Conv2DTranspose(512, (2,2),strides=(2,2), activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(drop5)
merge6 = concatenate([drop4,up6],axis=3)
But getting the same error message.
TypeError Traceback (most recent call last)
in ()
8 myGene = trainGenerator(2,'./data/membrane/train','image','label', data_gen_args, save_to_dir = None)
9
---> 10 model = unet()
11 model_checkpoint = ModelCheckpoint('unet_membrane.hdf5', monitor = 'loss',verbose = 1, save_best_only = True)
12 model.fit_generator(myGene,steps_per_epoch = 300, epochs = 1, callbacks = [model_checkpoint])
/project/bioinformatics/Rajaram_lab/s183574/myCopy/unet-master/model.py in unet(pretrained_weights, input_size)
32
33 #up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))
---> 34 #merge6 = merge([drop4,up6], mode = 'concat', concat_axis = 3)
35 up6 = Conv2DTranspose(512, (2,2),strides = (2,2), activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(drop5)
36 merge6 = concatenate([drop4, up6],axis = 3)
TypeError: 'module' object is not callable

did you solve it?  I am facing the same thing


Add endpoint to retrieve all request mappings as JSON

This introduces an HTTP-endpoint /endpointdoc which delivers all mapped HTTP endpoints as JSON objects. Hereby all instances (types and methods) annotated with @RequestMapping annotations in @Controller classes are analyzed.
This is very useful to build an automated API documentation for all HTTP endpoints.
Sample output:
{
    name : null,
    patternsCondition : {
        patterns : [ "/my-super-shogun-endpoint" ],
        empty : false
    },
    methodsCondition : {
        methods : [ "POST", "PUT" ],
        empty : false
    },
    paramsCondition : {
        expressions : [],
        empty : true
    },
    headersCondition : {
        expressions : [],
        empty : true
    },
    consumesCondition : {
        expressions : [ {
            mediaType : {
                type : "application",
                subtype : "json",
                parameters : {},
                qualityValue : 1,
                charSet : null,
                wildcardType : false,
                concrete : true,
                wildcardSubtype : false
            },
            negated : false
        } ],
        empty : false,
        consumableMediaTypes : [ {
            type : "application",
            subtype : "json",
            parameters : {},
            qualityValue : 1,
            charSet : null,
            wildcardType : false,
            concrete : true,
            wildcardSubtype : false
        } ]
    },
    producesCondition : {
        expressions : [],
        empty : true,
        producibleMediaTypes : []
    },
    customCondition : null
}

Nice! How well does this play with swagger? Is this related somehow? Is the format you return otherwise standardized?

This is default format Spring offers us out of the box. I doubt that there is any relation to Swagger or that this is a standardized format.
springfox seems to offer a nice way to get a Swagger-compatible API-documentation. Maybe someone finds the time to test and come up with another PR extending this EndpointDocController.

Ping @buehner wrt to security
I am 👍

Nice addition! Thx @chrismayer
Regarding security we can think about encapsulating the trivial controller logic in a service. The service could easily be secured with an annotation to avoid that everyone can consume this endpoint to retrieve information for possible attacks!?
But i'm not really sure in this point...

Thanks for the feedback guys!
I agree that we can encapsulate the logic in service, so we are able to secure this interface.
@buehner: Do you have a suggestion for a security level? I would suggest
hasRole(@configHolder.getSuperAdminRoleName()) or hasPermission(returnObject, 'READ')
like is done in other GET requests.

I think hasPermission(returnObject, 'READ') only makes sense if there is a PermissionEvaluator for the returnObject. But even without a permission evaluator this would not break anything, so :+1: as we can still add a permission evaluator for such a scenario in follow up PRs

Right @buehner, hasPermission(returnObject, 'READ') makes no sense here. I just added another commit introducing a service layer, which is secured, so only "admins" can access the endpoint.
Would be cool if you could have another look at it.

Just found a small typo. If this is fixed: Feel free to merge :+1:

Thanks again for your review @buehner! I adressed your comment and will merge now.


no-vendor-prefixes confuse standard properties and pseudo-elements

I am using gulp-sass-lint v1.3.2 with sass-lint v1.10.2.
For the following code:
progress {
    &::-ms-fill { ... }
}

(from foundation-sites / forms/_progress.scss)
The following warning is thrown:
  86:8  warning  Vendor prefixes should not be used  no-vendor-prefixes

The sass-lint config is :
...
  no-vendor-prefixes:
     - 1
     - ignore-non-standard: true
...

(from .sass-lint.yml)
Sass-lint seems (I'm not sure about it) to consider the progress pseudo-element ::-ms-fill as standard because of the fill property. But ::-ms-fill is specific to IE10 (see https://developer.mozilla.org/fr/docs/Web/CSS/::-ms-fill). I expect ignore-non-standard to not check for a standard pseudo-element in a list of property.

:tada: This issue has been resolved in version 1.13.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add information on server-side OData querying.

Docs Build status updates of commit 0f2efa2:
:clock10: Full build: the average full build time is 6 min(s) 47 sec(s), this is based on the last 210 build(s) for this repository.

Docs Build status updates of commit 0f2efa2:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/query-azure-cli.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile) View (azure-cli-2018-03-01-hybrid)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Going to actually cut this. This is the responsibility of the command documentation to direct users to it - OData is not used in any general-purpose way throughout the CLI. It should be done on a per-service or per-command basis in the long-form documentation.


docs(compute-samples): added samples and tests for custom machine type

Added Compute Machine type samples and tests.
Docs: https://cloud.google.com/compute/docs/instances/creating-instance-with-custom-machine-type

Kindly provision the required permission: roles/compute.instanceAdmin.v1 and roles/compute.storageAdmin
TIA!

Thanks @Shabirmean for the review. Addressed all your comments.
The class structure was supposed to be in the format you suggested. Looks like the auto code linter had other plans :) Thanks for bringing it up!


fix: Add _fresh to .gitignore on update

Fixes #1680

I added some test but there are some things I would like to add.

I found it difficult to find the tests
There is some room for refactoring of the update tests

I don't mind picking up some of this work. I would suggest splitting the tests into init_test.ts and update_test.ts.
Also, why import the asserts from deps.ts and not directly from $std?

I don't mind picking up some of this work. I would suggest splitting the tests into init_test.ts and update_test.ts.

Agree, splitting these up makes sense.

Also, why import the asserts from deps.ts and not directly from $std?

That was before my time, but I'd guess that this was done so that there are less places to update when moving to a newer version of std.


Feature/rserv 58 style refactor

Hey @RobertToScale I'm just adding you so you are aware of changes. Best to commit your current changes then do git pull origin main followed by git pull origin stage ... then do npm install in the root directory


Feature Request / enable the creation of alias with cmdlet and its params together

Summary of the new feature/enhancement

Currenly, we can only create alias to a cmdlet name only, it would be nice to be able to create alias to the cmdlet and with its params also.
The workaround that I could find till now is to create in advance a new custom cmdlet (via function{}) which calls the target cmdlet with the expected params, and then create an alias that points to this new cmdlet.
# The workaround by creating a new cmdlet,
# ok for complexe functions,
# but not cool for calling just a cmdlet with params in one line.

function Out-StringWithStream {
    Out-String -Stream
}

Set-Alias oss Out-StringWithStream

Proposed technical implementation details (optional)
Set-Alias oss Out-String -Stream

# or with ScriptBlock
Set-Alias oss {Out-String -Stream}

That you want already implemented as proxy functions https://devblogs.microsoft.com/scripting/proxy-functions-spice-up-your-powershell-core-cmdlets/

Proxy functions came to mind for me as well, but maybe this should be a discussion about improving that experience.  I use proxy functions a lot in my profile, but I feel like I'm one of few because of how obtuse they are.  For instance, the above would have to be written as:
function Out-StringWithStream {
    [CmdletBinding(DefaultParameterSetName='NoNewLineFormatting', HelpUri='https://go.microsoft.com/fwlink/?LinkID=113368', RemotingCapability='None')]
    param(
        [ValidateRange(2,<PHONE_NUMBER>)]
        [int]
        ${Width},

        [Parameter(ParameterSetName='NoNewLineFormatting')]
        [switch]
        ${NoNewline},

        [Parameter(ValueFromPipeline=$true)]
        [psobject]
        ${InputObject})

    begin
    {
        try {
            $outBuffer = $null
            if ($PSBoundParameters.TryGetValue('OutBuffer', [ref]$outBuffer))
            {
                $PSBoundParameters['OutBuffer'] = 1
            }

            $PSBoundParameters.Add('Stream', [System.Management.Automation.SwitchParameter]::Present)

            $wrappedCmd = $ExecutionContext.InvokeCommand.GetCommand('Microsoft.PowerShell.Utility\Out-String', [System.Management.Automation.CommandTypes]::Cmdlet)
            $scriptCmd = {& $wrappedCmd @PSBoundParameters }

            $steppablePipeline = $scriptCmd.GetSteppablePipeline($myInvocation.CommandOrigin)
            $steppablePipeline.Begin($PSCmdlet)
        } catch {
            throw
        }
    }

    process
    {
        try {
            $steppablePipeline.Process($_)
        } catch {
            throw
        }
    }

    end
    {
        try {
            $steppablePipeline.End()
        } catch {
            throw
        }
    }
    <#

    .ForwardHelpTargetName Microsoft.PowerShell.Utility\Out-String
    .ForwardHelpCategory Cmdlet

    #>
}

Yeah most of it is generated for you with ProxyCommand.Create but for new to intermediate folks, it'll be challenging to even figure out where to put $PSBoundParameters.Add('Stream', [System.Management.Automation.SwitchParameter]::Present).
I don't think Set-Alias is the place for something like that, but I'd love to see that made a lot easier.

Would be awesome to have a New-ProxyCommand cmdlet built in that can generate the needed code and add modifications right from the command line. 🙂

@iSazonov
Thx for sharing the ProxyCommand, but sorry, after reading the blog, imho, I dont think it's the answer to this issue. The tag  Resolution-Answered should be removed.
First of all, it's too complicated, as a lazy windows admin, I really dont like to write such ProxyCommand just to create an alias to a cmdlet + params ? And if I have 10 or more proxy commands to create ?
Then, even for the powershell experts, when they remote connect to a server for a live troubleshooting, and that server only has the default cmdlets set, do they like to or have the time to write such thing ?
BTW, in the Linux world, the alias can be redirected to anything already, it would be nice to port it to powershell.

My old pspx module makes this easier. I could refresh that for PowerShell v.Current. It's one of those things where I felt like a rather extreme propeller head and wondered if anyone was actually using the functionality it provided, but it would allow for something like this:
New-ProxyFunction `
    -CommandName Out-String `
    -ProxyName Out-WithStringStream `
    -RemoveParameter Stream `
    -Begin {
        $PSPassThruParameters.Add('Stream',$true) > $null
    } `
    -DefineNow

That sounds like something that would be very very nice indeed. :)

In general, new proxy cmdlet would has new logic. How would New-ProxyCommand help? I believe it is not help. We could address a simple scenario like hiding parameters but it's not worth the effort.
There is a common solution sample - RBAC in Exchange Server.

@iSazonov I disagree with the resolution of this issue. Answering a request by saying "just go do this really complicated thing" (proxy commands are not trivial) is not an answer. At a minimum, it should be left open to see if others have similar interest or to continue the discussion. Having it marked as closed and answered signals that there isn't a discussion to be had here.

@KirkMunro We already have up to 2000 issues without discussion and progress. The issue is not locked and everyone can discuss and share ideas.
A resolution was to learn Exchange Server RBAC - that tehnology would addresses the feature request. Perhaps it could be implemented as a community module.


An account is required?

~/gocode/pkg/mod  micro run github.com/micro/services/helloworld
An account is required
How do you deal with this?

$ micro env set local
$ micro login
# user: admin
# pass: micro
$ micro user

thanks very much ,I've solved it


performance drop when fintunning DMC model from ACMMM2022

Congratulations on such remarkable results that surpass VVC!
I ran the test code with the pre-trained model you provided and I reproduced the results you reported in the paper.
Unluckily, when I tried to finetune it for some steps following the training strategy (lr=1e-5, multi-frame forward loss averaging) in (sheng et al2021), the RD  curve will drop around 0.7 dB after just hundreds of steps.
The training code is based on DVC and the regular rate-distortion loss is optimized.
Any suggestions will be appreciated.
Thank you!

I ran into the same problem. Performance drops just a few steps after fine-tuning.
I wonder if the authors can release their train code.
Any suggestions will be appreciated.
Thank you!

Hi, do you solve the problem?

Can I discuss the problem with you？my wechat：zxl_Nefelibata

Hi, do you solve the problem?

sorry, I have not solved the problem.

@jinxixiang Can I discuss the problem with you？my wechat：zxl_Nefelibata

@kesixcross Can I discuss the problem with you？my wechat：zxl_Nefelibata

Can someone train  DCVC? After compressing the MV, I trained the loss of the reconstructed frame, and the PSNR has been unable to go up. The PSNR of the distorted frame can reach 32, 33, but the PSNR of the reconstructed frame is only more than 20.my wechat:lin1034725950

Same here.
It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and DCVC-DC cases.
I guess training "adaptive quantization" could be non-trivial, but there's no quite much training details mentioned in the papers.
It'll be a lot helpful if the author by any chance can share more training details, thanks.

Me too. It would be appreciated if anyone solved this problem and shared it.

Congratulations on such remarkable results that surpass VVC!
I ran the test code with the pre-trained model you provided and I reproduced the results you reported in the paper.
Unluckily, when I tried to finetune it for some steps following the training strategy (lr=1e-5, multi-frame forward loss averaging) in (sheng et al2021), the RD curve will drop around 0.7 dB after just hundreds of steps.
The training code is based on DVC and the regular rate-distortion loss is optimized.
Any suggestions will be appreciated.
Thank you!

Hi！
I also want to fine-tune DCVC for my research, but I don't know how to train it.
Could you share your training code with me?
I will appreciate it if you can help me on fine-tune DCVC model !

Hi Tian,
I’m actually no longer working on this topic now, but one major thing you
might need to be careful about is the quantization.
You should use additive noise for quantization instead of rounding, just
like their released code does, during training.
If you already done this, I’m afraid I may not be able to provide further
details.
Wish you the best.
BR
wenxin Tian @.***>於 2023年11月23日 週四，下午11:46寫道：

Same here. It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and
DCVC-DC cases. I guess training "adaptive quantization" could be
non-trivial, but there's no quite much training details mentioned in the
papers. It'll be a lot helpful if the author by any chance can share more
training details, thanks.
@tl32rodan https://github.com/tl32rodan I use the training code of the
compressai to fine-tune DCVC-TCM for 1 epoch, the performance drops highly,
I don't know why cause this, could you share your fine-tune code for me?
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/DCVC/issues/2#issuecomment-1824241040, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AJNI6RZPJVLW3AEC76SARBDYF4VN7AVCNFSM55UTXZJKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBSGQZDIMJQGQYA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi Tian, I’m actually no longer working on this topic now, but one major thing you might need to be careful about is the quantization. You should use additive noise for quantization instead of rounding, just like their released code does, during training. If you already done this, I’m afraid I may not be able to provide further details. Wish you the best. BR wenxin Tian @.>於 2023年11月23日 週四，下午11:46寫道：
…
Same here. It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and DCVC-DC cases. I guess training "adaptive quantization" could be non-trivial, but there's no quite much training details mentioned in the papers. It'll be a lot helpful if the author by any chance can share more training details, thanks. @tl32rodan https://github.com/tl32rodan I use the training code of the compressai to fine-tune DCVC-TCM for 1 epoch, the performance drops highly, I don't know why cause this, could you share your fine-tune code for me? — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNI6RZPJVLW3AEC76SARBDYF4VN7AVCNFSM55UTXZJKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBSGQZDIMJQGQYA . You are receiving this because you were mentioned.Message ID: @.>

Thanks for your reply!!!

Same here. It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and DCVC-DC cases. I guess training "adaptive quantization" could be non-trivial, but there's no quite much training details mentioned in the papers. It'll be a lot helpful if the author by any chance can share more training details, thanks.

Hello, I'm very interested in DCVC-TCM, but I don't have the ability to fully reproduce the training phase of the code. I'm wondering if you would be willing to share the training phase code with me. My email is<EMAIL_ADDRESS>I would greatly appreciate it!

**YY-RR-ZZ ** commented Apr 18, 2024

Have you got the training code?


Add support for Laravel 9

WHY
BEFORE - What was wrong? What was happening before this PR?
Could not install Backpack on Laravel 9.
AFTER - What is happening after this PR?
Can install, but only Backpack 4.2. I think this is a good way to proceed, because if you have the time to upgrade Laravel, you should also have the time to upgrade Backpack.
HOW
How did you achieve that, in technical terms?
Bumped composer.json requirements.
Is it a breaking change or non-breaking change?
Non-breaking for 4.2.
How can we test the before & after?
Try installing without - shouldn't work. Try installing with - should work.

While adding support, I've discovered quite a few issues in our dependencies. These ALL need to be fixed, in order to be able to provide full support for Laravel 9:
MUST-DOs, otherwise we can't say we provide support:

[ ] exception on all pages, because prologue/alerts AlertsMessageBag::count() declaration must be compatible with Illuminate\Support\MessageBag::count() in v9; fixed by https://github.com/prologuephp/alerts/pull/39
[ ] error on @php artisan package:discover because fideloper/proxy has not tagged a new version in 1.5 years (workaround - use dev-master as 4.4.99);
[ ] conflict because of DevTools, which requires Livewire, which requires symfony/http-kernel v5 (will probably solve itself pretty fast);

SHOULD-DOs, otherwise some Backpack addons won't work under Laravel 9:

[x] conflict because of BackupManager, which requires spatie/backups v6 and does not support v8 yet; fixed by https://github.com/Laravel-Backpack/BackupManager/pull/98
[ ] confict because of FileManager, which requires barryvdh/laravel-elfinder, which requires flysystem v1 (OUCH!);

Turns out Backpack/CRUD doesn't really need much to support Laravel 9. So I'm going to merge this into 4.2, so we can also test against it. We still need to add support to two of our dependencies in order to "fully" support L9, though. But I've mentioned that in https://github.com/Laravel-Backpack/demo/pull/342 which will stay open until we do (could take a few days).

@tabacitu will this laravel 9 support lands on 4.x eventually or its only available for 5.x ?

Hi @lakuapik ,
We don't plan on adding L9 support to Backpack 4.1 - it supports already L6, L7 and L8.
If we added L9 support to B4.1 it would mean people would be using it B4.1 for 1+ year going forward, which would mean bug reports and feature requests 1+ year going forwards. Unfortunately we don't have that kind of resources, we can only focus on one major version at a time - and right now that version is 5.x. We'll still add a few minor fixes we want in 4.1, and security fixes if they are ever needed... but we can't maintain it any longer, I'm afraid, so L9 support is a no-go.
Plus, v5 is a super-easy upgrade for most people, there's no reason not to upgrade.
Cheers!


Wrong variable name in check for Dism version in New-LabImage

I have a fresh install of Windows 10 and on it Lability throws an error saying that I need dism version <IP_ADDRESS>
(I have this version installed, it seems the check doesn't see that correctly)
Turns out, Line 69 of New-LabImage checks $labDefaults.version, but I think this should be $labDefaults.DismVersion
I've attached a file with the (small) fix
New-LabImage.ps1.txt

Bad timing for me,  first time trying this and trying to learn DSC. Sitting here thinking I'd flunked.
Great module anyway guys

@andrekamman @gabrielmccoll Sorry - been on holiday for a couple of weeks 😢. I'll double-check the unit tests as this should be covered (and obviously isn't).

That's okay you're doing great work ! Hope you had good holiday
On Mon, 4 Sep 2017, 10:17 Iain Brighton<EMAIL_ADDRESS>wrote:

@andrekamman https://github.com/andrekamman @gabrielmccoll
https://github.com/gabrielmccoll Sorry - been on holiday for a couple
of weeks 😢. I'll double-check the unit tests as this should be covered
(and obviously isn't).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/VirtualEngine/Lability/issues/247#issuecomment-326911297,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYrFMaxxFsbQUNIz_K7uBAcME1d49ES5ks5se8BDgaJpZM4PATFB
.

What a guy. Like I said on other thread if I can help out am happy to do so
On Mon, 4 Sep 2017, 12:22 Iain Brighton<EMAIL_ADDRESS>wrote:

Closed #247 https://github.com/VirtualEngine/Lability/issues/247 via
#248 https://github.com/VirtualEngine/Lability/pull/248.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/VirtualEngine/Lability/issues/247#event-1233379907,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYrFMRE7H6QdaihojEdily3MM0ijHbPsks5se92LgaJpZM4PATFB
.


Need to implement a unit testing framework for exporter code

The exporter code is becoming difficult to debug when it doesn't work. It would be useful to have a unit testing framework in place that encourages us to write more testable code.

Coverage.py looks interesting for this:
https://coverage.readthedocs.io/en/coverage-5.1/


ADC module lacks simple useful #defines

It would be pretty useful to have a defines for ADC module that state

max reading in bits
reference voltage

for actual reading conversion in applications. Also a simple macro for converting a GPIO pin to ADC channel could be implemented easily.

reference voltage

@Kucmasz this depends on hardware external to the RP2040 (ADC_AVDD). The datasheet plainly states that the ADC is 12-bit.
In practice there are often voltage dividers and other signal conditioning circuitry that would go from some application signal to the ADC input, so it doesn't seem that useful to have #define macros for this because:

#define is horrible
everything ADC related in firmware is application specific


Is there any additional way to improve the performance?

We are currently using the Whisper-Tiny multilingual model and seeking ways to improve its performance. We would appreciate any insights or suggestions on how to enhance the model's accuracy, speed, and overall efficiency.
Model:
Whisper-Tiny Multilingual TFlite (decord_id is Korean) https://github.com/nyadla-sys/whisper.tflite/discussions/15#discussioncomment-7362798
Apply:
https://github.com/vilassn/whisper_android/issues/4#issuecomment-1846846235
+To achieve real-time speech processing, we are using sendData(sample) in Recorder.java
The accuracy is low; are there any ways to improve it?
I understand that the accuracy is low partly because it is a Tiny model and it has been converted to a tflite model, but I would still like to improve the performance as much as possible.

@KihongK The primary reason for the low accuracy of the Whisper-Tiny model is due to how the audio data is being segmented. Currently, we are feeding 3-second audio clips without considering the natural pauses in speech, which often results in cutting off words and phrases. To improve the accuracy, we should segment the audio based on pauses in the speech rather than fixed time intervals. This can be achieved by implementing voice activity detection (VAD) to detect and segment speech more naturally.
To improve the performance, improve Audio Segmentation. Utilize voice activity detection (VAD) to ensure that audio clips are cut at natural pauses, avoiding mid-word and mid-sentence breaks.


New command to scaffold a module: fop:module:generate

This command scaffold new module.
Command

fop:generate:module

Arguments

<module name>:  your module name to generate
<name space>:    the namespace of the module

Thanks @luigimassa for your PR ! Nice work.
We have a naming convention, the command should be fop:module:generate instead of fop:generate:module

run vendor/bin/php-cs-fixer fix ./ add and commit the file and you're done (probably).

If it's not ok,
You can run vendor/bin/php-cs-fixer fix --dry-run -q ./ || echo 'nop :( if "nop :(" is printed, the file is not fixed.
vendor/bin/php-cs-fixer fix --dry-run --diff . shows the diff.
Just in case, because your vendor directory is maybe not right, delete it and run composer install

run vendor/bin/php-cs-fixer fix ./ add and commit the file and you're done (probably).
If it's not ok,
You can run vendor/bin/php-cs-fixer fix --dry-run -q ./ || echo 'nop :( if "nop :(" is printed, the file is not fixed.
vendor/bin/php-cs-fixer fix --dry-run --diff . shows the diff.
Just in case, because your vendor directory is maybe not right, delete it and run composer install

fixed

Yeah! :D

Great the tests are now ok !
Let's dive into the command behaviors tests now :smile:
Thanks @luigimassa for your work :+1:  !

all tests are green 👍🏼
Is it ok to merge @nenes25 & @Kaudaj or do you think some things are mandatory before merging ?

@SebSept Already resolved: https://github.com/friends-of-presta/fop_console/pull/239/commits/a5978415e8b45eaa2c8ad9f518e2631a4a210364 😉
Some of my requests haven't been resolved but they were not critical. Maybe the // TODO: Edit could be harmonized, it looks quite messy right now (there are 'xxxxxx' values, '????' values,  // edit comment...).

@SebSept Already resolved: a597841 wink
Some of my requests haven't been resolved but they were not critical. Maybe the // TODO: Edit could be harmonized, it looks quite messy right now (there are 'xxxxxx' values, '????' values, // edit comment...).

@luigimassa will fix this or do you prefer that the merge in the current state ?

thanks @luigimassa for your geat work,

No need to rush: quality > quickness!

Please can you merge?

changes requested by @tom-combet are satisfied.
I guess changes requested by @nenes25 are also satisfied.
I generated a module, there is no php errors, I think everything is alright...

Is it ok for you @nenes25 ?

A huge thanks for this great job @luigimassa , well done !

Thank you guys!


Unexpected end of input. in IE11, progress-bar

Bug, feature request, or proposal:
Bug in IE11
What is the expected behavior?
Console without errors
What is the current behavior?
Produce this error:
XML5602: Unexpected end of input.
Line: 1, Column 252

What are the steps to reproduce?
You can follow this link by using IE 11 and see console:
https://material.angular.io/components/component/progress-bar

This seems to be due to the SVG background in buffer mode. Some of the more obvious fixes didn't work.


add path_key and path_root parameters to client.py

This request is for a modification/enhancement.
The modification adds input parameters for path_key and path_root so that you have the option to supply the device key and root CA cert as well as the device cert. You can still enter just the device cert if that is all you require.
Params are added to client.py in init and enter methods.
I found this modification to be necessary in order to connect to my Juniper devices using TLS.

Thanks!


Migrate to Travis CI for Windows build

Now that Travis CI supports Windows, should we drop Appveyor and move to Travis CI?
cc @luftywiranda13

At the moment, all Windows tests are failing on Travis: https://travis-ci.org/okonet/lint-staged/builds/444312179

GitHub Actions is currently in beta, and I believe scheduled to be GA later this month. They also offer free builds on Windows and seem more up to date than Travis. You may consider this instead.

I’d love to move to it!

I'll see if I can mock up a workflow file sometime this week. I should be able to see builds run once I fork the project and setup a workflow for GitHub actions (I'm in the beta).

https://github.com/nemchik/lint-staged/commit/69ca1c59df1537fe8db7a52e28a82adc9a08d02b/checks
Here's a start. Obviously not finished. I also don't have much experience with yarn, so I tried to have it use npm instead. Not sure if that's an issue. Also I've made a bit of a massive build matrix which may be a bit unnecessary.

I'm a lot closer with my branch https://github.com/okonet/lint-staged/compare/master...nemchik:gh-actions
https://github.com/nemchik/lint-staged/runs/224657305 as you can see here the failures in the build seem to be from differences in the output of the git CLI which I'm thinking might be easy to fix, but I'm not sure (first time with jest).
Fixing that and then adjusting the package build to actually be able to push builds should be all that's needed. It looks like https://github.blog/2019-08-08-github-actions-now-supports-ci-cd/ says actions will be GA in November, but you might be able to get into the beta before then. Either way this should cover everything you're currently doing with appveyor and Travis (and then some, I threw in Mac builds).

Also the Windows builds are upsetting prettier due to line endings. I think that's a matter of the way git clones on Windows. I'll see if I can dig up a fix.

@nemchik you should be able to work around that by adding a .gitattributes file to the lint-staged repo root in your branch.
Also note that we run codecov and greenkeeper on all pushes, and additionally semantic-release on certain conditions, as seen in the .travis-ci.yml.

I started trying to work with the .gitattributes file but I don't think I've got it working yet https://github.com/okonet/lint-staged/compare/master...nemchik:gh-actions
I temporarily removed greenkeeper while trying to fix the line endings. I don't think it needs to run on the whole matrix, just maybe one build. Same with codecov.

It looks like there are some people working on adjustments to the starter workflow (focused around the process of publishing releases) which was used as the template for the GH workflow in my branch. I'm going to let that dust settle before working on release publishing in my branch. In the meantime I'll continue when I have time to work on confirming that the rest of the things (actual unit testing on various OS) works correctly.


Porting: gammafunction: Add constants for ieeeQuadruple reals

Setting MAXGAMMA to the same as for ieeeExtended reals means the range of the gamma function will be limited to the same range as ieeeExtended. However, until someone reviews the algorithm to work fine for ieeeQuadruple ranges, this workaround seems fine. We just lose some 128bit real features, but we're still as good as x86.
This is a revival of https://github.com/dlang/phobos/pull/4047

Thanks!


add drag&drop examples

lack of documentation how to implement plupload  drag&drop with this directive

+1


Add and use new release_or_current_path

As discussed in PR https://github.com/deployphp/deployer/pull/2485, This PR enables users to run tasks outside a deployment by providing a new release_or_current_path option.
This option will always return the release_path if it exists and fallback to the current_path if it doesn't.
That way, outside of deployment, we can still run dep my:task and expect our task to be executed on the latest active release.
This PR replaces the use of release_path with release_or_current_path in all the custom tasks of platform-specific recipes — like laravel, symfony, etc. It also replaces the use of release_path in the deploy:vendor task, allowing users to run composer install on the current release if they need to.
Let me know if there are other places that would benefit from this new path.
Let me know what you think. 🙂

[ ] Bug fix #…?
[ ] New feature?
[ ] BC breaks? No, the existing behaviour stays the same but we're adding the possibility to also run tasks directly to the current release.
[ ] Deprecations?
[ ] Tests added?
[x] Changelog updated?

Awesome! Thanks for also fixed other recipes as well! Will release a new beta right now.


Multiple fixes to the parser output format documentation

@rpgoldman Thanks for the fixes! The mutex section actually always only contains pairs of predicates. I'll clarify this.


[1.20.1] Items inside of drawer appearing in other blocks info

DO NOT REPORT AN ISSUE IF YOU HAVEN'T TESTED THE ISSUE WITH THE LATEST VERSION IN CURSEFORGE
Functional Storage Version:
1.20.1-1.2.11
Titanium Version:
1.20.1-3.8.27
Crashlog If Applicable (Upload to pastebin/gist):

I dont add support for JADE so its probably not my fault


CPU and GPU usage

Describe the bug
Using more CPU and GPU usage that it should when animation is paused/cancelled
To Reproduce
Steps to reproduce the behaviour:

Start LottieSharp.Sample
Click on Pause Animation
CPU and GPU usage is higher then it should be

Expected behaviour
CPU and GPU usage should be lower then it is when animation is running
Desktop (please complete the following information):

OS: Windows 10 17763.437

Additional context
Mainly GPU usage should be lower when paused because it isn't doing any animating

@AzyIsCool Actually not because it's rendering the current frame. when you pause the animation you only pausing the animation progress not rendering and you can see the GPU usage because of rendering not animating

Maybe have a way to make the current frame a bitmap or a DrawingImage when paused then to allow usage to be lower because I know that they basically use nothing on CPU and GPU usage?

No. Not possible.


No support for x_auth_access_type with get_authentication_tokens method?

See https://dev.twitter.com/oauth/reference/post/oauth/request_token
The x_auth_access_type header is used when you want your app to be able to read-write in some cases and only read in others, and there's no way to set it right now, unless I'm mistaken?
Thanks!

I made a new branch bugs/371. I added a new method Twython.obtain_request_tokens, parameters are oauth_callback which is required, and the optional x_auth_access_type.
Let me know if this works how you expected, sorry for such a late response!

Sorry for a much later response. I haven't gotten a chance to test this as I'm not using twitter API anymore, but thanks for this fix!


Improve error handling in distribution lib

We should raise an error if the number of model replicas (batch dim) is not divisible by the number of processes because otherwise, the splitting fails (we don't support padding).

Codecov Report
Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 57.50%. Comparing base (d8b3314) to head (2ab5345).

Files
Patch %
Lines

keras/src/backend/jax/distribution_lib.py
0.00%
3 Missing :warning:

:exclamation:  There is a different number of reports uploaded between BASE (d8b3314) and HEAD (2ab5345). Click for more details.
HEAD has 6 uploads less than BASE

Flag
BASE (d8b3314)
HEAD (2ab5345)

keras
4
1

keras-torch
1
0

keras-tensorflow
1
0

keras-jax
1
0

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #20082       +/-   ##
===========================================
- Coverage   79.29%   57.50%   -21.80%
===========================================
  Files         501      501
  Lines       46955    46956        +1
  Branches     8652     8653        +1
===========================================
- Hits        37235    27003    -10232
- Misses       7971    18277    +10306
+ Partials     1749     1676       -73

Flag
Coverage Δ

keras
57.50% <0.00%> (-21.65%)
:arrow_down:

keras-jax
?

keras-numpy
57.50% <0.00%> (-0.01%)
:arrow_down:

keras-tensorflow
?

keras-torch
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Implement CRUD operations for Datasource Metadata

Description
This PR has the following changes:

Implementation of create and get methods for data source metadata
Adds DataSourceMetadataHelper - a helper class to fetch and process PromQL query metric data
Adds metadata constants

Type of change

[ ] Bug fix
[x] New feature
[ ] Docs update
[ ] Breaking change (What changes might users need to make in their application due to this PR?)
[x] Requires DB changes

How has this been tested?
Test Configuration

Kubernetes clusters tested on: minikube

Checklist :dart:

[x] Followed coding guidelines
[x] Comments added
[ ] Dependent changes merged
[ ] Documentation updated
[ ] Tests added or updated

Additional information
This PR has a dependency on PR 1144 to be merged first, as it uses the abstractions
Database changes for datasource metadata will be in next PR

@shreyabiradar07 Please rebase and fix the conflicts


Added contributors section to README.md for issue #38

Closes #38

Added contributors section to README.md
Included a dynamic image from contrib.rocks

The Changes -

The contirbutors 👍

nice work.


License ?

Hello, under what license is this project released under ? Thank-you.

All my projects are WTFPL


Get 0.29.x working on Python 3.11

This is a cherry pick of https://github.com/cython/cython/commit/9d1ffd5dcdcc0f2f719f86a9a30ce08a67641f10 and https://github.com/cython/cython/commit/3f56484704742c1af33c9737642125c55b0634d5 for Cython 0.29.x. It is not a hugely clean cherry-pick because __Pyx_PyCode_New has a different number of arguments in the two versions.
It also disables CYTHON_FAST_PYCALL on Python 3.11 (which I imagine is a performance regression). This is because the frame object no longer has a f_localsplus field to use. On Cython 3 this is fine - we use CYTHON_VECTORCALL instead, however that doesn't exist on 0.29.x. Therefore this is the least intrusive change available I think.
I've tested it to the extent of building Cython with these changes with Python versions 2.7, 3.7, 3.8, 3.9, 3.10, and 3.11 (twice, so it builds, then runs the compiled cython to rebuild itself) but haven't run anything more of the test-suite.

It also disables CYTHON_FAST_PYCALL on Python 3.11 (which I imagine is a performance regression). This is because the frame object no longer has a f_localsplus field to use. On Cython 3 this is fine - we use CYTHON_VECTORCALL instead, however that doesn't exist on 0.29.x. Therefore this is the least intrusive change available I think.

As an aside - this might be a good reason to target a release of Cython 3.0 sooner rather than later - it sort of feels like Cython the features needed to support modern versions of Python are in the Cython 3alpha only and it's beginning to diverge too much to easily maintain both.

It also disables CYTHON_FAST_PYCALL on Python 3.11 (which I imagine is a performance regression). This is because the frame object no longer has a f_localsplus field to use. On Cython 3 this is fine - we use CYTHON_VECTORCALL instead, however that doesn't exist on 0.29.x. Therefore this is the least intrusive change available I think.

Finally comment for now: It's probably possible to dedefine __Pyx_PyFrame_GetLocalsplus to go frame->interpreter_frame->localsplus and get the working again. But that's becoming a more significant change.

It also disables CYTHON_FAST_PYCALL on Python 3.11 (which I imagine is a performance regression).

That's fine. Users who really want Py3.11 support (and all of the other goodies) can switch to Cython 3.0 now. Everyone who chooses not to do that can certainly live with a reduced feature set, since that's specifically what they choose.

Closing and reopening to get a CI run.

Not complete, but good enough for a merge. Thanks!

Thanks for this cool fix!
I found this PR while trying to write a fix for include "longintrepr.h" while building lxml on Python 3.11. I wrote a fix and then I noticed that it's already fixed by this PR :-)
Would it be possible to have a Cython 0.29.x release including this fix?
It would help me to test my other changes to port projects to Python 3.11, like https://github.com/python/mypy/pull/11652 : running the mypy test suite requires to install lxml which uses Cython ;-)

Yeah, I was planning for new releases a few times but ran into various issues each time I tried. 0.29 finally has CI support again, so it's much safer now to try a release again. I'll give it another try soon.

Oh obvisouly, please tell me if I can help you ;-)

FYI, 0.29.25 is out.

Yeah, that's great! Thanks a lot @scoder! It's great to have Cython compatible with alpha versions of Python, it helps me a lot for testing.


Add option 'location'

Implementation of GNU xgettext --add-location option:
‘--add-location=type’
Generate ‘#: filename:line’ lines (default).

The optional type can be either ‘full’, ‘file’, or ‘never’. If it is not given or ‘full’, it generates the lines with both file name and line number. If it is ‘file’, the line number part is omitted. If it is ‘never’, it completely suppresses the lines (same as --no-location).

https://www.gnu.org/software/gettext/manual/html_node/xgettext-Invocation.html

awesome work!


Update capacity-planning-guide.md

@tysonn : Thanks for your contribution! The author, @billmath, has been notified to review your proposed change.


[Bug]: Pyarrow __getitem__ fails with integer for column position

Describe the bug
Unlike pandas-like and polars, pyarrow backend fails to return a series when sliced with column position integer
Steps or code to reproduce the bug
import narwhals as nw
import pandas as pd
import polars as pl
import pyarrow as pa

data = {"x1": [0, 1, 2], "x2": [.1, .2, .3]}

@nw.narwhalify
def func(df):
    return df[:, 1]

polars:
func(pl.DataFrame(data))

shape: (3,)
Series: 'x2' [f64]
[
	0.1
	0.2
	0.3
]

pandas
func(pd.DataFrame(data))

0    0.1
1    0.2
2    0.3
Name: x2, dtype: float64

pyarrow
func(pa.table(data))

TypeError: Got unexpected argument type <class 'slice'> for compute function

Expected results
Return series by index position
Actual results
Raise an error
Please run narwhals.show_version() and enter the output below.
System:
    python: 3.12.2 (main, Feb 25 2024, 16:35:05) [GCC 11.4.0]
    machine: Linux-<IP_ADDRESS>-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
     narwhals: 1.8.0
       pandas: 2.2.2
       polars: 1.7.1
         cudf:
        modin:
      pyarrow: 17.0.0
        numpy: 2.1.1

Relevant log output
No response

I will take it


[EBPF-424] Add calls to remove memlock to fix flaky tests

What does this PR do?
Fixes flaky tests in GenericMap tests by calling RemoveMemlock before each.
https://datadoghq.atlassian.net/browse/EBPF-422
https://datadoghq.atlassian.net/browse/EBPF-423
https://datadoghq.atlassian.net/browse/EBPF-424
Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided. Except if the qa/skip-qa label, with required either qa/done or qa/no-code-change labels, are applied.
[ ] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
[ ] If applicable, the config template has been updated.

/merge


Allow passing additional labels and annotations when using external tasks

Feature request
When using Tasks CRD as taskRef, all the annotations and labels need to be defined inside the Task resource itself.
There is no way to pass additional labels or annotations as far as I understood. https://github.com/tektoncd/pipeline/blob/a9b729b9e83959dbc0f00baeb83b9c8b46780f17/pkg/apis/pipeline/v1beta1/task_types.go#L176-L189.
Use case

We would like to use Vault sidecar injection to provide sensitive information into the Task and in order to enable the sidecar injection, we need to provide annotations that are required into the task.
The problem why we can't define the annotations statically in the task because the sidecar template will be generated dynamically based on the secrets that users want to pass into the task.

All labels from the TaskRun are attached to the Pod.  Not the annotations though atm.

@sbwsg I believe the annotations should be attached to the Pod as well, because I have successfully done this when I define the tasks in TaskSpec. However, if I want to use TaskRef, there is no way to define the annotation.
FYI, I have tried combining TaskRef (containing the actual steps to be executed) and TaskSpec (containing only the required annotations) at the same time, but I got this error admission webhook "validation.webhook.pipeline.tekton.dev" denied the request: validation failed: expected exactly one, got both: spec.pipelinespec.tasks[0].taskRef, spec.pipelinespec.tasks[0].taskSpec missing field(s): spec.pipelinespec.tasks[0].taskSpec.steps.

/remove-lifecycle rotten
/lifecycle frozen

/label help-wanted
It sounds like the really tricky bit here is that we don't have support for adding annotations / metadata from a PipelineRun's PipelineTask taskRefs. But we do support this for PipelineTask taskSpec.

Another use-case here would be for when you want to add annotations to specific PipelineTasks in a PipelineRun that map to functionality on a per-Task basis.

Ahh, interesting, I don't imagine we'd want to nest an additional metadata field inside TaskRef, since this would allow for some slightly odd scenarios like:
kind: TaskRun
apiVersion: tekton.dev/v1beta1
metadata:
  annotations:
    # ...
  taskRef:
    metadata:
      annotations:
        # ...

So either only allow the annotations to copied down from the PipelineRun to the TaskRun pods, or create a new type, PipelineTaskRef, as a copy of TaskRef with the addition of metadata?

Hi @sbwsg @wlynch, was looking into this issue and digesting the discussion..
thinking If considering the initial use case - annotations/labels to be applied for the task(s) as need, propagating these metadata perhaps leads to extra annotations on the pipeline and task(s) which don't need.
I tended to think about more on adding the optional metadata field into the existing TaskRef, as a new type (with a minor diff on the metadata) leads to more explanation on docs and bring a bit confusing here. Also, if thinking this requirement will be also applied on TaskRun/PipelineRun, one or two naming could be needed here.
But the concern on the confusing nest naming is true. while I was thinking if the structure of yaml could help here, such as the same name fields under the diff levels.

Let's clarify the problem a bit before we proceed here. I think we need to be sure of what we are trying to solve.
Below is an example Task + Pipeline + PipelineRun. Each resource has its own annotation (foo:foo, bar:bar, baz:baz):
kind: Task
apiVersion: tekton.dev/v1beta1
metadata:
  name: foo-task
  annotations:
    foo: foo
spec:
  steps:
  - name: echo
    image: alpine:latest
    command: ["echo", "hello world"]
---
kind: Pipeline
apiVersion: tekton.dev/v1beta1
metadata:
  name: foo-pipeline
  annotations:
    bar: bar
spec:
  tasks:
  - name: echo
    taskRef:
      name: foo-task
---
kind: PipelineRun
apiVersion: tekton.dev/v1beta1
metadata:
  name: foo-pipelinerun
  annotations:
    baz: baz
spec:
  pipelineRef:
    name: foo-pipeline

After running this through Tekton I observe the following:

The PipelineRun object is created with annotations  bar and baz. These come from the PipelineRun and the Pipeline.
A TaskRun object is created with annotations foo, bar and baz. These come from the PipelineRun, the Pipeline, and the Task.
Finally, the Pod is created with annotations foo, bar, and baz. These mirror what's on the TaskRun in (2).

Annotations are copied Pipeline -> PipelineRun -> TaskRun -> Pod.
The original use-case is to add annotations at "runtime" so that Vault can dynamically provision Secrets and mount them into a TaskRun Pod. We already know that TaskRuns copy their Annotations to their Pods, so the only remaining runtime type we need to worry about is PipelineRuns. taskRef is a field of Pipeline and is therefore an "authoring time" field and unsuitable for adding annotations at runtime. So I don't think adding metadata to taskRef solves for the original use-case. Instead the user should be able to define PipelineTask-specific metadata in the PipelineRun, since this is a "runtime" type.
Today we support a similar dynamic runtime behaviour for other fields of a PipelineTask by using PipelineRun's taskRunSpecs field. Maybe we could add annotations and labels field to that type, which would allow a PipelineRun to map annotations into one specific TaskRun Pod at runtime. @wlynch @austinzhao-go wdyt?

Guess I see your point. Thanks the logic explanation @sbwsg
As for letting fields "sit" in a proper context, here it will be more meaningful to "locate" the annotation(s) and label(s) under a runtime toward/specific field, like taskRunSpecs field (here the field name also can tell us). For the "authoring time", I understood like the field is for configuring the Pipeline work which is a more explicit step defined by user for their work/task(s).
And I agreed that the direction of adding these 2 (sub-) fields under a suitable existing field will be more straightforward (as for later docs and using).
A minor question here -- seems we only need to add annotation and label as (sub-) fields or still a metadata?

Yeah I completely agree @austinzhao-go - I think labels and annotations alone should be enough to meet the criteria of the original feature request.

/assign @austinzhao-go
@wlynch perhaps comments?
the draft for changes:
2 (sub-) fields, "annotation" and "label" will be added under the existing taskRunSpecs field while considering for the runtime context.
the following steps after the changes will be

[ ] Update related docs with added supporting and a clear use case

Just wanted to follow up on the discussion to add: since this modifies fields in one of Tekton's user-facing APIs it would need to go through a TEP. The process for taking a design through that is described here: https://github.com/tektoncd/community/blob/main/process.md#proposing-features

thanks @sbwsg for further details on the TEP thru chat and with some good examples I could structure as.
bring this changing idea for the TEP draft. target to ready it for collecting feedback in the coming week - week#15 of the year

Hi @sbwsg @wlynch, thinking if you have some examples and posts related to the use-case/requirement on memory?
When arranging the doc, I found I might lack of a bigger view over the problem space, like not limited to the single mentioned case. So I could keep the writing more general for the problem via more readings.

Hi @anthonysuterag, I am looking into this issue now. Perhaps any updates or future details you got for this use case in the past months?

related TEP
https://github.com/tektoncd/community/pull/690

thank you @austinzhao-go 🎉
/close


This is not how you deal with logits

Describe the bug
This algorithm produces the wrong outputs: https://github.com/ntedgi/node-efficientnet/blob/main/src/EfficientNetResult.ts#L26
To Reproduce
Steps to reproduce the behavior:

Set topK = 3
Let code predict something
All three labels are approximately 33%

Expected behavior
Get original model probabilities.
Screenshots
No.
Additional context
Google's mobilenet implementation illustrates how to deal with logits: https://github.com/tensorflow/tfjs-models/tree/master/mobilenet

which EfficientNetCheckPoint did you use?

Ah, this may be due to me using Efficientnet v2 which churns out logits.