Exec can't launch Logstash child

Hi,
I have the need in my workflow to launch a Logstash with a configuration depending of the value of an event in my logs.
The process launching the Logstash instance is another Logstash using the exec plugin. What I've noticed is that the child Logstash is unable to start because of some environment variables. I'm not sure whether it is a Logstash issue or an Exec issue.
First issue is the JAVA_OPTS variable :

WARNING: Default JAVA_OPTS will be overridden by the JAVA_OPTS defined in the environment. Environment JAVA_OPTS are  -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInit
iatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Xmx1g  -Xss2048k
Usage: java [-options] class [args...]
(to execute a class)
or  java [-options] -jar jarfile [args...]
(to execute a jar file)
where options include:
-d32          use a 32-bit data model if available
-d64          use a 64-bit data model if available
........
........
........
-agentpath:[=]
load native agent library by full pathname
-javaagent:[=]
load Java programming language agent, see java.lang.instrument
-splash:
show splash screen with specified image
See http://www.oracle.com/technetwork/java/javase/documentation/index.html for more details.
(Prepend -J in front of these options when using 'jruby' command)
After unsetting it in my script launching the child logstash, I got following exception :
Bundler::GemNotFound: Could not find gem 'ci_reporter_rspec (= 1.0.0) java' in any of the gem sources listed in your G emfile or installed on this machine.   verify_gemfile_dependencies_are_found! at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/reso lver.rb:328                                     each at org/jruby/RubyArray.java:1613   verify_gemfile_dependencies_are_found! at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/reso lver.rb:307                                    start at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/reso lver.rb:199                                  resolve at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/reso lver.rb:182                                  resolve at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/defi nition.rb:192                                    specs at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/defi nition.rb:132                                specs_for at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/defi nition.rb:177                          requested_specs at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/defi nition.rb:166                          requested_specs at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/envi ronment.rb:18                                    setup at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/runt ime.rb:13                                    setup at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler.rb:1 22                                   (root) at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/setu p.rb:18                                  require at org/jruby/RubyKernel.java:1040                                   (root) at /opt/logstash/vendor/jruby/lib/ruby/shared/rubygems/core_ext/kernel_requir e.rb:1
At this point I decided to unset every new environment variable that are created once we are in a Logstash process :
unset GEM_HOME; unset GEM_PATH; unset JAVACMD; unset JAVA_OPTS; unset RUBYLIB; unset RUBYOPT
And after that I was able to spawn a new Logstash with the exec plugin.
What do you think would be the best approach to tackle this issue ? Shall we write a fix for the Logstash project or for the Exec plugin ? Or we could simply document this behavior and keep it as is.
Let me know !

I have the need in my workflow to launch a Logstash with a configuration depending of the value of an event in my logs.

That's sounds crazy, but okay.

What do you think would be the best approach to tackle this issue ? Shall we write a fix for the Logstash project or for the Exec plugin ? Or we could simply document this behavior and keep it as is.

In general I favor inheriting the original environment unless it's important that the environment is under very tight control (like e.g. if you're running software builds or automated tests). One option could be to keep the original environment but introduce an environment option in the plugin that allows the user to set up the environment from scratch. Alternatively, cull the environment by default and allow the user to make additions.


Better line wrapping for markdown files (and other relevant file types)

VSCode Version: 1.8.1
OS Version: Windows 10

Not sure if this is possible, but it'd make the markdown editing experience better. Line wrapping currently only considers indentation. Is there any way to adjust the wrapping indent level based on additional contextual information?  Let me know if you have any thoughts

The wrapping indent computation is indeed at this time language agnostic and simply indents at the same indentation level. This can be somewhat configured via editor.wrappingIndent, but it would still be unaware of the syntactic meaning of -.

I was wondering if we can just have word wrap enabled or disabled by type.  It would generally just be for markdown files where I would enable word wrap though.

@trajano That sounds like a separate problem. With VSCode 1.9, you have have language specific settings such as:
"[markdown]": {
   "editor.wordWrap": "on",
   "editor.lineNumbers": "off"
}

@mjbvz thanks I found it too, it took a bit for me because I was using [md] rather than [markdown] and was wondering why it didn't work.  I wish there was an example but that's another problem.

I wonder if this could also be applied when wrapping jsdoc comments, e.g. - use virtual (uneditable) characters on wrapped lines to visualise the indent; e.g.
/**
 * Here is a really long text with many, many, many, many, many, many, many, many, many, many, many, many, many words
 **/

Could wrap like so:
/**
 * Here is a really long text with many, many, many, many, many, many, many, many, many, many,
 * many, many, many words
 **/

The indent and asterisk on the second line of the description would be a virtualised result of the wrapping, with the specific wrapping behaviour defined by the language mode.

To keep the number of issues in our inbox at a manageable level, we're closing issues that have been on the backlog for a long time but haven't gained traction: we look at the number of votes the issue has received and the number of duplicate issues filed. Thank you for your time and understanding.
If you disagree and feel that this issue is crucial: we're happy to listen and to reconsider.


Add page redirection to login when no permissions available

This is to handle the case when a user with a role that has no permissions. They should now be redirected to the login page where an appropriate error is displayed.
This also prevents the bug where the error is shown on the login page regardless if permissions are present.

Any tips on how to test this? Is there an easy way to create a user with no permissions?

No, unfortunately there's no way to create a user without permissions yet. The easiest way is to temporarily override the event.locals.permissibleQueries value being set in hook.server.ts to just being {}.


[K32W0] Update Docker image to accommodate SDK 2.6.9

This PR makes the necessary changes to accommodate SDK 2.6.9 for K32W0 platform.

@andrei-menzopol - please merge with master. I tried to resolve the conflict myself, however it seems that the NXP/matter.git has maintainer access disabled.

@andrei-menzopol - please merge with master. I tried to resolve the conflict myself, however it seems that the NXP/matter.git has maintainer access disabled.

done, thanks


Fix false proton gradient generating cycles.

I found a common type of erroneous energy generating cycle, a pair of transporters that move protons around for free, e.g. this one.
How do I fix these?
I know Devlin had a solution to this, it may have been his leakage reactions, but I don't remember for sure. The leakage reactions are for pairs of Metabolite objects that represent the same real-world compound in different subcellular compartments. Devlin's code created a new reversible reaction that reversibly interconverts them with a flux of
up to +/- bound (which should be a relatively small non-negative non-zero number). This represents "leakage" of this metabolite across the boundary separating the two compartments.
I'll ask Devlin.

Maybe a first step is to make a minimal model with this cycle, and show that is generates ATP. Then I can try to add the leakage flux to fix it.

I removed reaction 05206, and then the MEMOTE function didn't find any cycles anymore.

Daniel says the ion transporters shouldn't effect the kind of metabolism we care about anyways.


Added ability to create target server without ssl_info configuration

Hello,
ApiGee target server behaves differently when created like:
{
  "name" : "asdf-target",
  "host" : "host.123",
  "isEnabled" : true,
  "port" : 80,
  "sSLInfo": {}
 }

and like:
{
  "name" : "asdf-target",
  "host" : "host.123",
  "isEnabled" : true,
  "port" : 80
 }

So... there should be the ability to create such target server (without ssl_info object defined).
I have added that possibility, fixed acceptance tests for this resource and added new test checking new functionality. It would be great to have that merged :)
NOTE: small change in go-apigee-edge was needed to make it work (https://github.com/zambien/go-apigee-edge/pull/10).

Thanks for the contribution!


[SPIKE] [CT-2650] Materialization macros should support dispatch

Copied from https://github.com/dbt-labs/dbt-core/issues/4646
Adapter-specific materializations work differently from adapter-specific macros. Namely:

{% materialization table, adapter = 'snowflake' %} creates a macro named materialization_table_snowflake, rather than (as one would expect) snowflake__materialization_table
"Parent" adapter inheritance is not supported for materializations, as it is for all other macros. This causes confusion for external adapter maintainers: https://github.com/dbt-msft/dbt-synapse/pull/74#issuecomment-997102690, https://github.com/databricks/dbt-databricks/pull/40#discussion_r795513529

Let's make materialization macro generation / discovery work more like dispatch. We'd need to offer backwards compatibility / avoiding breaking changes for folks who currently rely on the materialization_<name>_<adapter> construction.

This is important in cases where we want to "call" a materialization macro from within another macro (perhaps from within another materialization). I found a concrete use case for doing this in my spiking on dbt clone (https://github.com/dbt-labs/dbt-core/pull/7258), whereby the clone materialization would in certain instances want to "shell out" to the view materialization.
Specific comment & spot in the spiking code where this is relevant: https://github.com/dbt-labs/dbt-core/pull/7258#discussion_r1155320591
Depending on the implementation we pursue for #7442, this could also be a relevant capability there.

Materializations are a variant of Jinja macro (jinja extension) which is created in core/dbt/clients/jinja.py, in MaterializationExtension.  They are saved in the manifest.macros dictionary with names such as: "macro.dbt.materialization_incremental_default"
In order to use adapter.dispatch and use its macro resolution code we would need to normalize the names. (Could we have synonyms?)  (Note: we currently have two macro resolvers, MacroResolver and MacroNamespace.)
In the "def execute" method in dbt/task/run.py and dbt/task/clone.py we do manifest.get_materialization_macro_name. Instead we would have to execute adapter.dispatch to get the materialization macro, which would return a MacroGenerator object so it wouldn't need to be created. We don't currently call adapter.dispatch in Python, but the materialization macros are the "launcher" macros that get everything started so this might be an exception.
Concerns here are the inconsistency in naming conventions between materializations and "normal" macros and the changes that would need to be made in the adapter repos. It remains to be seen how much incompatibility there would be. Possibly we might be able to have a temporary mapping from old to new materialization macros. But it's also possible that this would be a fairly big breaking change.

Users define these as:
{% materialization table, adapter="snowflake" %}

Lowest-risk solution is add the materialization macro by both names:
"macro.dbt.materialization_table_snowflake"
"macro.dbt.snowflake__materialization_table"

The main thing this gets us is allowing someone to write this and have it work:
adapter.dispatch("materialization_table")

We should raise a deprecation warning if people are still calling a materialization directly by the old name. In order to do this, we need to update our own materialization lookup logic (get_materialization_macro_name), so that we're fetching it by the new (rather than old) name.
Potential acceptance criteria:

Add materialization macro to manifest by both names
Change materialization macro lookup logic
Add deprecation logic when old materialization macro name is called explicitly

@aranke @gshank to keep discussing

On thinking about this a bit more, I think it's probably a bad idea to have duplicated macros with different names. It would complicate partial parsing quite a bit. I'm now thinking that the best way to do this would be to add code in adapter.dispatch so that if we fail to find a materialization macro we use special resolution code which will mangle the name into the "standard" materialization name.

Having a way to call a materialization as a function (with reliable dispatch + ability to pass in arguments instead of patching the context) would also give us more options for implementing this Mesh UX improvement:

#7442

In conversation with Kshitij just now, it seems that the main changes need to be in core/dbt/context/macros.py, in MacroNamespace.get_from_package where we can add special processing of "materialization" macros. The "get_from_package" method is called from adapter.dispatch, which comes from the adapter "contextproperty" of the ProviderContext in core/dbt/context/providers.py. The actual "dispatch" method is in the BaseDatabaseWrapper.
The materialization macro can be found in the "execute" method of the clone task (and probably the same method in runnable.py) by doing context.adapter.dispatch from the constructed context. This should already be a MacroGenerator callable, so we wouldn't need to construct a MacroGenerator as we do further down in "execute".

@gshank @aranke Thanks for continuing the conversation! I'm going to close out this spike, and open a new issue to implement what we've landed on above

Closing in favor of https://github.com/dbt-labs/dbt-core/issues/8031


Components with manf# and digikey# may create empty groups (crashing spreadsheet creation)

I pulled the fix for #302.
I instrumented the code in spreadsheet.py, added a debug breakpoint:
    # Then, order the part references with priority ref prefix, ref num, and subpart num.
    def get_ref_key(part):
        match = re.match(PART_REF_REGEX, part.collapsed_refs)
        try:
            return [match.group('prefix'), match.group('ref_num'), match.group('subpart_num')]
        except Exception as e:
            import pdb; pdb.set_trace()
            raise e

In this case, part.collapsed_refs is an empty string (i.e. repr(part.collapsed_refs) yields '').
$ kicost -i main.xml --include digikey  -var bat_pwr --overwrite -d1 --throttling_delay 0 --currency USD
Exchange rate: 1 EUR = 1.18 USD
# Getting from XML 'main.xml' KiCad BoM...
Getting authorship data...
Getting parts library...
Getting components...
# Removing do not populate parts...
Splitting subparts in the manufacture / distributors codes...
# Grouping parts...
Getting groups of identical components...
Checking the seemingly identical parts group...
Merging field asked in the identical components groups...
Propagating field values to identical components...
Create HTML page for parts with custom pricing...
Initialising digikey
Starting new HTTPS connection (1): www.digikey.com:443
https://www.digikey.com:443 "GET / HTTP/1.1" 200 45736
# Configuring the distributors locale and currency...
https://www.digikey.com:443 "GET /en/resources/international HTTP/1.1" 200 None
https://www.digikey.com:443 "GET / HTTP/1.1" 200 45736
# Scraping part data for each component group...

Problem merging price tiers for Digikey part CL10B102KB8SFNC with alternate packaging!
https://www.digikey.com:443 "GET /products/en?keywords=587-1210-2-ND HTTP/1.1" 200 None
https://www.digikey.com:443 "GET /product-detail/en/taiyo-yuden/UMK105CG820JV-F/587-1210-1-ND/930987 HTTP/1.1" 200 None
https://www.digikey.com:443 "GET /product-detail/en/taiyo-yuden/UMK105CG820JV-F/587-1210-6-ND/1817994 HTTP/1.1" 200 None
Problem merging price tiers for Digikey part 587-1210-2-ND with alternate packaging!

Writing the global parts informations...
Traceback (most recent call last):
  File "/home/epoulsen/.local/bin/kicost", line 11, in <module>
    load_entry_point('kicost==0.1.47', 'console_scripts', 'kicost')()
  File "/home/epoulsen/.local/lib/python2.7/site-packages/kicost-0.1.47-py2.7.egg/kicost/__main__.py", line 286, in main
    local_currency=args.currency)
  File "/home/epoulsen/.local/lib/python2.7/site-packages/kicost-0.1.47-py2.7.egg/kicost/kicost.py", line 348, in kicost
    user_fields, '-'.join(variant) if len(variant)>1 else variant[0])
  File "/home/epoulsen/.local/lib/python2.7/site-packages/kicost-0.1.47-py2.7.egg/kicost/spreadsheet.py", line 182, in create_spreadsheet
    parts, user_fields, collapse_refs)
  File "/home/epoulsen/.local/lib/python2.7/site-packages/kicost-0.1.47-py2.7.egg/kicost/spreadsheet.py", line 468, in add_globals_to_worksheet
    parts.sort(key=get_ref_key)
  File "/home/epoulsen/.local/lib/python2.7/site-packages/kicost-0.1.47-py2.7.egg/kicost/spreadsheet.py", line 467, in get_ref_key
    return [match.group('prefix'), match.group('ref_num'), match.group('subpart_num')]
AttributeError: 'NoneType' object has no attribute 'group'

main.zip

Your issue is related with some behavior of your XML file. Debugging, after KiCost group the part, it return:
...
['D301', 'D302']
D301 {'libpart': 'device:D', 'manf#_qty': '1', 'value': 'BAS16GWJ', 'manf#': 'BAS16GWJ', 'spn': '1727-7320-2-ND', 'manf': 'Nexperia USA Inc.', 'footprint': 'Diodes_SMD:D_SOD-123'}
D302 {'libpart': 'device:D', 'manf#_qty': '1', 'value': 'BAS16GWJ', 'manf#': 'BAS16GWJ', 'spn': '1727-7320-2-ND', 'manf': 'Nexperia USA Inc.', 'footprint': 'Diodes_SMD:D_SOD-123'}
[]
[]
['J9']
J9 {'digikey#_qty': '1', 'libpart': 'conn:Conn_01x02', 'manf#_qty': '1', 'value': 'B2B-PH-SM4-TB(LF)(SN)', 'digikey#': '455-1734-2-ND', 'manf': 'JST', 'manf#': 'B2B-PH-SM4-TB(LF)(SN)', 'spr': 'digikey', 'footprint': 'Connectors_JST:JST_PH_B2B-PH-SM4-TB_02x2.00mm_Straight'}
...
The [] means two empty groups of components. I never saw this behavior before, could you provide more information of your schematic? (If used some not conventional filed in the parts, KiCad version and the "Edit table field" list).
I am trying to track this behavior.

This issue appear to be created because the group algorithm and component without digikey# field.
The error just appear when Digikey distributor is active generating the empty groups.

I do a fair amount of Python -- Is this something I could dig into, or is it pretty involved?  Maybe give me a few hints around where these hooks are?

The problem is in the group_parts() function in the file eda_tools.py, need some correcting / re-manufacture. And I need to merge here the #266.
It is a function to group the the parts and propagate the the manf# / digikey# / ... to the semi-identical part (by default KiCost propagate the manf# to parts of same kind, footprint and value, that are empty).
All help and new developers are welcome.

KiCost works in this sequence:

Read the file (files);
Merge the groups;
Group the parts and propagate the manf# (the function here are the issue);
Scrape the distributors (as multi-thread);
3a. Configure the currency and language of the distributor (in beta / just working for Digikey);
3b. Look for the specific part number;
3c. Look for tree of part that match if 3b doesn't return;
Assembly the spreadsheet.

I am hitting this bug. Are there any work-arounds for this?

This bug are not present on new implementation #314  (will be released in few days).
If you would like to have a try, replace the current code installation by https://github.com/xesscorp/KiCost/tree/octopart
Re-open the discussion if the bug persists.


Use varargs instead of single-value parameters in $search/$setWindowFields query building API

This PR applies the following approach:

Instead of accepting T as a single-value parameter in an overloaded method, accept T.... This approach supports all usages supported by T, and more, i.e., it is more powerful.

When at least one value must be provided, use T first, T... more.
When omitting all values is allowed, use T....

Note that overloads accepting Iterable<? extends T> are still present in the API.
Commits are well-organized (don't tend to override each other) and may be reviewed one by one if that's more convenient.

Tests fail because AggregatesFunctionalSpecification.'$setWindowFields with empty output' fails. The server requires that the output document is specified, but allows it to be empty. It is weird, and I can't think of how would specifying empty output be useful. I suspect that back when the API was introduced, I added the test just because I observed this server behavior, but I doubt I figured out why the server behaves this way. Tomorrow (Jun 20), when the US employees come back to work, I'll ask the server API authors about this behavior.
PR can be reviewed regardless of the outcome: at the worst, I will roll back the last two commits that touch setWindowFields (39e2a219a8bc17ab3700178b6e4d157cdd502235, 3f61318c7f5fc2aa7fa7810aef125c3b1d307237).

my only concern is this approach adds yet another way of doing things in my API and if using it would confuse users. I don't think it would as the use of the API would be the same from a users perspective.

Yes, the using code looks exactly as if varargs were used without dedicating a parameter for the first value.

So the question is it beneficial to us developing it and should the 5.0 version of the driver adopt this approach over the other?

It does: method(T, T...) supports the same usages as method(T), and more. And the development price for that is negligible in my opinion (certainly smaller than the price of an overload).
If we decide that we should not use varargs at all in our API (which I would be totally fine with), then we should not have method(T, T...). So far there has been no discussions about forbidding them.


Initial EKS Implementaion

Added the following EKS endpoints with testing:

Create Cluster
Create Nodegroup
Delete Cluster
Delete Nodegroup
Describe Cluster
Describe Nodegroup
List Clusters
List Nodegroups

https://github.com/spulec/moto/issues/3870

I see the failing tests; looking into it.

Hey @bblommers, sorry to bother.   Looks like the 2.7 test is failing because the test_constants file it not found, but it's definitely there in the same directory/module.  What am I missing?
It also looks like all of the 3.* ones are failing due to S3 tests which I didn't touch and aren't included in my "files changed" tab so that may be something to look into?

Hey @ferruzzi, no problem. See my comments - that should get it working in Py2.
Not sure about the S3-tests, they all pass on my end. Let's see what happens with the init-file renamed 🤞

Alright, sorry for the delay.  I think I addressed all of your comments.  If I missed something, please let me know.   I ran the tests under 2.7 and under 3.8 this time to make sure I have the coverage you want.  Sorry for missing that before.

Hi @ferruzzi, I made some changes to get the server mode tests to run. Looks like the majority of the failures were due to missing URL paths.
Some negative tests are still going to fail (in server mode), because errors are not returned in the expected format.
Are you able to run a negative test case against AWS itself, for example the test when creating a cluster with a name that already exists, and intercept the HTTP response?
(In order to mimick the expected response from AWS, we typically have to format it similar to this: https://github.com/spulec/moto/blob/master/moto/dynamodb/responses.py#L111, but the exact response tends to change between services...)

Thanks for the help on that, and sorry about the linting on that last one.  I'll have a look at the error messages today and see what I can do.

Thanks for the curl-response @ferruzzi - that's exactly what I had in mind.

Is the responses.py file only used in server mode?
No, it's used for everything

Is there some documentation on what works differently in server mode

We should have some documentation on this, yes. One day. :)
In the meantime, in a nutshell:
There are three types of tests that verify different behaviours of moto.

Unit tests: Verify that the decorator works
ServerMode tests: Verify that Moto can act as a standalone server, so that users can point any SDK to it (Python, Java, .NET, etc.)
test_server.py: Verify that the build-in Flask server returns the expected raw data, what we would expect to see when using the HTTP API

The test_server tests has the advantage that we have complete control over the request and response, so we can verify scenarios that we can't test with just the boto3-client.
I'm about to push two tests for test_server.py with the correct assertions, and an incomplete implementation to return the correct response for these tests.

Alright.   I think what I am seeing in your commit there is that on exceptions.py:L13 the self.description should represent the expected curl result.  In which case I should add the nodegroup name, addon name, and the short version of the message to that dumps, as verified in test_server.py:L68.  Is that correct, or is the message being added somewhere behind the scenes that I am missing?

Hey @bblommers .   I've implemented the Server Tests, tested under py2.7 and 3.9, and squashed a few of the latter commits to keep your log a bit cleaner.

Looks like there's an issue with one of the tests in server mode:
=================================== FAILURES ===================================
________ test_create_nodegroup_throws_exception_when_cluster_not_active ________

NodegroupBuilder = <function NodegroupBuilder.<locals>._execute at 0x7f46e0a448c8>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f46f8446588>

    def test_create_nodegroup_throws_exception_when_cluster_not_active(
        NodegroupBuilder, monkeypatch
    ):
        client, generated_test_data = NodegroupBuilder(BatchCountSize.SMALL)
        expected_exception = InvalidRequestException
        expected_msg = CLUSTER_NOT_READY_MSG.format(
            clusterName=generated_test_data.cluster_name,
        )

        with mock.patch("moto.eks.models.Cluster.isActive", return_value=False):
            with pytest.raises(ClientError) as raised_exception:
                client.create_nodegroup(
                    clusterName=generated_test_data.cluster_name,
                    nodegroupName=random_string(),
>                   **dict(NodegroupInputs.REQUIRED)
                )
E               Failed: DID NOT RAISE <class 'botocore.exceptions.ClientError'>

tests/test_eks/test_eks.py:474: Failed

Test lives in /test_eks.py:L460

In ServerMode, moto runs in a separate Docker container, so it's not possible to patch the moto.eks.models.Cluster.isActive-attribute.
Probably easiest to only run this test in normal mode. (Example on how we skip tests, just with a different text: https://github.com/spulec/moto/blob/master/tests/test_acm/test_acm.py#L544)

Added the "skip if in server mode" check to three tests which manipulate time and one that uses a the patch mentioned above.

Codecov Report

Merging #3981 (740c613) into master (b670962) will increase coverage by 0.07%.
The diff coverage is 99.46%.

@@            Coverage Diff             @@
##           master    #3981      +/-   ##
==========================================
+ Coverage   94.88%   94.95%   +0.07%
==========================================
  Files         454      460       +6
  Lines       49513    50275     +762
==========================================
+ Hits        46982    47741     +759
- Misses       2531     2534       +3

Flag
Coverage Δ

servertests
55.89% <98.11%> (+0.20%)
:arrow_up:

unittests
94.83% <99.46%> (+0.07%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

moto/backends.py
69.23% <ø> (ø)

moto/eks/utils.py
88.88% <88.88%> (ø)

moto/__init__.py
98.30% <100.00%> (+0.01%)
:arrow_up:

moto/eks/__init__.py
100.00% <100.00%> (ø)

moto/eks/exceptions.py
100.00% <100.00%> (ø)

moto/eks/models.py
100.00% <100.00%> (ø)

moto/eks/responses.py
100.00% <100.00%> (ø)

moto/eks/urls.py
100.00% <100.00%> (ø)

moto/logs/responses.py
93.61% <0.00%> (-0.14%)
:arrow_down:

moto/s3/models.py
95.72% <0.00%> (-0.01%)
:arrow_down:

... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b670962...740c613. Read the comment docs.

Great success!
The ServerMode tests didn't play nice with the mocking inside the fixtures - not sure why. Mocking everything using decorators works fine though, that's why I changed them all

Nice!   Thanks!    As mentioned at the beginning of this journey, I'm going to need a that in a new release when you get the time so I can ingest it into an Airflow contribution to add EKS support, which is waiting for it.

Already taken care of @ferruzzi - 2.0.10 was released a couple of hours ago :)

Amazing.  Thanks!


Member updates 1.24.22

Pre-submission checklist:
Please check each of these after submitting your pull request:

[ ] Are you only including a repo_url if your project is 100% open source? If so, you need to pick the single best GitHub repository for your project, not a GitHub organization.
[ ] Is your project closed source or, if it is open source, does your project have at least 300 GitHub stars?
[ ] Have you picked the single best (existing) category for your project?
[ ] Does it follow the other guidelines from the new entries section?
[ ] Have you added your SVG to hosted_logos and referenced it there?
[ ] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[ ] Does your project/product name match the text on the logo?
[ ] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[ ] ~15 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

Build failed because of:
Empty twitter for Mail.Ru Cloud Solutions (member): https://twitter.com/mailru
Empty twitter for Mail.Ru Cloud Containers: https://twitter.com/mailru
Empty twitter for SeaTunnal: https://twitter.com/ASFSeaTunnel
Empty twitter for Spotinst Functions: https://twitter.com/spot_hq
Empty twitter for Karmada: https://twitter.com/karmada_io
No cached entry, and Sangfor EDS has issues with twitter: https://twitter.com/SangforUSA, Request failed with status code 404
Catalogic Software (member)  has a crunchbase https://crunchbase.com/organization/catalogic-software which does not start with 'https://www.crunchbase.com/organization'

LGTM @caniszczyk


⚠ Samsung has degraded performance

In 079eb9d, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2758 ms

Resolved: Samsung performance has improved in d46d854.


Authentication error when user decided to cancel the authentication via the directed federated IDP

Steps to recreate the issue

Create Google and Facebook as federated IDPs
Create an SP (ex: travelocity.com)
Connect the SP to authenticate with either Facebook or Google
Go to Travelocity app and select Facebook to authenticate
When the facebook login page is loaded select 'Not Now' option (scenario - User directed to the Facebook page, but then user decided not to sign in using the facebook, therefore select the 'Not Now' option to go back to the application login page)

Refer to https://wso2.org/jira/browse/IDENTITY-6713

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Unable to convert between types via numpy.astype

Consider the following code:
a = np.array([1, 2, 3], dtype=np.int32)
a = a.astype(np.float64)

The following code in C# does not work:
np.array(1, 2, 3, 4, 5).astype(np.float64);

Exception:
Test method NumSharp.UnitTest.NpRandomUniformTests.Basic threw exception:
System.NotImplementedException: _ChangeTypeOfArray from Int32 to Double
    at NumSharp.Backends.TypedArrayStorage._ChangeTypeOfArray(Array arrayVar, Type to_dtype) in C:\Users\Eli-PC\Desktop\NumSharp\src\NumSharp.Core\Backends\TypedArrayStorage.cs:line 135
   at NumSharp.Backends.TypedArrayStorage.SetData(Array values, Type dtype) in C:\Users\Eli-PC\Desktop\NumSharp\src\NumSharp.Core\Backends\TypedArrayStorage.cs:line 718
   at NumSharp.NDArray.astype(Type dtype, Boolean copy) in C:\Users\Eli-PC\Desktop\NumSharp\src\NumSharp.Core\Backends\NDArray.cs:line 132
   at NumSharp.UnitTest.NpRandomUniformTests.Basic() in C:\Users\Eli-PC\Desktop\NumSharp\test\NumSharp.UnitTest\Random\np.random.uniform.Test.cs:line 16

I can fix it simply by adding support for all type conversions in _ChangeTypeOfArray like it is in python.

No, thanks. I’ve already fix that issue on my local.

Was fixed in PR #287


Added Attribute control System.

@narumzj Just double-check the code on this when you get a chance.


📝 Update includes docs/en/docs/advanced/response-change-status-code.md

Thank you! @handabaldeep :rocket:


CRAN author name not read from DESCRIPTION file

I see you have struggled with the author names from the DESCRIPTION files already. I'm not sure why this one doesn't work: http://citeas.org/cite/https://cran.r-project.org/web/packages/rowr/index.html Perhaps the file is constructed incorrectly?
I wasn't sure if you would like this added on to #11 where author names are a secondary concern within a different issue, but please combine if more appropriate.

Hello. The reason that authors are not displayed for that package is because the owner has not formatted the DESCRIPTION file in a way the software can understand it. This web page shows the recommended way to format authors in a DESCRIPTION file, which is like this:
Authors@R: c(person("Matt","Nunes",role=c("aut","cre"),
        email="nunesrpackages@gmail.com"),person("Dennis", "Prangle",
        role="aut",email="d.b.prangle@newcastle.ac.uk"),
	person("Guilhereme", "Rodrigues", role="ctb",
	email="g.rodrigues@unsw.edu.au"))

The rowr package you referenced has the author formatted like:
Author: Craig Varrichio<EMAIL_ADDRESS>Looking at the ABCtools package, for example, it is parsed correctly by CiteAs due to the authors being listed in the proper format:
http://citeas.org/cite/https://cran.r-project.org/web/packages/abctools/
That being said, I scrolled through several packages and do see that some others are using the plain format. So I will attempt to support it shortly.

Hello again! I just made an update so almost every author format is pulled from the DESCRIPTION file now. The example you mentioned is working. Please take a look.


yaml schema import and builtin checks

Addressing #91 (yaml import) and also partially #74 (built-in checks).
This is just a draft pull request for making the development more transparent and offering a place to discuss the code.
With what I have now it is possible to create a DataFrameSchema from yaml. Only the Checks are not yet serialized.
The idea is that the parameters in the yaml file exactly reflect the keyword arguments of the Python code.
E.g:
 dataframe:
        checks:
        columns:
          int_col:
            dtype: Int
            checks:
            nullable: False
        strict: False

addresses the equally named parameters of the DataFrameSchema and Column objects.
So the Python equivalent would be:
DataFrameSchema(checks=None,
                columns={'int_col': Column(int, checks=None, nullable=False)},
                strict=False)

The yaml import function should move into the DataframeSchema as from_yaml(). But for now I kept everything in separate modules to avoid conflicts when the pandera module gets restructured.
Does this approach make sense so far? I am curious about any change ideas.

Codecov Report

Merging #97 into master will increase coverage by 0.5%.
The diff coverage is 96.49%.

@@            Coverage Diff            @@
##           master      #97     +/-   ##
=========================================
+ Coverage   92.98%   93.48%   +0.5%
=========================================
  Files           2        4      +2
  Lines         342      399     +57
=========================================
+ Hits          318      373     +55
- Misses         24       26      +2

Impacted Files
Coverage Δ

pandera/checks.py
100% <100%> (ø)

pandera/yaml_schema.py
92% <92%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f67050d...360add7. Read the comment docs.

Too many things have changed, in the discussions and with the restructuring. I will close this PR and rather create a new one from a clean branch when the time is right.


Documentation | Feature: make epub

Feature: make epub of the Volto documentation
Similar to one written in plone/training

Closed as this feature is probably not needed.


toFormat doesn't work in a loop

I have attached bignumber.js to a phaser.io project, and while the BigNumbers are working fine, the "toFormat" function is not.
Specifically, in the below code block (which is run once every second) , the function works the first time the code runs, and returns the properly-formatted BigNumber:
var format = {
groupSeparator: ','
, groupSize: 3
};
BigNumber.config({ FORMAT: format });
return this.moneyPool.toFormat(); //for 1000000, returns "1,000,000" during the first execution
However, the second time the code runs, I get this error on the last line (from Chrome console):
VM828:56 Uncaught TypeError: this.moneyPool.toFormat is not a function
at Object.GameModel.getMoneyPool (:56:35) //reference to the last line of the code block above
at Game.timerTrigger (:105:60)
at Phaser.Timer.update (phaser.js:67390)
at Phaser.Time.update (phaser.js:66682)
at Phaser.Game.update (phaser.js:38925)
at Phaser.RequestAnimationFrame.updateRAF (phaser.js:61994)
at _onLoop (phaser.js:61978)
This is completely baffling, and I don't know why it keeps happening. I did try to move the configuration code in an init function (so that BigNumber.config only runs once), but the exact same error occurred.
If it matters, I'm not using node; I just use the following line to include your file:
game.load.script('bignumber', 'vendor/bignumber.min.js');
Thanks.

If this.moneyPool.toFormat is not a function then this.moneyPool is probably not a BigNumber.

Afraid it is. My code is here: https://github.com/aphorism44/townfolkCartel/blob/master/js/GameModel.js. See line 18:
this.moneyPool = new BigNumber(1000000);
Also, as I mentioned, this.moneyPool.toFormat does run the first time, just not the second.

Okay, I found this:
When I logged out a typeOf for this.moneyPool, the first pass showed it was an object (correct), but the second pass showed it was a number.
Between the two loops, I do perform "normal" arithmetic on them, such as:
this.moneyPool += this.lastIncome;
Where this.lastIncome is a number, not a BigNumber.
I will guess this means that all operations on a BigNumber must be done with other BigNumbers, and using BigNumber methods (e.g., ".plus" instead of "+"). However, I couldn't find anything in the documentation on this happening, so it is very confusing.

Yes, you must use BigNumber methods not number operators, which are not normally applied to objects.
And yes, it would be better to just run config once.


Derived exceptions not caught by C#

For example:
class Test(SystemExit): pass

raise Test

does not terminate the process. The C# code expects a SystemExitException but the exception ends up being of type Exception.
Also, an IndexError (which becomes an IndexOutOfRangeException) would not be caught by LookupException handler in C# even though LookupError is a base of IndexError. For example:
try {
    PythonCalls.Call(context, somethingthatthrowsindexerror);
} catch (LookupException e) {
    // do stuff
}

would not catch the exception.
This came up in https://github.com/IronLanguages/ironpython3/pull/1011.

Probably related: raising a subclass of SystemExit displays a stack trace rather than exiting silently.
Found this in docopt module. It raises DocoptExit which is a subclasss of SystemExit.

Apparently, by design, any user-defined exception is a subclass of System.Exception, as per documentation. My guess is that this is to allow multiple inheritance.
A working way of catching Python exceptions in C# including their subclasses is with a when clause:
try {
    PythonCalls.Call(context, somethingthatthrowsindexerror);
} catch (Exception ex) when (PythonOps.IsInstance(ex.GetPythonException(), PythonExceptions.LookupError)) {
    // do stuff
}

Note: the methods used are currently internal. An idea: define a public helper method checking exception subclasses.

Apparently, by design, any user-defined exception is a subclass of System.Exception, as per documentation. My guess is that this is to allow multiple inheritance.

Hmm, still seems like it would be cleaner to have it be a subclass of the more specific .NET class (e.g. SystemExitException).

Note: the methods used are currently internal. An idea: define a public helper method checking exception subclasses.

As you know my preference is to have everything be internal unless there's a specific reason (or request) for it to be public. A helper method could be interesting (and possibly more efficient than an IsInstance call).
Anyway, at the very least we should special case SystemExit since we know it's an issue...

I guess multiple inheritance on user types makes it complicated to have something other than System.Exception as the .NET exception type:
class MyError(ValueError, AttributeError): pass


[5.7] Add an API to reload the workspace state from workspace-state.json

Cherry-picks https://github.com/apple/swift-package-manager/pull/4310 to release/5.7

Explanation: Adds an API to reload the workspace state, which will be used by SourceKit-LSP in https://github.com/apple/sourcekit-lsp/pull/484
Scope: Just moving code around in SwiftPM to create a new entry point for SourceKit-LSP
Risk: Very low, no functionality change, just moving code around
Testing: Ran test suite
Issue: rdar://89551074
Reviewer: Awaiting review on https://github.com/apple/swift-package-manager/pull/4310

@swift-ci Please test

replaced by https://github.com/apple/swift-package-manager/pull/5515


added windows 10 sdk link

as per the user request #2048 .   i inserted the windows 10 sdk link
https://developer.microsoft.com/windows/downloads/windows-10-sdk/

@RAJU2529 - Thanks for the PR, but I decided to add links to the content that is actually online. More people tend to use that instead of downloading the standalone chm file in the SDK.


KAFKA-3398: avoid just sending to one partition when record key is empty string

What if a user wants to use an empty string as a valid key? I think using null to represent no-key and assuming any other value is a key is safest. If a user want's they can put this logic in their client or even provide their own partitioner with custom logic.

+1 on keeping the current logic.

@granthenke @SinghAsDev Thanks for your reply. It's better to keep the current logic, and I will close this JIRA as 'Not a Problem'.


Unable to deploy Bot Framework Project ZIP using AZURE CLI

This is autogenerated. Please review and update as needed.
Describe the bug
Follow all the basic steps to make a Web App Bot in AZURE using this documentation
https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-deploy-az-cli?view=azure-bot-service-4.0&tabs=userassigned%2Cnewgroup%2Ccsharp
I am now stuck on the last step, deployment to the Web App in Azure
Command Name
az webapp deployment source config-zip
Errors:
The command failed with an unexpected error. Here is the traceback:
[Errno 21] Is a directory: '/usr/csuser/clouddrive/bot/EXPLORE_BOT/CSharp/Commerce/src/CommerceBot'
Traceback (most recent call last):
  File "/opt/az/lib/python3.10/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_job
    return cmd_copy.exception_handler(ex)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/command_modules/appservice/commands.py", line 39, in _ex_handler
    raise ex
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/command_modules/appservice/custom.py", line 528, in enable_zip_deploy_webapp
    return enable_zip_deploy(cmd, resource_group_name, name, src, timeout=timeout, slot=slot)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/command_modules/appservice/custom.py", line 554, in enable_zip_deploy
    with open(os.path.realpath(os.path.expanduser(src)), 'rb') as fs:
IsADirectoryError: [Errno 21] Is a directory: '/usr/csuser/clouddrive/bot/EXPLORE_BOT/CSharp/Commerce/src/CommerceBot'

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az webapp deployment source config-zip --resource-group {} --name {} --src {}

Expected Behavior
Environment Summary
Linux-5.4.0-1086-azure-x86_64-with-glibc2.28 (Cloud Shell), Common Base Linux Delridge (quinault)
Python 3.10.5
Installer: DEB

azure-cli 2.38.0

Extensions:
ai-examples 0.2.5
ssh 1.1.2

Dependencies:
msal 1.18.0b1
azure-mgmt-resource 21.1.0b1

Additional Context

route to CXP team

@ElvisOkereke Thank you for reaching out, we are looking into it.

@ElvisOkereke Hope you are passing --src the correct path with /filename.zip,


[QUERY] Not clear on how to create Application Insights using the SDK

Query/Question
I have many web applications I need to enable app insights for and am trying to script it out. I see examples for this on az / powershell but not for the SDK. I'm assuming this is possible but it's still not clear.
When I access the Azure interface I don't see any references to application insights that I can use. Is this possible, and if so how?
Why is this not a Bug or a feature Request?
It's a how to question - but I'm not sure if it's possible. If not, then it's a feature request :)
Setup (please complete the following information if applicable):

OS: Windows 10 Pro
IDE : VS 2019
1.29 Fluent

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[ ] Query Added
[ ] Setup information Added

Duplicate with #https://github.com/Azure/azure-libraries-for-net/issues/27. Please track the issue on the dependency one. Thanks.

@carsontwilson
You can use this SDK: https://www.nuget.org/packages/Microsoft.Azure.Management.ApplicationInsights/0.2.0-preview
The code is here: https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/applicationinsights
The management plane SDKs are mostly generated by AutoREST.
That version doesn't (yet) support the new Azure Identity SDK, so I created a bridge for you here:
https://github.com/jongio/azidext
This test: https://github.com/jongio/azidext/blob/master/net/JonGallant.Azure.Identity.Extensions.Tests/AppInsightsTests.cs shows you how to create an app insights resource
Please have a look and let me know if this solves your problem.
Thanks,
Jon

@yaohaizh - I reopened this because it isn't really a dupe.  This one is about using the SDK and the other one is about the Fluent SDK in a differnet repo so it is two different efforts.  Let's use this thread to help Carson figure out how to use the existing API.

@carsontwilson - Did you get this working?

@carsontwilson  We are going to close this issue because we haven’t heard from you in a few days, but please re-open it if the issue still persists.


Syntax error, unexpected tLABEL & syntax error, unexpected ',', expecting keyword_end

Issue Description
I'm new to Ruby and Fastlane.
I was trying to pass variables to lane provision. I called fastlane build_app but got syntax error below. IIRC, I still can pass variables this way with previous fastlane version (1.97.0).
Complete output when running fastlane, including the stack trace and command used
Error finding relevant GitHub issues: undefined method `map' for nil:NilClass

[!] Syntax error in your Fastfile on line 13: Fastfile:13: syntax error, unexpected tLABEL
      app_name: "my.app.name",
               ^
Fastfile:13: syntax error, unexpected ',', expecting keyword_end
Fastfile:14: syntax error, unexpected ',', expecting keyword_end
Fastfile:15: syntax error, unexpected ',', expecting keyword_end
Fastfile:17: syntax error, unexpected ')', expecting keyword_end

Configuration Files
Fastfile:
fastlane_version "1.97.2"

default_platform :ios

platform :ios do

  before_all do

  end

  lane :build_app do
    provision (
      app_name: "my.app.name",
      language: "English",
      app_version: "1.1",
      sku: "xxxxxxxxx"
    )
    screenshot
    build
    upload
  end

  lane :provision do |options|
    produce(
      app_name: options[:app_name],
      language: options[:language],
      app_version: options[:app_version],
      sku: options[:sku]
    )
    cert
    sigh(force: true)
  end

  lane :screenshot do
    snapshot
  end

  lane :build do
    gym
  end

  lane :upload do
    deliver
  end

end

Environment
fastlane version (run fastlane -v): fastlane 1.97.2
Do you use bundler to execute fastlane (i.e. bundle exec fastlane)? NO
Do you use a Ruby environment manager (e.g. chruby, rbenv, rvm)? NO

Removing the space in provision ( should fix this.

@neonichu Thank you, it almost took me several hours!


Fix netty-all artifact

Motivation:
612ab584489eae74b28ce0bc1b75fb23c5d24b57 did change the way how netty-all was produced by unfortunally it messed up the dependency graph for our native artifacts. This commit reverts changes done by 612ab584489eae74b28ce0bc1b75fb23c5d24b57 and also clean up the profiles
Modifications:
netty-all is useable again
Result:
Fixes https://github.com/netty/netty/issues/11272

/cc @fredericBregier

@normanmaurer OK, I will wait for a new release to be able to check, I think it shall be ok. Thanks !

@chrisvest sadly yes :/

/cc @candrews

Instead of reverting, I feel like fixing the problem would be preferable. I'll take a look - I see others are doing so as well

@candrews generally speaking yes... that said I think we need to release a regression fix release asap. And as it has shown we  need to take more time to really ensure that everything works as expected.

I agree with @normanmaurer. Since this turned out to be unexpectedly tricky, and there are other fixes waiting for release, I think it's a better trade-off to do the revert in this PR, regressing the netty-all class path behaviour to the state it's been for years prior, and then there's no time pressure on #11278.

Let's pull this in and then #11278 can revert this change and add its own changes on top of it.


[FEATURE] possibility to set ResultSetHandlerFactoryBuilder globally

This request is for integration with SMF. Please read here.

Vedi pull #330


fix Sphinx errors for tutorials and install ToCs

Description
This PR fixes Sphinx errors with the tutorials and install index pages.
Sphinx requires a line break after the option block and before the table of contents list.
It was throwing this error for tutorials:
/home/ubuntu/incubator-mxnet/docs/tutorials/index.md:1: ERROR: Error in "toctree" directive:
invalid option block.

.. toctree::
   :hidden:
   basic/index.md
   c++/index.md
   control_flow/index.md
   embedded/index.md
   gluon/index.md
   nlp/index.md
   onnx/index.md
   python/index.md
   r/index.md
   scala/index.md
   sparse/index.md
   speech_recognition/index.md
   tensorrt/index.md
   unsupervised_learning/index.md
   vision/index.md

It was throwing this error for the install page:
/home/ubuntu/incubator-mxnet/docs/install/amazonlinux_setup.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/build_from_source.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/c_plus_plus.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/centos_setup.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/download.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/osx_setup.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/raspbian_setup.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/tx2_setup.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/ubuntu_setup.rst:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/validate_mxnet.md:: WARNING: document isn't included in any toctree
/home/ubuntu/incubator-mxnet/docs/install/windows_setup.md:: WARNING: document isn't included in any toctree

@aaronmarkham Thanks for the contribution!
@mxnet-label-bot [pr-awaiting-review]


Questions about from_rotation_matrix() for the Rot2 class

Hi! I found that in Rot3 class, it has the method from_rotation_matrix():
https://github.com/symforce-org/symforce/blob/04ccac19aa25bf657c1f312539fc6964fd402128/symforce/geo/rot3.py#L190
This enables converting a $3\times 3$ numpy matrix to symforce.
However, for Rot2 class, it seems only support convert from_angle() or from_tangent().
Could you please give me some suggestions about how to convert a SO(2) matrix to symforce?
Thank you very much!

to_rotation_matrix for Rot2 is this:  https://github.com/symforce-org/symforce/blob/04ccac19aa25bf657c1f312539fc6964fd402128/symforce/geo/rot2.py#L148-L152
So to invert this, you could do:
def to_rotation_matrix(m: sf.M22) -> sf.Rot2:
    return sf.Rot2(sf.Complex(real=M[0, 0], imag=M[1, 0]))

There's should be a way of looking at all of the elements to give a Rot2 that's optimal e.g. by Frobenius norm when m is not exactly a rotation matrix, but I don't know the solution for that off the top of my head

Thanks for your help!
I assume the input matrix $R\in SO(2)$. In this situation, I found the method from_storage() possible to convert an SO(2) matrix to symforce.
https://github.com/symforce-org/symforce/blob/04ccac19aa25bf657c1f312539fc6964fd402128/symforce/geo/rot2.py#L50
This is a simple demo. The input matrix W_init must in SO(2).
import sym
import numpy as np

W_init = np.eye(2)
W_sym = sym.rot2.Rot2.from_storage(W_init[:,0])

The first column of W_init will be the input of sym.rot2.Rot2.from_storage().


White spaces not going no matter what i try

We have written the needed data into your clipboard because it was too large to send. Please paste.

White spaces not able to remove


Difficult to read the IME inputting text with the background color in GNOME

I can't remember when is the first time of I noticed...

https://www.reddit.com/r/chrome/comments/1gq03t6/search_result_text_turned_orange/ related?

I can reproduce it in GitHub dark theme.

<input id="query-builder-test" name="query-builder-test" value="" autocomplete="off" type="text" role="combobox" spellcheck="false" aria-expanded="false" aria-describedby="validation-ca30e773-5c61-4f56-9c97-4321e4785b75" data-target="query-builder.input" data-action="
          input:query-builder#inputChange
          blur:query-builder#inputBlur
          keydown:query-builder#inputKeydown
          focus:query-builder#inputFocus
        " data-view-component="true" class="FormControl-input QueryBuilder-Input FormControl-medium" aria-controls="query-builder-test-results" aria-autocomplete="list" aria-haspopup="listbox" style="width: 300px;">

color picker say it is #ffdd55
> pastel color ffdd55

  ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  Hex: #ffdd55
  ▀▀            ▀▀  RGB: rgb(255, 221, 85)
  ▀▀            ▀▀  HSL: hsl(48, 100.0%, 66.7%)
  ▀▀            ▀▀
  ▀▀            ▀▀  Most similar:
  ▀▀            ▀▀         gold
  ▀▀            ▀▀         khaki
  ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀         yellow

https://stackoverflow.com/questions/2781549/removing-input-background-colour-for-chrome-autocomplete

Also unreadable in default light mode...

https://zenn.dev/qnighy/articles/674e1dc8c93944

In Windows + Chrome

https://github.com/ungoogled-software/ungoogled-chromium/issues/3091
Might be same issue
https://github.com/kachick/dotfiles/blob/20275bb8844a3e9c9b2f87de8e78d39372911fd1/nixos/desktop/default.nix#L213
https://github.com/kachick/dotfiles/issues/689

https://issues.chromium.org/issues/40113488
:tea:


Add position and rotation to reset

This adds position and rotation to the reset message. The reset position is set by various mechanics (ex. by server lua setPositionRotation, by teleporting with F7 in-game, holding INSERT and recovering somewhere)
This prevents the vehicle from resetting at its initial spawn rotation, and interpolating in ways that could wreck other players cars.
This will require a version bump for the next release as it changes networked data

Add version bump to this pull request and we're good to go


[BEAM-7389] Add code examples for Filter page

Adding code examples for Filter page.
R: @rosetn can you take a look whenever you have a chance?

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Staged: http://apache-beam-website-pull-requests.storage.googleapis.com/9184/documentation/transforms/python/elementwise/filter/index.html

R: @rosetn can you take a look whenever you have a chance?

Please ping me after @rosetn reviews.

@aaltay All tests have passed :)


[Proposal] Embed credentials helper to simplify push / pull process

Currently in order to push / pull an image from Google Artifact Registry, we have to use GoogleCloudPlatform/docker-credential-gcr to generate the proper ~/.docker/config.json.
It works but there's a simpler UX solution: using the helper directly.
It's done by Kaniko for example:
https://github.com/GoogleContainerTools/kaniko/blob/eea12bd0259c37f04443cca47f4aac3bf82d8c31/tools/tools.go#L25-L27

Is the machine where you're running Timoni in GCP? If so, we can add --odic-provider gcp|aws|azure to all commands and Timoni will use context-based authorization to request a token for the given IAM role.
PS. The OIDC auth for Google Artifact Registry, ECR and ACR are already implemented in fluxcd/pkg/oci, we just need to wire it to flag.

Ah, didn't try passing the flag. Indeed I dive (after creating the issue) in the code and saw we were using fluxcd/pkg/oci. I was wondering if there's a reason behind the use of this library instead of github.com/google/go-containerregistry (used under the hood).
I was about to start a refactoring then think there might be a rationale behind that decision (maybe because of the link between timoni and flux).
The nice thing about go-containerregistry is the existence of multi keychain auth. Using that you can get rid of any cli arg, and push to multiple providers. UX is better imo.
I did not find any use of NewMultiKeychain in luxcd/pkg/oci but didn't dive a lot either tbh.

How is go-containerregistry implementing ODIC login and token refresh? In Flux we do it here https://github.com/fluxcd/pkg/blob/main/oci/auth/login/login.go#L127
As for UX, I prefer to explicitly tell the tool which auth method to use, instead of magically try to mix OIDC with existing docker configs.


Input buffer contains unsupported image format - only after comopsite

I get this error message only after calling composite, here is my code:
var png = sharp(buffer);
var data = await png.raw().toBuffer();
var alpha = Buffer.alloc(width * height);
// ... some calculation to create alpha channel base on the data
png.joinChannel(alpha, {
    raw: {width, height, channels: 1}
});
if(createBluredLayer) {
    var overlay = png.clone().blur(1)
    png.composite([{
        input: await overlay.toBuffer(),
        top: 0,
        left: 0,
    }])
}
var newBuffer = await png.toBuffer(); // this line triggers the error

So, if createBluredLayer is false, all goes well, if it's true, I get the error when calling toBuffer().
I don't understand how composite of the same image could change the format.

In this code example, the response of await overlay.toBuffer() will be raw, uncompressed pixel data due to the use of png.raw() so you'll need to describe it via raw:
png.composite([{
  input: await overlay.toBuffer(),
+ raw: { width, height, channels }
  top: 0,

In this code example, the response of await overlay.toBuffer() will be raw, uncompressed pixel data due to the use of png.raw() so you'll need to describe it via raw:
png.composite([{
  input: await overlay.toBuffer(),
+ raw: { width, height, channels }
  top: 0,

Thank you.
I would like to optimize this code as much as possible as I do it a lot.
Is there any difference, speed wise, if I create the overlay image using png.clone() or create a new one using the same buffer sharp(buffer)?

Thank you.
I would like to optimize this code as much as possible as I do it a lot.
Is there any difference, speed wise, if I create the overlay image using png.clone() or create a new one using the same buffer sharp(buffer)?

clone is for Stream-based input, so if you're using Buffer-based input it probably won't help.

clone is for Stream-based input, so if you're using Buffer-based input it probably won't help.

I encounter now another error, composite2: images do not have same numbers of bands.
My new code looks as followed:
var png = sharp(buffer);
var data = await png.raw().toBuffer();
var alpha = Buffer.alloc(width * height);

// ... some calculation to create alpha channel base on the data

png.joinChannel(alpha, {
    raw: {width, height, channels: 1}
})
.png();
if(createBluredLayer) {
    var overlay = sharp(await png.toBuffer()).blur(1)
    png.composite([{
        input: await overlay.toBuffer(),
        top: 0,
        left: 0,
    }])
}
var newBuffer = await png.toBuffer(); // this line triggers the error

I want to use the calculated alpha channel on both layers, sharp one overlayed on blured one in order to create soft edges to the transparent image.

I encounter now another error, composite2: images do not have same numbers of bands.
My new code looks as followed:
var png = sharp(buffer);
var data = await png.raw().toBuffer();
var alpha = Buffer.alloc(width * height);

// ... some calculation to create alpha channel base on the data

png.joinChannel(alpha, {
    raw: {width, height, channels: 1}
})
.png();
if(createBluredLayer) {
    var overlay = sharp(await png.toBuffer()).blur(1)
    png.composite([{
        input: await overlay.toBuffer(),
        top: 0,
        left: 0,
    }])
}
var newBuffer = await png.toBuffer(); // this line triggers the error

I want to use the calculated alpha channel on both layers, sharp one overlayed on blured one in order to create soft edges to the transparent image.


Merge from develop and bump version to 2.0.0dev19

Codecov Report

Merging #2052 into v2 will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##               v2    #2052      +/-   ##
==========================================
+ Coverage   92.64%   92.71%   +0.06%
==========================================
  Files          58       58
  Lines       10571    10702     +131
==========================================
+ Hits         9794     9922     +128
- Misses        777      780       +3

Impacted Files
Coverage Δ

botocore/__init__.py
100.00% <100.00%> (ø)

botocore/credentials.py
98.64% <0.00%> (-0.31%)
:arrow_down:

botocore/discovery.py
100.00% <0.00%> (ø)

botocore/eventstream.py
100.00% <0.00%> (ø)

botocore/retries/quota.py
100.00% <0.00%> (ø)

botocore/parsers.py
99.42% <0.00%> (+<0.01%)
:arrow_up:

botocore/waiter.py
98.68% <0.00%> (+<0.01%)
:arrow_up:

botocore/utils.py
97.74% <0.00%> (+0.01%)
:arrow_up:

botocore/session.py
97.82% <0.00%> (+0.01%)
:arrow_up:

botocore/retries/adaptive.py
98.57% <0.00%> (+0.02%)
:arrow_up:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcdef7...ce7b76d. Read the comment docs.


Generate ids with itertools.count

Avoids the critical section with threading.Lock in favor of itertools.count.
count objects are threadsafe, and their critical section is implemented in C and provide better performance that Python level locking.
Preliminary Benchmarks
This update also provides some speedup, I will make no claims on the factor as the benchmarks are not rigorous.
Setup: Just ran these on my M2 macos with Python 3.12.5, better benching methodology can be done if requested.
After
$ python -m timeit -s "from pipecat.utils.utils import obj_count, obj_id" "obj_id()"
10000000 loops, best of 5: 31.7 nsec per loop
$ python -m timeit -s "from pipecat.utils.utils import obj_count, obj_id" "obj_count(object)"
5000000 loops, best of 5: 93.8 nsec per loop

Before
$ python -m timeit -s "from pipecat.utils.utils import obj_count, obj_id" "obj_id()"
2000000 loops, best of 5: 130 nsec per loop
$ python -m timeit -s "from pipecat.utils.utils import obj_count, obj_id" "obj_count(object)"
1000000 loops, best of 5: 226 nsec per loop

Awesome, thank you! LGTM!


Update chokidar - macOS fixes

Please consider updating chokidar to the latest version (this library is a full major version behind). They have just fixed several issues around macOS and Docker that would improve file watching.

@goyney AFAIR, main reason behind that was chokidar dropping compatibility with Node 6 in the major release. Considering that forever is run on legacy systems first and foremost, we have to be very conservative about dropping support for even old Node.js versions.

Double-checked - yup, 3.0.0 is Node 8+-only

I mean, that is what semver is for. Those instances wouldn't update? Seems like this could be a major version bump.

@goyney That is true. However, that would mean that those older instances will stop receiving any fixes at all, unless someone backports them, and doing any changes in forever is painful already.
Many systems that are running forever are using deprecated Node versions. Even when we dropped Node 4 there were complaints from users.
Check these statistics:
https://nodejs.org/metrics/summaries/version.csv
https://nodejs.org/metrics/summaries/version.png
Node 6 is still very much a thing (even though personally I would very much preferred if it wasn't)
In general forever is mostly a legacy software, meant for legacy systems, currently on life-support. For newer systems using pm2 or nodemon is a much better option.

FYI here's an NPM warning on clean install:
npm WARN deprecated<EMAIL_ADDRESS>Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.

Here's how babel plans to deal with it without dropping node 6 support: https://github.com/babel/babel/pull/11560

@Hypnosphi OK, this is pretty smart. Would you be open to submitting a PR implementing the babel solution?

They mention that it doesn't work for Node 6, though?..


Decide on supported browsers

There are currently two known IE8 bugs:

class does not work (class must be set using the className property)
input type must be set before inserting an input element into the DOM (#60)

Currently, supporting IE8 seems to only require modest code changes, though it may have additional impacts on future development.
For reference, Polymer and Angular 2.0 are targeting IE10+, Ember 2.0 targeting IE9 and React targeting IE8, though some polyfills are required.

class does not work (class must be set using the className property)

This'll work in IE 8, but not < 8.
function test() {
   return {
    prop: document.body['class'],
    propName: document.body['className'],
    attr: document.body.getAttribute('class'),
    attrName: document.body.getAttribute('className')
  };
}

>>JSON.stringify(test())
{"prop":null,"propName":"","attr":null,"attrName":null}

>>document.body['class'] = 'prop';
>>test()
{"prop":"prop","propName":"","attr":null,"attrName":null}

>>document.body.className =  'propName';
>>test()
{"prop":"prop","propName":"propName","attr":"propName","attrName":null}

>>document.body.setAttribute('class', 'attr');
>>test();
{"prop":"prop","propName":"attr","attr":"attr","attrName":null}

>>document.body.setAttribute('className', 'attrName');
>>test();
{"prop":"prop","propName":"attr","attr":"attr","attrName":"attrName"}

In my mind IE9+ sounds okay.

I'd be okay with >IE9.

+1 for IE9+

Would it be possible to expose some constants (ie. IncrementalDOM.PROPERTIES) that could be manipulated by a compat script to support more browsers?
Something like you see here https://github.com/facebook/react/blob/64dafb17a9d8e20557bef29ee16585df6bfba2a8/src/renderers/dom/shared/HTMLDOMPropertyConfig.js#L44-L193
Although at that point you might as well include the compat part.

@thejameskyle this was already mentioned here https://github.com/google/incremental-dom/issues/67#issuecomment-122161450 so it seems that Incremental DOM indeed should be configurable by the front-end, otherwise it cannot satisfy all.

Would it be possible to expose some constants (ie. IncrementalDOM.PROPERTIES) that could be manipulated by a compat script to support more browsers?

I'd like to go this route as well. I'm working on a PR to compete with https://github.com/google/incremental-dom/pull/71 to do it.

So I don't want to open a can of worms here but I would argue that incremental-dom should try and support IE8 and up if polyfills are simple/unobtrusive enough. Partially if it is for something as fundamental as setting a class or input field.
I see a few arguments for this

Its a designed as small library that others build from which is quite different to a large and opinionated  framework like AngularJS
IE8 usage is declining but still in the >1% depending on which survey you look at and sites like http://data.gov.uk/data/site-usage?month=2015-06#browsers_versions show even higher usage.

At the very least give a hook to Polyfill functionally in the same way facebook does with react. http://facebook.github.io/react/docs/working-with-the-browser.html#browser-support-and-polyfills

I have just been experimenting with what would be needed to polyfill or fix any issues with IE8.
Currently the biggest problem is caused by trying to set '__incrementalDOMData' to Text elements. IE8 flat out refuses by throwing this error Object doesn't support this property or method.
Looking into it a bit more it seems like its caused by IE deciding if something has the expando is set to false which can also be globally set.
This has been removed from IE9 so given the changed that would be needed to fix that issue I would now argue in only trying to support >IE9. If thats the case then you can also simplify usages of Object.create(null) in #110 for example.

After some thought, I think the best way forward is to not support for IE8. The code makes a lot of assumptions about the way things work with respect to DOM and JavaScript engines which are likely not relevant to IE8. Supporting IE8 feels like it would be misleading as the performance might be worse than other alternatives or a future change might make that be the case. As the usage share is relatively low and there are other alternatives to Incremental DOM that do support IE8, it doesn't seem to make sense to support officially or otherwise.

+1
On Sep 10, 2015 12:34 PM, "Sepand Parhami"<EMAIL_ADDRESS>wrote:

After some thought, I think the best way forward is to not support for
IE8. The code makes a lot of assumptions about the way things work with
respect to DOM and JavaScript engines which are likely not relevant to IE8.
Supporting IE8 feels like it would be misleading as the performance might
be worse than other alternatives or a future change might make that be the
case. As the usage share is relatively low and there are other alternatives
to Incremental DOM that do support IE8, it doesn't seem to make sense to
support officially or otherwise.
—
Reply to this email directly or view it on GitHub
https://github.com/google/incremental-dom/issues/69#issuecomment-139355384
.

It is perfectly fine to be slower on older browsers. Not supporting them at all is the easiest path to go. When we selected React, supporting IE8 was the single winning feature that no one had. It looks that one will keep it on top for the future.
The browser version usage statistics are almost never based on internal applications for organizations, which usually keep working for decades.

So this may be of interest to some people but I have an experimental version of idom for ie8 browsers here https://github.com/markuskobler/incremental-dom/tree/ie8
It does lose the functionally to pass in formatters into text('#format me', function() {...}) but I figure that can be implemented at a higher level if needed.

+1 @gramic


Command input being erased (on server)

When being on server, all of the chat messages visually erases your commands input, but command still work after pressing enter.

Try to write commands in chat

Moved into #109 for now


Calculated Policy - AWS > S3 > Bucket - Match tags on Bucket and CMK

https://github.com/turbotio/expeditors/issues/296

Describe the solution you'd like
We want to see if we can cross check two different resources.  For example, say I have an S3 bucket with PII data.  I would want that bucket encrypted with a kms key which is specifically for encrypting PII data.  To achieve this, I would have a compliance tag on my S3 bucket classifying it as PII as well as a compliance tag on my kms key classifying it as a key for encrypting PII data.  I’d want to be able to check that the compliance tag value on the S3 bucket matches the compliance tag value on the kms key and return compliant if they are the same.
Scenario 1:
Given: The initial context at the beginning of the scenario, in one or more clauses.

When: The event that triggers the scenario.

Then: The expected outcome, in one or more clauses.


⚠ Samsung has degraded performance

In ae32fc7, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2823 ms

Resolved: Samsung performance has improved in 95eef49.


storage: create data directory when RegisterMetadata is called

This fixes a bug in the recently merged continue on not ready functionality (#20).
Previously we'd only create the data directory within each volume's directory in the tmpfs when WriteFiles is called. Because WriteFiles is not called if the volume is not ready to be issued for, this directory never existed and so the data directory couldn't be mounted into the pod when Mount was called.
By moving the creation of the full directory structure to the point where RegisterMetadata is called, we then allow the volume's data directory to be mounted even if no data has been written to it yet.
/cc @JoshVanL @7ing

Nasty!
This change makes sense to me. Thanks @munnerz
Let's publish a patch release when this is merged
/lgtm


It's... uh.. Simple

and it worked lol

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2d3aa751-f468-0e8b-8d00-7b6f273c01dc
Version Independent ID: 1dc42d66-6f89-b55b-ebd0-73232935194b
Content: How to: Group Windows Forms RadioButton Controls to Function as a Set
Content Source: docs/framework/winforms/controls/how-to-group-windows-forms-radiobutton-controls-to-function-as-a-set.md
Product: dotnet-framework
GitHub Login: @dotnet-bot
Microsoft Alias: dotnetcontent

We're pleased that you found that the topic worked, @CloudOS.


Compile time improvement

The Compile time is very slow.
I improved it.
Compiling the cleaned project before:
37.4 seconds
Now:
15.8 seconds
Pipeline before:
4 minutes
Now:
2 minutes
I also removed an old timer for my parser ^^

I sadly can't test if it's faster or not as it anyways takes minutes for me instead of seconds :(. What is/was the exact reason for the slow compilation time?

I think the View that I changed was too complicated. It had many View Modifiers If else and so on and  I guess it took longer to type check. Splitting it up helped the compiler to deduce the correct type faster


Some rules do not work well

I found it during #6 (https://github.com/Quramy/typescript-eslint-language-service/blob/setup-linter/src/ast-converter.ts#L4) 🤔
There is no eslint CLI error report but my-plugin reports @typescript-eslint/no-unused-vars .

Minimal reproduce code:
import { Hoge } from "./types";
export const hoge: Hoge = { };

It seems that the no-unused-vars rule selector for type reference does not hit ? 🤔
https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/src/rules/no-unused-vars.ts#L43-L45


The filter seems to corrupt certain files

Hello,
After testing the latest filter, we found some case where when we upload certain files, the files that gets saved is "corrupted".
You can find an offending file here: https://github.com/julienfromentc/struts1filter-issue/raw/master/PDF.PDF, in this case the resulting PDF will be blank page.
Update:
It seems the issue happens with the following characters:

81
8D
8F
90
9D

Those characters are undefined in Windows1252 (http://en.wikipedia.org/wiki/Windows-1252). After the string conversion, those chars are transformed to '?'. So if we could convert the read the inputstream in an other encoding, the issue could be fixed

Sorry for the delay.
I tried settting the request encoding to UTF-8 before reading the inputStream: request.setCharacterEncoding("utf-8");
I also added the original encoding to the InputStreamReader:
String characterEncoding = this.getCharacterEncoding();
if (characterEncoding == null ) {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
}
else {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream, characterEncoding));
}
But I still get some bytes converted. Note that those bytes are read that way by inputStream. You can get the bytes from body and print them and you will see that the bytes after %PDF-1.2% are efbfbdef ... instead of e2e3cfe30d0a...
@rgielen any ideas?

@pwntester @rgielen Did you guys had any chances to look at this issue a bit?

nothing new since last update. @rgielen can you please look at it? Any idea if Struts is finally going to release a new version using the secure BeansUtil version??

In below PR, which is raised to struts repo, has change in RequestUtils which checks request.getParameterNames alone. But this filter checks input stream also. Is it really needed to check input stream?
https://github.com/apache/struts1/pull/1

Yep, it is since payload can be placed in a multipart post request

I think using a fixed version of Struts 1 is a much better option than using the filter since unfortunately its not as stable as we would like it to be. And actually applying a patch should not be that difficult. The official patch uses a new version of BeanUtils that basically applies the regular expression. It should be similar to:
https://gist.github.com/ykare/1fb2d5d11e16cb0003c3#file-2-beanutils-patch-struts1-0-2
But instead of patching BeanUtils (used by Struts), you can directly patch Struts library, like this pull request proposes:
https://github.com/apache/struts1/pull/1/files
Another alternative is to download a struts version that comes with the patch. Fedora has applied it:
http://koji.fedoraproject.org/koji/buildinfo?buildID=552377

I'm quite late in chiming in on this issue. We had to implement the filter temporarily to pass security requirements. I too came across this issue for multipart form data. Tracked issue back to the conversion of the InputStream which is byte[] array to a char[] array. You cannot convert binary data to char and back again and hope it won't change! The fix for this is to instead read the full byte[] stream and store that, whilst still doing conversion of body to char array to check for malicious strings. The subsequent call in getInputStream() should return the original byte[] array instead of body.
Side note that the 1.0.1-SNAPSHOT also doesn't work. It appears that the removal of the call to super.getParameterNames() in constructor causes the filter to fail! I haven't debugged through to see why its needed, however leaving it in keeps everything working

A test of a simple Struts 1 application extending Action
https://github.com/julianvilas/rooted2k15/tree/master/struts-tester/struts1/struts1.struts1-helloworld-0.0.1-SNAPSHOT/src/main/java/com/vaannila
worked without the filter but showed no input with the filter.  My test consisted of submitting good (userName=foobar, password=foobar) and bad (userName=foobar, password=foobar2) inputs and expecting the respective displays of success.jsp and failure.jsp.
Using Alberto Fernández's maintenance branch of Struts 1 which included the fix in ActionServlet and the fix in RequestUtils protected against the RCE/Tomcat attack and held water with a form submission test of expected and unexpected benign inputs.


An approach to consider multiple slides per patient with weak labels.

Discussed in https://github.com/jamesdolezal/slideflow/discussions/311

Originally posted by lysukhin September 13, 2023
Hello, @jamesdolezal and contributors!
First of all, let me thank you for this wonderful project that you develop and support. This work is really huge and important.
I have a question on possible plans to enable the following:
Usually, there is more than 1 histological slide per patient, but there's only 1 weak high-level label (e.g. malignant/benign) per patient. So the challenge is that not all of the patient's slides do actually provide meaningful information and features for this single label, like only 2 of 5 slides do really contain malignancy signals while the other 3 are actually 'clean'. Nevertheless, without additional annotations of each slide, this brings us to the same problem of noisy data as classical per-tile classification with weak labels.
So I wonder if there are any plans on how to deal with it, maybe in some shape of adding one more aggregating 'MIL'-level on top of features for all patients slides? So the model can choose among all feature bags, not just a set of them from a single slide.
Thanks for your attention and work, and looking forward to hear from you.

Thanks again for the suggestion - we can continue high-level discussion in #311. I've created the branch 316-mil-slide-aggregate, and I'll update this issue when a first-pass implementation is ready.

This feature has been implemented on the dev branch. You can now aggregate bags of features at the patient level, rather than the slide level, by setting the MIL parameter aggrevation_level to "patient" (defaults to "slide"):
from slideflow.mil import mil_config

config = mil_config(..., aggregation_level='patient')

You can then train as usual. The saved predictions, slide manifest, and reported metrics will all reflect patient-level predictions rather than slide-level predictions.
Thanks again for the suggestion! And thanks to @matte-esse for the implementation. Let us know if you encounter any issues with this new feature or have other suggestions.

Actually going to reopen this for now until we can do more testing - looks like evaluation might not be working correctly.

Wonderful news, will certainly check this out in a few days. Thanks for the responsiveness and pace!

Ok - latest update on dev (c601f85) should have this patched. Training and evaluating an MIL with aggregation_level="patient" should work as intended.  Let me know if you encounter any issues! Otherwise, we'll fold this into the next 2.1 release.

Going to try this on custom data in a day or two, can post feedback here as an early access tester :)

This has now been released in version 2.2.0, going to close the issue out for now.
Thanks again for the suggestion! Let me know if there are any issues.


The logs uploader module is not enable for the flutter_drone recipe

@jason-simmons:

The recipe library that uploads logs set both FLUTTER_LOGS_DIR and FLUTTER_TEST_OUTPUTS_DIR to the same path
(see https://flutter.googlesource.com/recipes/+/refs/heads/main/recipe_modules/logs_util/api.py#27)
So tools should be able to use either one.  (I don't know why both of these variables exist given that they appear to be  identical)
AFAICT the flutter_drone recipe used to run the test linked above is not using the log upload library.  The recipe itself would need to be modified in order to get that functionality.

That's unfortunate, as flutter_drone is used over 100+ times.

That's unfortunate, as flutter_drone is used over 100+ times.

I would happily review a recipes CL for this. See go/flutter-luci-recipes for the documentation on how to contribute to the recipes.

That's unfortunate, as flutter_drone is used over 100+ times.

I would happily review a recipes CL for this. See go/flutter-luci-recipes for the documentation on how to contribute to the recipes.

🫡 . I'll put this on my background list of tasks for next week.

Appears to have stuck this time as https://flutter-review.googlesource.com/c/recipes/+/59349.


Fix params.EpochDays to params.NextEpochDays

Problem Definition
Currently, end-of-epoch uses params.EpochDays directly when allocating farming rewards. However, if params.EpochDays are changed by Parameter ChangeProposal in the middle of the epoch, the already started epoch will perform farming allocation based on the changed params.EpochDays. Therefore, even if Epoch Days is changed in the middle of the epoch, it is necessary to ensure that the changed Epoch Days can be used in the next epoch without affecting the farming reward allocation of the epoch.

Proposal
To this end, CurrentEpochDays(global state) need to be added to check the current Epoch Days, and the existing params.EpochDays is renamed and repurposed to params.NextEpochDays, so if CurrentEpochDays and params.NextEpochDays are different at the start of the new epoch, then update CurrentEpochDays to params.NextEpochDays

Bullet Points

[ ] Add CurrentEpochDays or EpochEra
[ ] Rename and fix logics for params.EpochDays to params.NextEpochDays
[ ] Update spec and description

References

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

Ref: https://github.com/tendermint/farming/pull/89


Questions Regarding Model Training Requirements and Dataset

I am interested in your RECCE project  and have a few questions regarding the training requirements and dataset preparation:

Minimum hardware requirements for training:
Could you please specify the least GPU that can be used for training? Is it possible to train the model using a single 3090 GPU?
Regarding the dataset preparation, I extract the images using c40. But the dataset is still very larage. Could you provide an estimate of the final size of the datasest after the extraction?

Hi, thanks for your interest in our project.

In experiments, we used multiple 2080Ti GPUs, and I think a single 3090 is also sufficient to run the code. If you encounter an "out of GPU memory" issue, you can decrease the batch size.
After preprocessing (extracting images from videos, face alignment, and cropping), we got about 90G data for FaceForensics++ c23 and c40 in total.

Thank you for your response！We tried using RetinaFace as you mentioned to extract the facial images from the sequences and store image, but we found that it seems that RetinaFace can only process images. How did you process the video?

Hi, to process the video, you may need to extract frames from each sequence first. This can be done with the cv2.VideoCapture API.

Close due to inactivation. Please feel free to reopen this issue if you still have related problems.


Update normal-module-replacement-plugin.mdx

describe your changes...

[x] Read and sign the CLA. PRs that haven't signed it won't be accepted.
[x] Make sure your PR complies with the writer's guide.
[x] Review the diff carefully as sometimes this can reveal issues.
[x] Do not abandon your Pull Request: Stale Pull Requests.
Remove these instructions from your PR as they are for your eyes only.

@atadevos Can you accept CLA?

Accepted

I'm wondering if it's the same issue as https://github.com/webpack/webpack.js.org/issues/5059