Change us-west-2 check, as only working in EC2 instances

In pull request #424 API access to store._running_in_us_west_2() is added in the form of a printed statement that the user is (or is not) running in AWS region us-west-2.
Unfortunately the check in store._running_in_us_west_2() only works for EC2 instances. It is not working for example for ECS (Elastic Container Service) instances running in region us-west-2. Now the big question is: is it intended that only EC2 instances in region us-west-2 can access the data OR can it be any computing instance running in region us-west-2.
In the second case store._running_in_us_west_2() could be adapted using boto3 for checking the region following the code snippet in issues #231 :
if (boto3.client('s3').meta.region_name == 'us-west-2'):
    return True
else:
    raise ValueError('Your notebook is not running inside the AWS us-west-2 region, and will not be able to directly access NASA Earthdata S3 buckets')
    return False

The check is intended to verify in-region execution but shouldn't be limited to EC2, I think your change would be a valid PR if it works the same way in EC2!

Thanks @betolink. This is great to hear!
I did not use exactly the same code snippet as given in #231. But we used the following to get the current region, which should also work from EC2:
my_session = boto3.session.Session()
my_region = my_session.region_name

I don't think boto3/botocore is going to do what you want -- namely, determine which region you're actually running in.
Boto is going to pull the session information from your AWS config or AWS_* environment variables, so it's more checking what region you're configured to access (what APIs to hit) than what region you're actually running in.
For example, on my laptop:
>>> import botocore.session
>>> botocore.session.get_session().get_config_variable("region")
'us-west-2'

because I have my default region in my ~/.aws/config set up like so:
[default]
region = us-west-2

Likewise, if I instead do:
>>> import os
>>> import botocore.session
>>> os.environ['AWS_DEFAULT_REGION'] = 'us-east-1'
>>> botocore.session.get_session().get_config_variable("region")
'us-east-1'

AFAIK, from an EC2 instance, the only way to determine what region the AWS instance is running in is to hit this special IP address:
http://<IP_ADDRESS>/latest/meta-data/placement/region

And you'll need to handle IMDSv1 or IMDSv2 metadata rquests -- see this answer on SO:
https://stackoverflow.com/a/77902397
This should work on ECS configured for EC2, but I don't know if it works on ECS configured for Fargate (I suspect it will though).

It looks like on Fargate, the AWS_REGION environment variable is set, so the botocore method should work there, but it's not robust in that it's just checking environment variables that are mutable and used to primarily select which APIs to interact with, not to determine what region you're running in.

On ECS, here's a summary of available metadata:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint.html
So, determining this is non-trivial and requires knowing a bit about the service you're running in.

Thanks @sabinehelm, and agreed!
@jhkennedy, can you comment on what you'd recommend for #231 and #424? If I understand what you're saying correctly, our current implementation in #424 would not properly keep the user from running a notebook out of region (or tell us if someone really is working out of region), depending on how their config parameters are set.

@JessicaS11 I'm not sure -- this is hard.
So, I'll start by saying the answer to:

Was the spirit to check specifically for us-west-2, or to enable the user to see what region they are running in?

It should probably be yes to both, or at least confirm what region they are in.
So, overall, I think I'd recommend:

creating an earthaccess.aws module with:

get_ec2_region, which has most of the current implementation inside _running_in_us_west_2
https://github.com/nsidc/earthaccess/blob/main/earthaccess/store.py#L146-L155
get_fargate_region, which would need to do the same, but for fargate as detailed here:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/fargate-metadata.html
(maybe?) get_ecs_container_region as detailed here:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/container-metadata.html
(maybe) get_config_region which would use boto3/botocore as in #424
And then a get_region (get_running_region?) method that'd go through the above 4 in order and return the first one that didn't raise (or return None depending on implementation). If it hit 4, I'd probably also throw a warning like "Region inferred from AWS config or environment variable and may not represent the region you're running in."

Then you could also have a convenience method like
def earthaccess.aws.ensure_in_region(region: str = 'us-west-2') -> bool:

on top of it and use it in the Earthaccess Store.
Note: Actual method/function names could be improved

All that said, this seems like something that should already exist so it'd be worth spending some time searching GitHub/PyPI...

For ec2, it might be worth just using this package:
https://github.com/adamchainz/ec2-metadata
It looks well maintained, is sponsored, and is a "critical" project on PyPI.
I still don't see anything for ECS and Fargate, however.

I like what you're proposing @jhkennedy, and one would think this should already be a package that would work in all the execution environments in AWS, EC2, ECS, Lambda, etc

ec2-metadata will not work on any z2jh instances by default, as access to the metadata server is explicitly blocked by default (https://z2jh.jupyter.org/en/stable/administrator/security.html#block-cloud-metadata-api-with-a-privileged-initcontainer-running-iptables).

If I test the current approach to verify in-region execution it works from Openscapes (2i2c) is this the same endpoint? @yuvipanda
http://<IP_ADDRESS>/latest/meta-data/placement/region
https://github.com/nsidc/earthaccess/blob/e187ae001162d7b844e621c53abbbf26d87305f3/earthaccess/store.py#L140

@betolink yes, because we've intentionally unblocked that access point in the openscapes hub :) But we coupled it with appropriate IRSA roles so it is secure. At least when I last looked, just unblocking access to the metadata server without setting up IRSA or similar was pretty insecure.

Ah, this reminded me of an issue the VEDA hub reported, when using earthaccess the library didn't detect in-region execution and used the HTTP links, this is probably why this is happening. cc @abarciauskas-bgse

yeah, i'd suggest (similar to @jhkennedy elsewhere) to look at possibly looking for the redirects coming to figure out whatever you need to do internally, as ultimately everything else is going to be only a heuristic. For example we're going to do https://github.com/2i2c-org/infrastructure/issues/3273 soon for the openscapes hub, not sure what effect that will have on ec2-metadata.

@betolink sorry for the delay here but I verified that an updated earthaccess (v0.8.2) does open via S3 direct access on VEDA, whereas the current version installed on VEDA's Hub (v0.5.2) does not properly register that the instance is in-region. Hopefully this will be resolved once https://github.com/NASA-IMPACT/veda-jh-environments/issues/41 is addressed.

Overheard from maintainers of oss.smce.nasa.gov:

yes, we do indeed block the instance metadata (for security reasons), so the check that earthaccess is making is not ideal for what they’re trying to do.


Add milestones for SSL Handshake timings

Add two milestone entries to track TLS handshake timing.  Can add to the custom logging format for later analysis.

[approve ci clang-format]

the ts_lua plugin exposed milestone functionality as well. so i will update the plugin with the new milestones once this PR is merged.

Yes, we've been running with this code on our older image.


Add a scroll bar if the code validation error is too big to fit in the screen.

Can't see the full content (or close it) if the popup is too large:

I would limit the size of these hint popup's  - make tutorial authors constraint how much they can show here.  Get's very un-usable fast.


SublimeHQ.Sublimetext2 version 2.0.2

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Please use the PackageIdentifier: SublimeHQ.SublimeText.2


Fix corrupted gateway statistics for downlink packets.

There was an issue with the mapping between what's received from the Semtech Packet Forwarder over UDP and the Gateway Bridge stats.
Addresses https://forum.loraserver.io/t/rxpacketsreceived-ok-counter/2687

Great spot!

Great spot indeed and thanks for your fix!


[feature request] make MutableHashTableOfScalars and MutableHashTableOfTensors trainable

System information

TensorFlow version (you are using): 1.12
Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.
This issue is related to issue #19324, in which we need some technique to do embedding on a large sparse hash table, so that we can reduce memory usage and the probability of hash collision.
As far as I can see, ops like MutableHashTableOfScalars and MutableHashTableOfTensors in lookup table ops can be used to meet this demand. The main problem is that they are not trainable.
In other words, what we need may be a feature like this:
# define a Hash Table
table = tf.contrib.lookup.MutableHashTable(
    key_dtype=tf.string,
    value_dtype=tf.float64,
    default_value=-1)
params = table.lookup(["w1", "w2", "b"])
# define other ops with params
...
# optimize the table
train_op = optimizer.minimize(loss, var_list=[table], global_step=global_step)

Will this change the current api? How?
Yes, the current API would be changed.
In order to implement a feature like this, we may need to:

do modification on the lookup table ops to make it trainable. This involves several parts of fix like:

some fix in python layer as only variables are trainable now.
several new ops may be add in lookup table so that we can update the gradients.

modify the api of MutableHashTable, so that we can pass more args, like:

shard it to different parameter servers.
when value for a key doesn't exist, we can initialize it with different initial value.
we can make the KV pairs have TTL.

add a new API like embedding_lookup_sparse, or do modification on it.
make sure the checkpoint for hash tables can work properly

Who will benefit with this feature?
As a matter of fact, sparse models in recommend system may benefit with this feature a lot.

Looking forward to the feature.

Looking forward to the feature.

this is very useful and important . In China, tensorflow is dramatically used in recommender system. However, when we want to learn something from user's behavior, large embedding layers are always required.

Looking forward to the feature.

Looking forward to the feature.

This was discussed in https://github.com/tensorflow/tensorflow/issues/19324 and the RFCs from Tencent/Alibaba are under discussion https://github.com/tensorflow/community/pull/237 .

Hi,
Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.
The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.
Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.
Thank you!


Can't register qmi.powerstrip.v1 on Smartthings

I have manually added qmi.powerstrip.v1 in fison67/mi-connector:test version.

I try to register the device

I got the error - Undefined

My another device works correctly.
Logs

error Failed >> ADD Device Error : physicalgraph.app.exception.UnknownDeviceTypeException: Device type 'Xiaomi Power Strip' in namespace 'fison67' not found.
debug Data >> [modes:[object Object],[object Object], power:false]
debug Try >> ADD Xiaomi Device id=65848971, type=qmi.powerstrip.v1

Thanks! I added manually DTH on Smartthings and now it works.


Fix #3 in windows_sid

The issue resolved in windows_productkey.rb is also present in windows_sid. This PR fixes it the same way.
https://github.com/liamjbennett/puppet-win_facts/issues/3

Please can we get this merged? It prevents anyone from using this module if they're using the popular and puppetlabs supported registry module.

Looks good. Thanks Liam. Do you know when you'll get this release to puppet forge?


Update priority-accounts.md

Made changes to the navigation to add priority accounts.
fixes#https://github.com/MicrosoftDocs/microsoft-365-docs/issues/10811

@JohanFreelancer9 Please copy review

@kwekuako Please merge this PR. Thanks

@kwekuako Gentle reminder.


Keras Version Mismatch Error When Running LST-AI on University of Michigan HPC

Hello,
I hope this message finds you well.
I am attempting to run the LST-AI container on the University of Michigan HPC system but keep encountering an issue. The process runs for about an hour (on CPU) and then fails, showing an error message indicating a version mismatch with Keras. Since this is a container, I am unsure how this could be the issue or how to resolve it.
Attached is an image of the error for reference.
Below is the singularity call I am currently using to execute LST-AI:
singularity exec --nv -e -B pre:/custom_apps/lst_input:ro -B lst_output:/custom_apps/lst_output -B lst_output:/custom_apps/lst_temp lst-ai_latest.sif lst --t1 /custom_apps/lst_input/3DT1.nii.gz --flair /custom_apps/lst_input/FLAIR.nii.gz --output /custom_apps/lst_output --temp /custom_apps/lst_temp --device cpu
I would greatly appreciate any advice or guidance you can provide to resolve this issue. Please let me know if there is additional information or context I can provide to assist in troubleshooting.
Thank you for your time and assistance!

Hi @sopkoc,
Sorry to hear you're running into issues! This problem sounded familiar, so I checked, and it seems related to this issue, which was resolved by downgrading to Python 3.9 (I noticed you're using Python 3.10 as well). At the moment, our setup.py does not enforce this restriction, but we may need to consider adding it.
Could you share how you're building the Singularity container? Are you compiling this by yourself, or are you using the pre-compiled docker from dockerhub?

Hey,
I am using the docker and just converting it to a singularity so it is compatible with our HPC

Sorry, just to confirm again, are you getting the docker from dockerhub (i.e. docker pull jqmcginnis/lst-ai:latest), or compiling the docker container with the Dockerfile by yourself? :slightly_smiling_face:

I am actually using it as a singularity container from your docker container of a HPC. I am working out how to pin python version 3.9 in the container.
module load python /3.9
module load singularity
singularity exec --nv -e -B pre:/custom_apps/lst_input:ro -B lst_output:/custom_apps/lst_output -B lst_temp:/custom_apps/lst_temp lst-ai_latest.sif lst --t1 /custom_apps/lst_input/3DT1.nii.gz --flair /custom_apps/lst_input/FLAIR.nii.gz --output /custom_apps/lst_output --temp /custom_apps/lst_temp – device cpu

Hey,
without me editing your container, any recommendations on how to get it to run with python 3.9?

Hey Cindy, I am working on an updated version of the container, will let you know once I have it! I should hopefully have it today or tomorrow! Thank you so much for your patience!

Ran it with the new container and it runs fine on our HPC now! Thanks so much for fixing this in your container!!


Redshift Version Fix  available_plugins.md

Fixing Redshift Version

Thanks. Could you amend and sign off the commit? See previous commits for ref, or CONTRIBUTE.md

Done Tal

Will create a new request for the same,


Invoking commands from the command panel is broken, but right-click command works fine

When I remove newlines or paste without newlines from command bar, the cursor is thrown up in the document while nothing is pasted or formatted. If I do these actions while my cursor is on a list environment, the frontmatter is messed up - newlines are added in front of the document. However, right click commands are perfectly fine.

I'm sorry to hear it doesn't work as intended. Thank you for creating an issue! Could you give me some more information on your environment?

What OS are you using?
Is your obsidian version the most recent one, or are you using a much older version?
Do you have any other text-editimg related plugins installed?
Can you give me a minimal example that duplicates the bug?

That would allow me to duplicate and study the bug so I may fix it.

Should be fixed now in 1.0.4. Thank you for reporting the bug. Please update your plugin and let me know if it works.

This works now, many thanks!


FEAT: Added past and upcomming events

Added all HackNights 2023 Season's Episodes
Added Bits 2023
Added HackNights Ep 1 Season '24

Wrong PR, sorry for the inconvenience


[HELM] Mention gp2 as the default installation setting for Pachyderm's pvs and add IOPS/Disk size requirementss

AWS: https://docs.pachyderm.com/2.0.x-beta/deploy-manage/deploy/amazon_web_services/
gp2 is back to being the default disk type for our pvs.
Update documentation to give users both options (given that gp3 is not the default but is the production recommendation)
AB#70

@nadegepepin can you verify that this is done?


Bug for __post_init__ logic

Problem description
When nesting dataclasses, if there is a default value for inner dataclasses that impacts the logic in their __post_init__ method, the new values from command line do not impact the logic in __post_init__ .
Code to reproduce:
@dataclass
class ArgumentsOne:
    num_gpus: int
    num_cpus: Optional[int] = None

    def __post_init__(self):
        if self.num_cpus is None:
            self.num_cpus = 2 * self.num_gpus

@dataclass
class ScriptArguments:
    dataset: str = "X"
    one: ArgumentsOne = field(default_factory=lambda: ArgumentsOne(num_gpus=1))

print(tyro.cli(ScriptArguments))

Command: python main.py --one.num_gpus=2
tyro version: tyro==0.5.18
Current output:
ScriptArguments(dataset='X', one=ArgumentsOne(num_gpus=2, num_cpus=2))

Expected output:
ScriptArguments(dataset='X', one=ArgumentsOne(num_gpus=2, num_cpus=4))

Related problem
There is also a similar situation for sanity checks. For example, if ArgumentsOne checks to make sure we have GPUs when asking for them with an statement like the following in the __post_init__, we will face a similar problem.
...
class ArgumentsOne:
    ...
    def __post_init__(self):
        ...
        if self.num_gpus > 0:
            # raise value error if we do not have access to gpus

In this case, if the original script is configured to run on a machine with GPUs by default but we want to run it on a machine without GPUs with --one.num_gpus=0, we will get a value error.
Example use cases
Huggingface transformers adds a significant amount of logic to its TrainerConfig __post_init__ method, for example here.
This is used in this example script. For the reasons explained above, you cannot use this script as is on older GPUs even if you do --training-args.bf16=False and you will get an error saying that your machine does not have the ability to use bf16 data type.

Hi!
To clarify your first example, if you wrote it (equivalently) as:
@dataclass
class ScriptArguments:
    dataset: str = "X"
    one: ArgumentsOne = field(default_factory=lambda: ArgumentsOne(
        num_gpus=1,
        num_cpus=2,
    ))

would you expect --one.num-cpus to default to None, or default to 2?  Your problem seems to be because tyro chooses the latter behavior.
In your latter problem, I speculate that the problem with this is:
class ArgumentsOne:
    ...
    def __post_init__(self):
        ...
        if self.num_gpus > 0:
            # raise value error if we do not have access to gpus

when you've written:
    one: ArgumentsOne = field(default_factory=lambda: ArgumentsOne(num_gpus=1))

tyro needs to run the default_factory in order to inspect what you want the default value to be (and then allow it to be overridden). This will instantiate an ArgumentsOne object with num_gpus=1, which will throw an error. Is that incorrect behavior?

Thank you for your quick response!
To avoid confusion, let me first explain the ideal outcome in my use case and answer your questions at the end.
I think there are two approaches or design choices for dealing with the default values.
The first approach is to first create a default object and then override the values one by one. I am not familiar with the inner workings of tyro, but it seems like this is what tyro is doing currently. This will look something like this:
# python  main.py --one.num_gpus=0

object = ScriptArguments(one=ArgumentsOne(num_gpus=1))
object.one.num_gpus = 0
# We continue doing this for all key values mentioned in the command line.
return object

The downside with this approach is that we are unable to influence the logic in the __post_init__ from the command line and it is always executed with default values. So, the logic in __post_init__  always assumes that we are requesting one GPU (i.e., num_gpus=1) and throws an error if our machine does not have GPUs even if we specify on the command line that we do not need GPUs with --one.num_gpus=0.
The alternative approach is to merge the values from the command line with the default values and instantiate a new object. Basically, to determine the key values passed as arguments to the constructor of the dataclasses, use the key values passed on the command line. But, if a key value is in the hard-coded default in the script but not on the command line, use the hard-coded default value. This will look something like this:
# python main.py --one.num_gpus=0
object = ScriptArguments(one=ArgumentsOne(num_gpus=0))

# python main.py --dataset=mnist --one.num_gpus=0
object = ScriptArguments(dataset='mnist', one=ArgumentsOne(num_gpus=0))

# python main.py --dataset=mnist --one.num_cpus=2
object = ScriptArguments(dataset='mnist', one=ArgumentsOne(num_gpus=1, num_cpus=2))

# python main.py --dataset=mnist
object = ScriptArguments(dataset='mnist', one=ArgumentsOne(num_gpus=1))

Following this approach, we can change the behavior of the __post_init__ from the command line. For example, if we specify on the command line that we do not want GPUs (--one.num_gpus=0), then the __post_init__  logic is not gonna check if the machine has GPUs and will not throw an error, which makes the script more portable.
Another example of __post_init__ logic is derived values. In a simple case, if the value of field num_cpus is derived from the value of num_gpus by some logic, then regardless of what we choose for num_gpus from the command line, num_cpus is gonna get the same value since __post_init__  logic uses  the default value of num_gpus to calculate num_cpus. The only option is to also override num_cpus from the command line as well, which in this simple case is feasible. But, for more complex libraries like huggingface, there are many many variables to override on the command line and the logic is sometimes too complex to apply manually.
To answer your questions:

would you expect --one.num-cpus to default to None, or default to 2? Your problem seems to be because tyro chooses the latter behavior.

We want it to default to 2 because in the code sample you provided it is specifically mentioned that num_cpus should be 2 when instantiating the new object. And if we follow the second design approach, which is the desired behavior in this case, then since we have not specified a value for num_cpus on the command line, it should use the default value of 2.

... which will throw an error. Is that incorrect behavior?

I would say it is not the expected behavior for this specific application and in general for situations where the __post_init__ implements a significant amount of logic.

If the developers decide not to cover these more complex cases like in huggingface, then tyro is doing what it is expected to and this is not incorrect but that tyro is not the suitable tool to use in this case.
But if the intention is to be able to use tyro in this case, then, the behavior is incorrect and the class should not be instantiated with num_gpus=1 when we specify on the command line --one.num_gpus=0.

I would also like to briefly discuss the strength of tyro in my workflow specifically with huggingface in case it is helpful since tyro is still under active development.
One of the amazing things that huggingface does in libraries like transformers and trl is that it provides a huge number of options that cover most use cases. But HfArgumentParser is not compatible with nested structures. So you have to either watch out for all the possible name collisions that occur now or in the future between different configuration objects or alternatively, define a set of command line arguments that you want to use and then initialize the configuration objects manually with command line values. The latter is what most scripts do which makes for lengthy scripts like this one. or this one (Here and Here in the code)
tyro is quite useful here because it supports nesting so no worries about name conflicts and unlike the second example, we do not have to update the script every time we want to use a new flag for huggingface Trainer that we have not used before.

Thank you! It makes perfect sense. I think then there is nothing that can be done for when the default initialization throws an error because it has to be run with the default values to generate helptext, etc. before the user even enters custom values through the command line. But I think the end user can work around it by choosing more general defaults that are more portable.
As for the problem of derived values, it seems like the issue arises only when part of the __post_init__ logic is executed in certain situations like when some field like num_cpus is None. And now the second time that tyro calls the __post_init__ function after overriding the values, then the logic is ignored since the field (num_cpus in this case) already has a value as a result of the first __post_init__ call and is not None anymore.
I think my proposed approach is not applicable currently because tyro has no way of knowing how default_factory creates the instance so it cannot update the constructor arguments and instantiate a new object, it can only call default_factory which always initializes the object the same way with default values.
Maybe in the future, there can be an extra utility that takes a callable and default arguments to create the instance. For example something like tyro.custom_factory(init_fn, init_args={key: value}) and then to create the final object, it can combine the command line values with default values (e.g., custom_args = init_args + {values specified on the command line}) and call the given init function with these values to get the final object, something like init_fn(**custom_args).


Fixed Mixins

Oh boy, this is embarrassing- fixed misnamed mixin properties.

Nice catch, Dana.  Good luck on your crusade. :)
Lee - does the version bump need to happen because of the docs?

I figured it was worth a bump since existing code changed...  When do you normally bump minor versions?

Oh, my bad - I didn't see the code line change.  Yeah, anytime it's more than docs/comments, I typically bump.


  We should remove the file opt-out as it is not flexible enough for all situations.

I agree that we should only have one way to opt-out. If .NET Core uses the environment variable only, then we should do the same.

I agree - we should not litter the home folder.

Is opting out an option during the install now? Personally I would prefer to have to opt in.

@thezim

environment variables are stored in the in registry so this shouldn't be an issue.

We can set environment variable by GPP but security men require forced security policy.

Let's be clear that any new telemetry will likely be opt-in and be described in a RFC.  The current basic startup telemetry we have now helps provide data of PSCore6 adoption and determines if we increase or decrease our investments.  Since we ship packages outside of GitHub, we can't just track download numbers (which are unreliable anyways).


Add new step: Xcode project info 1.0.1

Hello, here's a new version with minor fixes suggested in #430.

Sorry, I forgot to mention: can you please delete 1.0.0 from your repository/fork? Right now both versions would be shared with this PR

Please also squash the commits if you want to update this PR - or you can just delete the related branch in your fork completely, and re-share the step

Ok, I've just deleted it and re-share the step.


Publish thanks.json with package

Hi,
I tried to use "thanks" but the following error occured:
IOError: [Errno 2] No such file or directory: '<my-virtualenv>/lib/python2.7/site-packages/thanks/thanks.json'

This PR should fix it :)

+100

Well now I feel sheepish.


Trim plugin

Trims blank lines from the beginning and end of code blocks, as well as removing indents and end-of-line whitespace.
This makes the "remove-initial-line-feed" plugin redundant.
This plugin needs to run before those that attach to the "complete" hook (e.g. line-numbers), or strange things may happen. There didn't appear to be any way to ensure this when running on "language-none" blocks, so I added a new "wont-highlight" hook for this use.

I like the idea.
FWIW, to match \s without [\r\n] you could do something like /(?![\r\n])\s/
More info on this technique: http://lea.verou.me/2012/05/hacking-lookahead-to-mimic-intersection-subtraction-and-negation/

Yes, that would arguably make it more readable. The negative lookahead would need to be repeated for every character though (i.e. /(?:(?![\r\n])\s)+$/), which I imagine could be quite slow…
Here's a JSPerf test: http://jsperf.com/negated-vs-modified-class. Safari does especially poorly with the lookahead version for me.

Closing this, since the whitespace normalizer plugin already does this. Thanks for your efforts, I hope you contribute again in the future!


Examples of MSDN entries of value not yet ported to MDN but already redirecting

-ms-touch-select: https://msdn.microsoft.com/en-us/library/hh975292.aspx

-ms-content-zooming: https://msdn.microsoft.com/en-us/library/hh771891.aspx

:unamused:
(At least there are short descriptions at https://msdn.microsoft.com/en-us/library/hh920761.aspx)

Thanks @Herst -- I've temporarily removed the redirects for the -ms- prefix CSS property pages that don't yet have equivalents on MDN (so they will load on MSDN). We are in the process of porting them over in the next few weeks and will then reimplement the redirections.
Hopefully I identified all of them, but if you can't find something you're looking for, don't hesitate to re-open this issue and I'll fix it ASAP. Thanks!

The docs on keyboardevents https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent
talks about IE's implementation being different and helpfully refers me to the msdn documentation to see more, which of course is gone. Can you hook me up with some docs on IE's events implementation?
I gotta say im not sold on redirecting everything before literally all the information that was there is ported. IE is notorious for its quirks and now the official source for information on those quirks is gone. Last couple weeks have been annoying. People ask me "well why doesnt this work on IE" and I have to look around for random ass blog posts from 2008 or whatever to figure it out since the actual docs are gone.

Here's an overview on the IE's legacy (deprecated) synthetic events implementation: https://msdn.microsoft.com/en-us/library/dn905219(v=vs.85).aspx
I looked up the MSDN doc linked from that MDN page (initKeyboardEvent), and it doesn't have much beyond what https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/initKeyboardEvent already has. The only difference is that IE's signature is:
KeyboardEvent.initKeyboardEvent(string eventType, bool canBubble, bool cancelable, string viewArg, string keyArg, string locationArg, string modifiersListArg, bool repeat, string locale);
. . . where Locale is the value of the "locale" attribute of the event. Also, here are the possible values for the modifiersListArg:
Alt
The left or right Alt key is pressed.
AltGraph
The Ctrl and Alt keys are pressed.
CapsLock
The Caps Lock toggle is enabled.
Control
The left or right Ctrl key is pressed.
Meta
The Meta/Control key is pressed.
NumLock
The Num Lock toggle is enabled.
Scroll
The Scroll Lock toggle is enabled.
Shift
The left or right Shift key is pressed.
Win
The left or right Windows logo key is pressed.

I'm in the process of copying all that to the MDN page, but stuck at the moment troubleshooting some MDN permission issues with the admins

Thanks - this is helpful!
Sent from my iPhone

On Nov 8, 2017, at 1:29 PM, Erika Doyle Navara<EMAIL_ADDRESS>wrote:
I'm in the process of copying all that to the MDN page, but stuck at the moment troubleshooting some MDN permission issues with the admins
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.


fix: Corrects links to spicy terms

Page was likely renamed from foundational terms to spice terms
Currently they are dead links

Do the maintainers need anything else for this pull request to be merged?


Add files via upload

www.ccc.edu

@dsandoz I am sorry, the domain ccc.edu is blacklisted because the school allows to register an email address there even if you are not a student of the school. If you are the student of this school, please apply with official document (OFFICIAL DOCUMENT tab at https://www.jetbrains.com/shop/eform/students).


Add admin area

The admin area has to contain these items:

List of telegram users.
Users list.

Since we've made a decision to not add an admin area to the public website.
So this issue is not relevant anymore.
Closing it.


Web-based player controls

[x] implement socket.io client to be used with JoshCunningHum/Dodong-webplayer
[x] add necessary events for sending/receiving/requesting player data
[x] multiple guild support
[ ] add server-sided authentication to verify user access for each server player (OAuth2?)
[ ] prevent spam/minimize traffic for both client and server

suggestions are welcome :)

https://github.com/nizeic/Dodong/pull/12


[WIP] Replace Dispatcher.Sort with Array.Sort

PR Details
Closes #1792 by removing Dispatcher.Sort and replacing all call sites with Array.Sort.
Just realized Dispatcher.Sort is public, so this is a breaking change, turning it into a draft for now.
Motivation and Context
Performance improvement.
Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ ] My change requires a change to the documentation.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

The previous implementation might need some love then.
You could embed the call to array sort within public call sites, and mention that this is temporary while an engineer investigates this performance issue further.

I don't think there's a problem with the implementation, it's just not the right algorithm, so I don't see it as temporary.
We could do better than Array.Sort, but ultimately this much per-frame sorting should not be necessary.
for example this is the first call site:
// Sort per render feature (used for later sorting)
// We'll be able to process data more efficiently for the next steps
Array.Sort(view.RenderObjects.Items, 0, view.RenderObjects.Count, RenderObjectFeatureComparer.Default);

So this sorts RenderObjects by (Root)RenderFeature.Index (for later sorting??), but this sorting is really just putting many RenderObjects into a few buckets. And then, each RootRenderFeature already has a list of associated RenderObjects, so the buckets already exist in some way, but they get mixed together at some point. I think investigating things like that might be worth it.
Getting back to Dispatcher.Sort - If we want to keep a parallel quicksort around, we should rename it and find a better place for it.


Revise install documentation around new install and deployment pattern

Change Type
[x] Addition
[x] Correction
[x] Deprecation
[x] Cleanup (formatting, typos, etc.)
Area
[x] Installation instructions
[x] Configuration parameters
[x] Administration/development

Proposed Changes
The key changes that we made in Nautobot for how it can be installed, configured, developed, administered, and deployed need to be revised. The initial fork and release only did the minimum required to make sure Nautobot could be installed. As indicated above, the following key documentation areas need to be revised:

Installation guide - spit into per-platform instructions, with a quick start guide for the common case)
Configuration guide - centralize all optional settings including permissions and authentication, into a single configuration section
Deployment guide - break install and production deployment into distinct sections. Getting Nautobot running is very different than running it in an enterprise-grade production environments
Development guide - How to use "invoke", the basics of the Docker container that comes w/ Nautobot and how to use that

Also need a generalized plugin installation and enablement guide, since it appears people are encountering issues in this area.

From ndom91 on Slack:

So there seems to be some kinda minor, kinda major issues i've come across / had others mention to me in using nautobot today.

There seems to be some indecision amongst the team whether to use /opt/nautobot/ as the primary config dir or ~/.nautobot and this shows itself at multiple stages while trying to install (i.e. copying static files to wrong dir, looking for config in wrong place, etc.)
Also all? plugins (can only confirm with chat-ops and nautobot-netbox-importer ) don't seem to allow nautobot to run while enabled. Screenshot to follow
nautobot-netbox-importer seems to ahve problems with IPAM prefixes and throws warnings about 'apparent duplicates', but during the later reconciliation stage (i.e. after "Data loading from Nautobot complete") it errors out because of these

diffsync.exceptions.ObjectAlreadyExists: Object <IP_ADDRESS>/31__None__{'name': 'abc'}__{'name': 'cba'} already present

See #74.

I noticed that in current documentaion there is no reference to  `local_requirements.txt``
see Installation section: ntc-nautobot-plugin-nornir

[...]
To ensure Nautobot Nornir Plugin is automatically re-installed during future upgrades, create a file named
local_requirements.txt (if not already existing) in the Nautobot root directory (alongside requirements.txt) and
list the nautobot_nornir package:
# echo nautobot-plugin-nornir>> local_requirements.txt
Once installed, the plugin needs to be enabled in your configuration.py
[...]


Automated cherry pick of #6531: fix: gcp bucket文件夹默认设为private

Cherry pick of #6531 on release/3.2.
#6531: fix: gcp bucket文件夹默认设为private

Codecov Report

Merging #6532 into release/3.2 will decrease coverage by 0.01%.
The diff coverage is 0.00%.

@@               Coverage Diff               @@
##           release/3.2    #6532      +/-   ##
===============================================
- Coverage         7.04%    7.02%   -0.02%
===============================================
  Files              649      650       +1
  Lines           111531   111575      +44
===============================================
- Hits              7858     7843      -15
- Misses          103065   103123      +58
- Partials           608      609       +1

Flag
Coverage Δ

#aFlag
7.02% <0.00%> (-0.02%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/compute/models/guest_queries.go
0.00% <0.00%> (ø)

pkg/compute/models/guests.go
0.56% <ø> (ø)

pkg/monitor/alerting/conditions/query.go
0.64% <0.00%> (ø)

pkg/monitor/alerting/conditions/reducer.go
95.87% <0.00%> (-2.02%)
:arrow_down:

.../monitor/alerting/conditions/suggestrulereducer.go
0.00% <0.00%> (ø)

pkg/multicloud/esxi/host.go
0.94% <0.00%> (-0.01%)
:arrow_down:

pkg/multicloud/esxi/storage.go
0.00% <0.00%> (ø)

pkg/multicloud/esxi/vdisk.go
0.00% <0.00%> (ø)

pkg/multicloud/esxi/virtualmachine.go
0.00% <0.00%> (ø)

pkg/cloudcommon/db/lockman/fifo.go
80.00% <0.00%> (-10.00%)
:arrow_down:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df8a942...95ded99. Read the comment docs.


Error with French locale

Hi,
there is an error in the way the French locale is configured. French does not use "point" but "virgule" (= comma in English) for decimal numbers.
That means, config_fr.properties should be modified like this:
point=virgule
Thanks for your library, very helpful !

Hello @PhilippeRo !
Thanks for your report. Please help me and w2ni18n with follow steps.

Fork the project (on this project side).
Checkout the project.
''' git clone https://github.com/PhilippeRo/w2ni18n.git '''
Change the config_fr.properties file with working change, you described replacing point=virgule
Best but optional add a line in the test file of your programming language.
Upload your change
''' git commit -m "correct French locale point name, see https://github.com/bastie/w2ni18n/issues/34" '''
''' git push '''
Create a Pull Request (on you project side https://github.com/PhilippeRo/w2ni18n.git) to include your change.

This is one good way to support our project.
with regards
Sebastian


NIFI-3031 - Multi-Statement Script support for PutHiveQL Processor

Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[ ] Is your initial contribution a single, squashed commit?

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Reviewing...

@dstreev I had been preparing to write this very enhancement to support Multi-Statement scripts for several weeks; thanks for taking care of it.
I didn't pay close attention to your changes for CSV, but the parts on multi-statement look good.  I will have to pull it down and give it a run.

@dstreev - multiple lines support was added for PutHiveQL only, not SelectHiveQL, right?


Rewrite trainign loop and forward of models

Currently the forward method of models such as double_flow implements the opt.step() and loss.backward() methods as well
I would like to move those steps to the training loop along with the writer
This has also the advantage of being able to call torch.no_grad() in the evaluation phase

I've settled on a compromise by setting the torch.no_grad() inside the forward method if it is called with val==True


spread strategy does not allow memory and / or CPU overcommit

Hi Folks,
first of all: this is not a bug report but a enhancement request / call for discussion.
Currently I'm testing swarm very intensive for our environment and have observed that memory overcommit is not possible then I use the default spread strategy.
This is because the method weighNodes (in the file scheduler/strategy/weighted_node.go) is called by the BinPack strategy (there it is OK) and unfortunately by spread, too.
In a environment there you reserve e.g. a max of 4 Gb for an Application Server container with a normal footprint of only 1 Gb (a typical scenario for Java App Server with different settings for Xms and Xmx) and want to start 10 of them on a docker host with 32Gb you will fail because the sum of the reserved memory (by the memory constraint) is larger then the free memory on the node. But in reality there is only something about 10 Gb used.
This is a different behavior compared to the default docker behavior there I can start any number of containers with memory constraints until the kernel complains about it and calls the oom-killer.
IMHO spread strategy should allow memory overcommit and maybe binpack, too (if applicable in a configurable way like in the kernel with /proc/sys/vm/overcommit_memory).

Thanks for the feedback. We've had many back and forth with this matter, feedback is very welcome.
A few things:

We actually already have a configurable overcommit.

It defaults to 5%: --cluster-opt swarm.overcommit=0.05

Overcommit in general:

The default is low for a reason - safety. For instance, in your example you have 10x4GB app servers (40GB total) on a 32GB node - but only 10GB are being actively used.
That's the timeless question of capacity planning: In this case it works fine, only 10 out of 32GB are used.
However, you are actually allowing the app servers to use up to 40GB.
If at some point they wake up and decide, hey, let's use all the memory we have, OOM is going to go on a rampage.
That's fine for app servers - not so much if you end up killing a single-instance database (e.g. MySQL). Coincidentally, the most common offenders that go from a relatively low usage up to their max are databases - they will run just fine at 1GB, then comes a weekly report query and they end up using their maximum amount of resources.

Overcommit implementation

Right now, overcommit is baked into the strategy and configurable through an option.
We could move it to a filter (such as resource filter) or something like that and make the strategies pick the best node without worrying about whether there are enough resources or not.
This means that you get to disable that safety net altogether (as in, infinite overcommit) if you like to do so.

Hi Andrea,
thank you for your detailed answer. I've found nothing in the docu and have missed it in the code.
I'll take a look into it again and may come back.

Issue is old, and the immediate question was answered... closing


Swift package < realm cocoapods

Hi, after cancellation of install realm by cocoapods, i tried use a realm via swift package.
Whilte using DB via package i noticed that data in my app not changing dynamically. It means that i should rebuild my ios app to see the latest data changes in RealmDB. So, anyone can explain me that?

There is no difference between Cocoapods and SPM Realms. It's only the way to deliver the same source code in the project. Yes, after you switch the project settings you need to rebuild the app.


apple/t2: kernel 6.12.2 -> 6.12.4

Description of changes
Bump the kernel version to fix regressions
Things done

[x] Tested the changes in your own NixOS Configuration
[x] Tested the changes end-to-end by using your fork of nixos-hardware and
importing it via <nixos-hardware> or Flake input

@mergify queue


Meltano Speedrun blog post does not configure required options for target-postgres

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/3544
Originally created by @WillDaSilva on 2022-05-28 20:48:17

The blog post Meltano Speedrun: From 0 to ELT in 90 Seconds has fallen out of date.
Both docs/getting-started.md and docs/src/_guide/installation.md link to it, it should be updated as necessary.
The issues arise from its instructions for configuring target-postgres:
meltano add loader target-postgres
meltano config target-postgres set postgres_username [username]
meltano config target-postgres set postgres_database speedrun

meltano elt tap-gitlab target-postgres

The required config options for target-postgres are currently user, password, and dbname, not postgres_username and postgres_database.

View 3 previous comments from the original issue on GitLab


[🎨] NT-1111 Adding 2 buttons in the new pledge footer

📲 What
Adding Continue and ProgressBar button in pledge screen footer.
🤔 Why
So I can use them in future PRs and not have conflicts in this file.
🛠 How
fragment_pledge_section_footer.xml

Added pledge_footer_continue_button button
Added fake pledge_footer_pledge_button_progress button

👀 See
Screenshots from XML previews in Android Studio

Continue
Progress
Pledge

📋 QA
These buttons are hidden, this PR is just so I can access them when I add functionality.
Story 📖
NT-1111

Yeah, I'm going to do a design sweep with Colleen after I'm all done. Ty, adding that to the list!


Can not display number of purchased units in the shopping cart when above 10 units

Solved! Thanks for your help :)


Mpeg1 Video bad quality

Dear ...
I'm so intersting with this project.
I tryed encode video from mp4 - h264 format to Mpeg1-TS by guide.
And my video output's quality is very bad.
plz helpme, the best way to encode video with better quality?
My file
source.zip
My output
out.zip
And my ffmpeg command:
ffmpeg -i source.mp4 -f mpegts -codec:v mpeg1video -codec:a mp2 -b 0 out.ts
Thank you so much!

There is a -q option to use fixed quality scale. And if I remember right 1 is the best quality, and 31 the worst.
ffmpeg -i source.mp4 -f mpegts -codec:v mpeg1video -codec:a mp2 -q 1 out.ts

@WebMaestroFr
Thank you so much!
We tryed your method, it's realy good.
Thank you!

@WebMaestroFr
Can you help me once again 👍 !!!!!

I download video from youtube with bellow infomation:
size: 44MB
resolution = 1920x1080 ,
bitrate = 125k,
audio sample rate = 44khz

Then i resize video to 640x360 by adobe video encoder. Output:
size: 86MB
resolution: 640x360
bitrate: 157k
audio: 48khz

Then in transcode 360p to MPG1-TS by command:
ffmpeg -i source.mp4 -f mpegts -codec:v mpeg1video -bf 0 -codec:a mp2 -q 1 out.ts

Ouput:
Size: 62MB
resolution: 640x360
bitrate: 384k
audio: 48khz
So i think the reason which output size 360P MPEG1-TS larger than  Youtube 1080P is bitrate. (It's only my think :) ).
Can you help me the way to reduce video size for MPEG1-TS output but video quality still good?
Thank you so much!

Can you help me the way to reduce video size for MPEG1-TS output but video quality still good?

No. the size of MPEG1 will be much larger for the same quality of other formats like h264.

You're setting -q 1 - which is the best possible quality. Try setting it to 4 or 8 instead. Or set a fixed bitrate for the video instead (eg.: -b:v 1m for 1mbit/s).
@zz85 is of course right: you can't get the same quality and resolution as H264 or other modern codecs at the same size with MPEG1.
Also, ffmpeg can resize video files, so there's no need to go through adobe video encoder or any other programs:
ffmpeg -i source.mp4 -s 640x360 -f mpegts -codec:v mpeg1video -bf 0 -codec:a mp2 -q 1 out.ts

There is a -q option to use fixed quality scale. And if I remember right 1 is the best quality, and 31 the worst.
ffmpeg -i source.mp4 -f mpegts -codec:v mpeg1video -bf 0 -codec:a mp2 -q 1 out.ts

Thanks, It's worked!

I just want to add that i am experiencing a better quality ( less visible blocking artifacts ) with a two pass encoding, for example :
ffmpeg -y -i test.webm -c:v mpeg1video -b:v 2500k -pass 1 -an -f null /dev/null &&
ffmpeg -i test.webm -c:v mpeg1video -b:v 2500k -pass 2 -c:a mp2 -b:a 128k twopass.ts


transition relation property to original_predicate property

This is for biolink compatibility (see https://github.com/biolink/biolink-model/issues/761)

tagging @amykglen so she will be aware of this change


gefyra run: confusing error message when container exits immediately

When the container exits immediately, the error message is not helpful. Additionally the info message still prints just as like it succeeded. For example when running gefyra run for the buzzword-counter without specifying a command:
~$ gefyra run -i buzzword-counter:dev -N buzzword-counter-dev -n default

[ERROR] Gateway patch could not be applied to 'buzzword-counter-dev': b'Cannot open network namespace "0": No such file or directory\nCannot open network namespace "0": No such file or directory\n'
[INFO] Container image 'k3d-buzzword-counter-registry/buzzword-counter:0.1.0, buzzword-counter:dev, k3d-buzzword-counter-registry:43287/buzzword-counter:0.1.0, k3d-buzzword-counter-registry:45319/buzzword-counter:0.1.0' started with name 'buzzword-counter-dev' in Kubernetes namespace 'default'

Running the command with the additional -c "bash -c 'tail -f /dev/null'" starts the container as expected.

It seems like this can also fail silently. I've used a hurricane command with -c flag, which seems to stop the container. Sometimes the error message appears, sometimes not. But the container never is running, which is not communicated by gefyra run.

Will you open up a PR to improve this behavior?


168-spike-check-a-newly-deployed-app-can-connect-to-its-paas-backing-service-timebox-1d

Make APIs use backing service checking healthcheck to ensure apps can connect to services
https://docs.google.com/document/d/18JV_ZTUqM500WLBxOt0lKtV12IYcS6Ay1SwJmxDOYb0
https://trello.com/c/ZxGiuNQb/1092-spike-check-a-newly-deployed-app-can-connect-to-its-paas-backing-service-timebox-1d

This change makes sense to me, I'd like to know though before merging whether it's going to impact the service much

I see problems when this is used against -api - it has the search api as one of its dependencies and its status endpoint considers it to be one too:
https://github.com/alphagov/digitalmarketplace-api/blob/fa6866bed73793679af8a3d9fdac9f5639f53801/app/status/views.py#L25
Passing the search_api_client to get_app_status causes it to consider it as a dependency. This would mean that we wouldn't be able to deploy a -api if the -search-api was down. Also if -search-api went down, all our -apis would get killed because their healthchecks would fail.

@risicle Good call
What about something like this?
https://github.com/alphagov/digitalmarketplace-antivirus-api/pull/59
https://github.com/alphagov/digitalmarketplace-utils/pull/550

Yeah I think I added "additional checks" for that reason. Wanted to consider clamd an "internal" dependency.

@katstevens @lfdebrux @risicle
Thanks for the comments all, closing this in favour of a new approach changing what is called by ignore-dependencies by changing the underlying get_app_status in
alphagov/digitalmarketplace-antivirus-api#59
alphagov/digitalmarketplace-utils#550
This will only affect the healthchecks (pingdom cannot hit apis). It means that additional_checks will always be called by the healthcheck request but defines a new arg called additional_checks_extended which will not run if ignore-dependencies is passed. See CHANGELOG.md in utils for more info.


How to write code using NEArithmeticAdditionKernel especially on broadcasting add.

Output of 'strings libarm_compute.so | grep arm_compute_version':
arm_compute_version=v19.05 Build options: {'arch': 'arm64-v8a', 'opencl': '1', 'neon': '1', 'examples': '1', 'asserts': '0', 'debug': '1', 'openmp': '1', 'os': 'linux', 'Werror': '0'} Git hash=4ba87dbdc3b22220eba4a792c1f5c87e7a88c7af
Platform:
HTG_Z920 board
Operating System:
Ubuntu 18.04

Problem description:
I'm using ACL for ONNX Runtime, with the corresponding ACLExecutionProvider.
While writing Add operator kernel for ACL, I encountered a problem with broadcast add.

Add node: Add_26
Input 0 Name: 233 Shape: {1, 128, 1}
Input 1 Name: 234 Shape: {}
I'm using the kernel as following:
  const Tensor* A = ctx->Input<Tensor>(0);
  const Tensor* B = ctx->Input<Tensor>(1);

  const T* a_data = A->template Data<T>();
  const T* b_data = B->template Data<T>();

  arm_compute::Tensor a {}, b {}, y {};

  std::vector<int64_t> Y_dims;
  auto &a_shape = A->Shape().GetDims();
  Y_dims = std::vector<int64_t>(a_shape.begin(), a_shape.end());
  Tensor* Y = ctx->Output(0, TensorShape(Y_dims));
  T* y_data = Y->template MutableData<T>();

  a.allocator()->init(arm_compute::TensorInfo(ACLTensorShape(A->Shape()), arm_compute::Format::F32));
  b.allocator()->init(arm_compute::TensorInfo(ACLTensorShape(B->Shape()), arm_compute::Format::F32));
  y.allocator()->init(arm_compute::TensorInfo(ACLTensorShape(Y->Shape()), arm_compute::Format::F32));

  a.allocator()->import_memory((void*) a_data);
  b.allocator()->import_memory((void*) b_data);
  y.allocator()->import_memory((void*) y_data);

  arm_compute::NEArithmeticAddition add_core;

  add_core.configure(&a, &b, &y, arm_compute::ConvertPolicy::WRAP);
  add_core.run();

The error comes from NEArithmeticKernel.cpp:483 @ validate_arguments
const TensorShape out_shape = TensorShape::broadcast_shape(input1.tensor_shape(), input2.tensor_shape());
I'm not sure why the output shows as "out_shape.total_size() == 0"
input1 has a dim of 3, and input2 has a dim of 1.
Can anyone help me fix the code?

Hi @JAEWOOKe
Can you please upgrade to the latest acl release v21.02 and let us know if you still see the problem?

Thanks for responding. I think the problem was that input2 is a scalar, which was meant to be a tensor with at least one dimension.


Write tests for handle methods

In index.js, upon receiving a protobuf encoded RPC message, we process each kind of control message in handleIHave, handleIWant, handleGraft and handlePrune. We need to test these methods.

[ ] Test for handleIHave

[ ] Test for handleIWant

[ ] Test for handleGraft

[ ] Test for handlePrune

closing..


Add support for HTTP encoding server responses.

When enabled, the server will look for the Accept-Encoding header from a client and respect it if set to gzip or deflate.
Encoding doesn't happen if

The user is publishing a response with Content-Encoding set. It's presumed the user has encoded the content themselves and we won't reencode.
Not part of accepted media types
The response has a small fixed content-length (e.g., not using chunked / framed encoding). For larger ones, we switch to chunked encoding and do the compression for a hopefully larger return.

@trustin PTAL

Current coverage is 64.39% (diff: 69.64%)

Merging #228 into master will increase coverage by 0.37%

@@             master       #228   diff @@
==========================================
  Files           276        277     +1
  Lines         10610      10663    +53
  Methods           0          0
  Messages          0          0
  Branches       1537       1551    +14
==========================================
+ Hits           6792       6866    +74
+ Misses         3077       3050    -27
- Partials        741        747     +6

Powered by Codecov. Last update 430610b...0e7d580

Could this ever be done as a decorator?

Tried thinking about a decorator, for the thread-safety and making sure the objects are encoded in the same order as they are written out, I wanted the encoding to happen after the object is released from QueueBasedPublisher's queue, which seems only possible in a subscriber (HttpResponseSubscriber) which isn't decoratable.
I could try making yet another queue, but the flow starts to blow my mind a bit.

You can create a new DefaultHttpResponse, subscribe to the original HttpResponse and forward the published HttpObjects to the new DefaultHttpResponse while compressing them. It may not be as efficient as yours because it involves an extra queue. You may want to write a delegating HttpResponse implementation that's more suitable for this sort of job.

What type of delegating HttpResponse would be more suitable? I do need the compression to happen after queuing to ensure order + thread-safety as compression is stateful. The current PR sticks with the current queue, while another option would be to have two queues, I don't think it's possible with the current implementation in a better way.
What do you think about adding the ability to register an object transformer to a QueueBasedPublisher? Right after queue.remove, it would call the transformer before sending to the subscriber.

How about something like this? https://gist.github.com/trustin/9c6b30bedbc376d673a793dce933761a

Thanks for the FilteredSubscriber suggestion, rewrote this as a decorator.

Looks good all in all. Please address the comments.

Thanks - updated

After reading again, I realized I have a few more things to say.

Thanks - updated

Thanks! :+1:


Add Mode for CRS formulas

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 96.86%. Comparing base (fbe96da) to head (dcb7b7d).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #160   +/-   ##
=======================================
  Coverage   96.86%   96.86%
=======================================
  Files          26       26
  Lines        1115     1116    +1
=======================================
+ Hits         1080     1081    +1
  Misses         35       35

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[ART-1122] make set_client_latest multi-version aware

https://jira.coreos.com/browse/ART-1122
I have a few assumptions here: (base on its only 1 point card, otherwise, it will be much more complicated than this)

set_client_latest job was invoked automatically by release/pre-release job so there won't be any version degraded, each RELEASE version should be bigger than the previous one, eg:  4.1.14 -> 4.1.15 -> 4.1.16

this is only for ocp folder OC_MIRROR_DIR not for other client like odo, crc... and not for ocp-dev-preview, cuz that one involves sth string like 4.3.0-0.nightly-2019-11-07-010532

@jupierce updated, PTAL

@shiywang I tried to make this multi-arch aware too in https://github.com/openshift/aos-cd-jobs/pull/2031 . It includes your commits.
The logic was complex enough I thought it needed to run on the upload-mirror system itself.

I think this PR is effectively rewritten by https://github.com/openshift/aos-cd-jobs/pull/2031


Feature request for std.asscii => function isNewline

bioinfornatics (@bioinfornatics) reported this on 2012-11-18T16:48:50Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=9045
CC List

araspik
bearophile_hugs
bugzilla (@WalterBright)
bus_dbugzilla
dlang-bugzilla (@CyberShadow)
dmitry.olsh (@DmitryOlshansky)
i6o34a+7y7j1p606trkc
issues.dlang (@jmdavis)
monarchdodra

Description
hi,

std.ascii provide some function to check ascii char, to get newline char from your OS but not a function to said if a char is a newline as does this function:

bool isNewline(dchar c) @safe pure nothrow {
    return ( c == 0x0A || c == 0x0D )? true : false;
}

is Possible to add it ?

regards

bearophile_hugs commented on 2012-11-18T16:59:06Z
See representation on various systems:

http://en.wikipedia.org/wiki/Newline

In particular:
On Unix, and Mac OS X: LF (1 char)
On Windows: CR+LF (2 chars)

issues.dlang (@jmdavis) commented on 2012-11-18T17:15:06Z
Just check whether it equals std.ascii.newline.

bioinfornatics commented on 2012-11-18T17:08:08Z
(In reply to comment #1)
> See representation on various systems:
>
> http://en.wikipedia.org/wiki/Newline
>
> In particular:
> On Unix, and Mac OS X: LF (1 char)
> On Windows: CR+LF (2 chars)

yes not easy or into string

bool isNewline(in dchar[] c) @safe pure nothrow {
    bool result = false;
    if( c[0] == 0x0A)
        result = true;
    else if( c.length >= 2 && c[0] == 0x0D && c[1] == 0x0A)
        result = true;
    ...
    ...
    return result;
}

monarchdodra commented on 2012-11-20T05:42:25Z
(In reply to comment #1)
> See representation on various systems:
>
> http://en.wikipedia.org/wiki/Newline
>
> In particular:
> On Unix, and Mac OS X: LF (1 char)
> On Windows: CR+LF (2 chars)

(In reply to comment #5)
> Technically speaking, if you don't know which type of line endings a file uses
>
> [SNIP]

Isn't the "line ending" a *file* totally irrelevant here? In the sense that it is a nothing more than the system's *storage* format?

On my windows machine, the *strings* I manipulate don't have "\r\n" as a newline, they have '\n'. That's the entire reason there is a "rb" and "r" option when reading a file.

If you *do* have an "\r\n" in your stream, then either:
* You have an actual a '\r' in your stream, which is then followed by a new line.
* You are actually erroneously manipulating a binary payload, which should be of type ubyte[], and should not be using the std.ascii functions with it.

Under these circumstance, and following the unicode definition, I'd say:

return 0x0A <= c && c <= 0x0D;

Is not only correct (for ascii), but any attempt to parse more than 1 character for this info would be incorrect...

PS: WTF is \u{D A}

dmitry.olsh (@DmitryOlshansky) commented on 2016-03-22T13:08:46Z
(In reply to Nick Sabalausky from comment #10)
> While an 'isNewline(dchar)' func wouldn't work for reasons already
> discussed, what *would* work and be very helpful IMO, is something similar
> to the 'std.conv.parse(...)' functions. Ie, just like this, but properly
> templated, range-ified and UTF8/16-ified:

skipNewLine taking string by ref that would return a bool indicating if there was a new line would indeed be useful.

bus_dbugzilla commented on 2013-03-26T10:04:56Z
While an 'isNewline(dchar)' func wouldn't work for reasons already discussed, what *would* work and be very helpful IMO, is something similar to the 'std.conv.parse(...)' functions. Ie, just like this, but properly templated, range-ified and UTF8/16-ified:

/// If 'str' starts with a Unix, Windows, Unicode or Mac9 newline, it is
/// removed from 'str' and returned. Otherwise, null is returned.
dstring parseNewline(ref dstring str)
{
    if(str.empty)
        return null;

    dstring ret;

    // Newlines are as defined in:
    // http://www.unicode.org/reports/tr18/#Line_Boundaries
    switch(str[0])
    {
    case '\r':
        if(str.length > 1 && str[1] == '\n')
        {
            ret = str[0..2];
            str = str[2..$];
            break;
        }
        goto case;

    case '\n':
    case '\f':
    case '\v':
    case '\x85':
    case paraSep:
    case lineSep:
        ret = str[0..1];
        str = str[1..$];
        break;

    default:
        ret = null;
        break;
    }

    return ret;
}

issues.dlang (@jmdavis) commented on 2012-11-20T12:29:09Z
Any code that only cares about \n or \r\n isn't going to work with a function  which returns true for both. And any code that doesn't care is going to be ill-served by a such a function, because the reality is that you need to watch for \r and \n as individual characters and properly handle the cases when they're separate as well as when they're apart. And the fact that one of them is multiple characters generally screws with trying to check for them with a single function in the middle of iterating. Rather, you have to watch for \r and \n individually and then figure out whether you're dealing with them singly or together and act appropriately. A function which returns true for a string is generally useless for the kind of situations where you'd be checking for newlines, so I'm highly skeptical that such a function is of any real value. Instead, you end up with stuff like

for(auto range = str.save; !range.empty; range.popFront())
{
    switch(range.front)
    {
        case '\r':
            auto temp = range.save;
            temp.popFront();
            if(temp.front == '\n')
            {
                range.popFront();
                goto case '\n';
            }
            goto default;
        case '\n':
            //do whatever you do for end of line
            break;
        default:
            //do whatever you do for individual characters
            break;
    }
}

And if you all you want to know is whether a particular string starts or ends with a newline, then it's easy enough to just do str.startsWith("\n", "\r\n") or str.endsWith("\n", "\r\n"). That gets uglier when you need to deal with unicode rather than just \n an \r\n, but then all I believe that you really need is to add [paraSep], and [lineSep] to the list.

I'm not sure that a function telling you whether a string designates the end of a line is completely useless, but in pretty much every case that I can see code caring, such a function wouldn't work very well.

bus_dbugzilla commented on 2013-03-28T17:35:23Z
*** Issue 8880 has been marked as a duplicate of this issue. ***

dmitry.olsh (@DmitryOlshansky) commented on 2012-11-19T10:49:39Z
Somewhat related.

Unicode new line sequence is defined as:
\u{A} | \u{B} | \u{C} | \u{D} | \u{85} | \u{2028} | \u{2029} | \u{D A}

Note that sequence '\r\n' counts as one line end.

See the note here for example:
http://www.unicode.org/reports/tr18/#Line_Boundaries

dmitry.olsh (@DmitryOlshansky) commented on 2012-11-20T12:13:48Z
(In reply to comment #7)
> (In reply to comment #1)
> > See representation on various systems:
> >
> > http://en.wikipedia.org/wiki/Newline
> >
> > In particular:
> > On Unix, and Mac OS X: LF (1 char)
> > On Windows: CR+LF (2 chars)
>
> (In reply to comment #5)
> > Technically speaking, if you don't know which type of line endings a file uses
> >
> > [SNIP]
>
> Isn't the "line ending" a *file* totally irrelevant here? In the sense that it
> is a nothing more than the system's *storage* format?
>

There is no system's encoding. It died and buried in the same toomb as FTP ASCII mode long time ago. After all files are transfered in many different ways expecting someone to transcode line-endings everywhere is plain impossible (you don't always know the target system). So by the end of day reasonable programs just deal with all the zoo of them.

> On my windows machine, the *strings* I manipulate don't have "\r\n" as a
> newline, they have '\n'. That's the entire reason there is a "rb" and "r"
> option when reading a file.

And I'd say rb option is a woefully broken thing. In fact putting \n does in fact store \r\n in this mode. You are far safer with binary mode at least it's WYSIWG.

> If you *do* have an "\r\n" in your stream, then either:
> * You have an actual a '\r' in your stream, which is then followed by a new
> line.

> Under these circumstance, and following the unicode definition, I'd say:
>
> return 0x0A <= c && c <= 0x0D;
>
> Is not only correct (for ascii), but any attempt to parse more than 1 character
> for this info would be incorrect...

No, no and no. It's the fact of life (or rather the standard) that \r\n is a single entity. And it can't be parsed other the by looking at two characters (or rather codepoints).

>
> PS: WTF is \u{D A}

It's 2 dchars : \r\n. That is 0x0D and 0x0A. They are being cute and use flexible width syntax not the old ones: \uXXXX and \UYYYYYYYY.

issues.dlang (@jmdavis) commented on 2012-11-18T23:28:15Z
Technically speaking, if you don't know which type of line endings a file uses, you can't possibly correctly determine when you've reached the end of a line. Best case, you have to assume that '\r\n' and '\n' both designate the end of the line, whereas it's perfectly legal to have a '\r' be the last character on a line (i.e. the one before the characters indicating the end of the line) in Linux, and it's perfectly valid to have '\n' be in the middle of line on Windows. So, parsing with the assumption that both '\r\n' and '\n' indicate the end of the line is actually incorrect no matter what OS you're on.

That doesn't mean that it's not entirely unreasonable to have such a function, but it does mean that it can't possibly be 100% correct.

On an unrelated note, I'd point out that

return ( c == 0x0A || c == 0x0D )? true : false;

is redundant. The ternary operator is completely unnecessary. It would be better if it were

return c == 0x0A || c == 0x0D;

dmitry.olsh (@DmitryOlshansky) commented on 2016-03-22T13:08:31Z
(In reply to Nick Sabalausky from comment #10)
> While an 'isNewline(dchar)' func wouldn't work for reasons already
> discussed, what *would* work and be very helpful IMO, is something similar
> to the 'std.conv.parse(...)' functions. Ie, just like this, but properly
> templated, range-ified and UTF8/16-ified:

skipNewLine taking string by ref that would return a bool indicating if there was a new line would indeed be useful.

bioinfornatics commented on 2012-11-18T23:19:27Z
(In reply to comment #3)
> Just check whether it equals std.ascii.newline.

yes but the problem is when you need to write a parser and detect end of line. You can get the given file from various operating system so you can"t use std.ascii.newline as is system specific.

bugzilla (@WalterBright) commented on 2019-11-20T07:30:23Z
Where has this been moved to?

araspik commented on 2018-10-22T20:45:10Z
Bump: When is this going to get implemented into Phobos? It makes my life much easier, mainly because 'newline' is a 'string' but since I'm using 'dstring' I require the usage of 'assumeWontThrow' everywhere. Example:

dstring str = "..."d;
str.findSkip(newline); // not nothrow
str.findSkip(newline).assumeWontThrow; // Ugh, long
str.findSkip!isNewline; // What would be nice _AND_ nothrow

dlang-bugzilla (@CyberShadow) commented on 2019-11-20T14:31:25Z
Undoing vandalism(?) by Parmigiano


Detect and return raw json data

This PR makes it so that result values that are valid json strings are retuned as json instead of stringified json. For now this will be the only supported behavior. Once we get to improving our query params ("flags/modes", etc), we can support a flag to specify the caller wants stringified json.
It works by Scanning all db row values into a special type UserValue that can identify strings that are already valid json and provide custom json Marshal() that simply retuned the valid json when appropriate.

@asutula looks like your proposal has been sufficiently consented to. Should we take this out of draft?

Hey @jsign give this another look over in the morning. I reverted to using UserValue and also realized that UserRows.Rows can now be of type [][]*UserValue which had a nice effect bubble up and eliminate places where we were asserting types from interface{}.


Restore many features in Firefox

Fixes #5988.
The Chrome stacktrace is starting with an additional line Error: Get stack. Firefox already starts with the first stack entry in the first line.
Test URLs

This PR (see that reaction-avatars is working again)
https://github.com/refined-github/refined-github (see that toggle-files-button is working again)
Basically every other page

I'll release a new version tonight probably. In the meantime I'm leaving this PR open in case anyone wants to code golf it 😁

Sounds fine, thanks! By the way, could you apply the hacktoberfest-accepted label to this PR please? And maybe also add the hacktoberfest topic to the repo (for future PRs)? See also https://togithub.com/refined-github/refined-github/issues/4871#issuecomment-937470330.


Kosaraju_Algo

Programming language

[ ] C
[x] C++
[ ] Java
[ ] Python

@Ayush7614 @Amit366  please assign this issue


feat: Add 'runner scale sets' to actions-runner-controller

This one is a bit of an awkward update.
I'm not convinced this is the best way so happy to change it
The actions-runner-controller has been adopted by GitHub officially, along with a new 'mode'.
Which is actually a completely independent set of CRDs and controllers, but its all in the same repo.
It's not clear if there will be any significant further development on the 'legacy' mode either.
See the release announcement here: https://github.com/actions/actions-runner-controller/discussions/2775
I've removed all but the last 2 of the 'legacy' versions and just added the new CRDs as 0.4.
We could make this a new lib, but as always, naming is the hardest bit.
Given the new 'mode' is still called the Actions Runner Controller i'm not sure what else to call it and renaming the existing lib to 'actions-runner-controller-legacy' or something will break existing uses of the lib.

👍 Can we merge this then?

It's in, sorry for the delay @hamishforbes !

It's in, sorry for the delay @hamishforbes !

Thanks! I was on holiday anyway ;)


fix clang format tool not install

What this PR does / why we need it:
fix clang format tool not install

/cc @nlgwcy

/lgtm


Versions after 1.0.68 make MultiSelect unusable

Expected Behavior
When updating from react-multi-select 1.0.68 to 1.0.69, there should not be any big changes

Actual Behavior
The MultiSelect component does not render correctly and overlays buttons under it (on the y-axis)

Steps to Reproduce the Problem
Other people seem to not have this problem, so here are the package.json dependencies
    "@kenshooui/react-multi-select": "1.0.69",
    "axios": "^0.17.1",
    "body-parser": "^1.18.3",
    "express": "^4.16.3",
    "moment": "latest",
    "njwt": "^1.0.0",
    "react": "^16.4.0",
    "react-copy-to-clipboard": "^5.0.1",
    "react-datepicker": "^0.61.0",
    "react-dom": "^16.1.1",
    "react-modal": "^3.8.1",
    "react-numeric-input": "^2.2.3",
    "react-scripts": "^3.0.1",
    "react-select": "^1.1.0",
    "react-table": "^6.7.4"

Specifications

Version: 1.0.69
Platform: Linux
Subsystem: Fedora

Hi! Sorry but after version 1.0.68 you need import<EMAIL_ADDRESS>for supporting native style multi-column-select (it was specified in readme  earlier).


Fixed with-iron-session to work on Windows systems

Issue
Currently $ npm run dev fails on Windows systems for the with-iron-session example. That is due to setting a variable on an npm script. To make it work for Windows you would have to alter the script which would probably fail on other operating systems.
Reproducing
Run $ npm run dev on a Windows machine
Fix
There's many solutions to this, but since this is an example I went with the most simple one, which is using the cross-env package.

Stats from current PR

Default Server Mode (Increase detected ⚠)

General

zeit/next.js canary
awareness481/next.js with-iron-session
Change

buildDuration
11.2s
11s
-184ms

nodeModulesSize
66.4 MB
66.4 MB
✓

Page Load Tests Overall increase ✓

zeit/next.js canary
awareness481/next.js with-iron-session
Change

/ failed reqs
0
0
✓

/ total time (seconds)
1.937
1.897
-0.04

/ avg req/sec
1290.8
1317.78
⚠ +26.98

/error-in-render failed reqs
0
0
✓

/error-in-render total time (seconds)
1.162
1.097
-0.06

/error-in-render avg req/sec
2150.72
2278.75
⚠ +128.03

Client Bundles (main, webpack, commons)

zeit/next.js canary
awareness481/next.js with-iron-session
Change

main-HASH.js gzip
6.59 kB
6.59 kB
✓

webpack-HASH.js gzip
746 B
746 B
✓

de003c3a9d30..1e23.js gzip
10.5 kB
10.5 kB
✓

framework.HASH.js gzip
39.1 kB
39.1 kB
✓

Overall change
57 kB
57 kB
✓

Client Bundles (main, webpack, commons) Modern

zeit/next.js canary
awareness481/next.js with-iron-session
Change

main-HASH.module.js gzip
5.67 kB
5.67 kB
✓

webpack-HASH..dule.js gzip
746 B
746 B
✓

de003c3a9d30..dule.js gzip
6.92 kB
6.92 kB
✓

framework.HA..dule.js gzip
39.1 kB
39.1 kB
✓

Overall change
52.5 kB
52.5 kB
✓

Legacy Client Bundles (polyfills)

zeit/next.js canary
awareness481/next.js with-iron-session
Change

polyfills-HASH.js gzip
26.3 kB
26.3 kB
✓

Overall change
26.3 kB
26.3 kB
✓

Client Pages

zeit/next.js canary
awareness481/next.js with-iron-session
Change

_app.js gzip
1.26 kB
1.26 kB
✓

_error.js gzip
3.25 kB
3.25 kB
✓

hooks.js gzip
881 B
881 B
✓

index.js gzip
222 B
222 B
✓

link.js gzip
2.06 kB
2.06 kB
✓

routerDirect.js gzip
279 B
279 B
✓

withRouter.js gzip
278 B
278 B
✓

Overall change
8.22 kB
8.22 kB
✓

Client Pages Modern

zeit/next.js canary
awareness481/next.js with-iron-session
Change

_app.module.js gzip
604 B
604 B
✓

_error.module.js gzip
2.04 kB
2.04 kB
✓

hooks.module.js gzip
383 B
383 B
✓

index.module.js gzip
223 B
223 B
✓

link.module.js gzip
1.52 kB
1.52 kB
✓

routerDirect..dule.js gzip
281 B
281 B
✓

withRouter.m..dule.js gzip
278 B
278 B
✓

Overall change
5.33 kB
5.33 kB
✓

Client Build Manifests

zeit/next.js canary
awareness481/next.js with-iron-session
Change

_buildManifest.js gzip
61 B
61 B
✓

_buildManife..dule.js gzip
61 B
61 B
✓

Overall change
122 B
122 B
✓

Rendered Page Sizes

zeit/next.js canary
awareness481/next.js with-iron-session
Change

index.html gzip
927 B
927 B
✓

link.html gzip
937 B
937 B
✓

withRouter.html gzip
924 B
924 B
✓

Overall change
2.79 kB
2.79 kB
✓

Serverless Mode

General

zeit/next.js canary
awareness481/next.js with-iron-session
Change

buildDuration
12s
12.1s
⚠ +111ms

nodeModulesSize
66.4 MB
66.4 MB
✓

Client Bundles (main, webpack, commons)

zeit/next.js canary
awareness481/next.js with-iron-session
Change

main-HASH.js gzip
6.59 kB
6.59 kB
✓

webpack-HASH.js gzip
746 B
746 B
✓

de003c3a9d30..1e23.js gzip
10.5 kB
10.5 kB
✓

framework.HASH.js gzip
39.1 kB
39.1 kB
✓

Overall change
57 kB
57 kB
✓

Client Bundles (main, webpack, commons) Modern

zeit/next.js canary
awareness481/next.js with-iron-session
Change

main-HASH.module.js gzip
5.67 kB
5.67 kB
✓

webpack-HASH..dule.js gzip
746 B
746 B
✓

de003c3a9d30..dule.js gzip
6.92 kB
6.92 kB
✓

framework.HA..dule.js gzip
39.1 kB
39.1 kB
✓

Overall change
52.5 kB
52.5 kB
✓

Legacy Client Bundles (polyfills)

zeit/next.js canary
awareness481/next.js with-iron-session
Change

polyfills-HASH.js gzip
26.3 kB
26.3 kB
✓

Overall change
26.3 kB
26.3 kB
✓

Client Pages

zeit/next.js canary
awareness481/next.js with-iron-session
Change

_app.js gzip
1.26 kB
1.26 kB
✓

_error.js gzip
3.25 kB
3.25 kB
✓

hooks.js gzip
881 B
881 B
✓

index.js gzip
222 B
222 B
✓

link.js gzip
2.06 kB
2.06 kB
✓

routerDirect.js gzip
279 B
279 B
✓

withRouter.js gzip
278 B
278 B
✓

Overall change
8.22 kB
8.22 kB
✓

Client Pages Modern

zeit/next.js canary
awareness481/next.js with-iron-session
Change

_app.module.js gzip
604 B
604 B
✓

_error.module.js gzip
2.04 kB
2.04 kB
✓

hooks.module.js gzip
383 B
383 B
✓

index.module.js gzip
223 B
223 B
✓

link.module.js gzip
1.52 kB
1.52 kB
✓

routerDirect..dule.js gzip
281 B
281 B
✓

withRouter.m..dule.js gzip
278 B
278 B
✓

Overall change
5.33 kB
5.33 kB
✓

Client Build Manifests

zeit/next.js canary
awareness481/next.js with-iron-session
Change

_buildManifest.js gzip
61 B
61 B
✓

_buildManife..dule.js gzip
61 B
61 B
✓

Overall change
122 B
122 B
✓

Serverless bundles

zeit/next.js canary
awareness481/next.js with-iron-session
Change

_error.js
844 kB
844 kB
✓

404.html
4.18 kB
4.18 kB
✓

hooks.html
3.81 kB
3.81 kB
✓

index.js
844 kB
844 kB
✓

link.js
882 kB
882 kB
✓

routerDirect.js
874 kB
874 kB
✓

withRouter.js
874 kB
874 kB
✓

Overall change
4.33 MB
4.33 MB
✓

I think you can go even simpler by making use of the new 9.4 Environment Variable feature. Removing the need for a package such as cross-env.


Memory leak with DB insert multiple records

Laravel Version: 8.15.0
PHP Version: 7.4.10
Database Driver & Version: MySql 5.7.24

Description:
Using DB::insert() with multiple records by passing an array of arrays (as per: https://laravel.com/docs/8.x/queries#inserts) seems to leak memory.
Steps To Reproduce:
    public function runBatch()
    {
        $tempTable = 'csv_test_batch';

        Schema::create($tempTable, function (Blueprint $table) {
            $table->string('field_1');
            $table->string('field_2');
            $table->string('field_3');
            $table->string('field_4');
            $table->string('field_5');
        });

        $recordCount = 0;
        while ($recordCount < 50000) {
            $chunkSize = 5000;
            $data = [];
            while ($chunkSize-- > 0) {
                $data[] = [
                    'field_1' => 'Some data 1',
                    'field_2' => 'Some data 2',
                    'field_3' => 'Some data 3',
                    'field_4' => 'Some data 4',
                    'field_5' => 'Some data 5',
                ];
                $recordCount++;
            }
            DB::table($tempTable)->insert($data);
            info("Batch Insert: {$recordCount} records, memory: ". number_format(memory_get_usage()));
        }
    }

Running this shows the increasing memory usage:
[2020-11-22 07:44:11] local.INFO: Batch Insert: 5000 records, memory: 30,458,104
[2020-11-22 07:44:12] local.INFO: Batch Insert: 10000 records, memory: 31,826,240
[2020-11-22 07:44:12] local.INFO: Batch Insert: 15000 records, memory: 32,965,304
[2020-11-22 07:44:12] local.INFO: Batch Insert: 20000 records, memory: 34,104,368
[2020-11-22 07:44:12] local.INFO: Batch Insert: 25000 records, memory: 35,243,432
[2020-11-22 07:44:12] local.INFO: Batch Insert: 30000 records, memory: 36,382,496
[2020-11-22 07:44:12] local.INFO: Batch Insert: 35000 records, memory: 37,521,560
[2020-11-22 07:44:12] local.INFO: Batch Insert: 40000 records, memory: 38,660,944
[2020-11-22 07:44:12] local.INFO: Batch Insert: 45000 records, memory: 39,800,008
[2020-11-22 07:44:12] local.INFO: Batch Insert: 50000 records, memory: 40,939,392

Running the same set of queries as single inserts (while much slower) does not seem to cause a memory leak:
    public static function runSingle()
    {
        $tempTable = 'csv_test_single';

        Schema::create($tempTable, function (Blueprint $table) {
            $table->string('field_1');
            $table->string('field_2');
            $table->string('field_3');
            $table->string('field_4');
            $table->string('field_5');
        });

        $recordCount = 0;
        while ($recordCount < 50000) {
            $chunkSize = 5000;
            while ($chunkSize-- > 0) {
                $data = [
                    'field_1' => 'Some data 1',
                    'field_2' => 'Some data 2',
                    'field_3' => 'Some data 3',
                    'field_4' => 'Some data 4',
                    'field_5' => 'Some data 5',
                ];
                $recordCount++;
                DB::table($tempTable)->insert($data);
            }
            info("Single Insert: {$recordCount} records, memory: ". number_format(memory_get_usage()));
        }
    }

The result is:
[2020-11-22 07:44:39] local.INFO: Single Insert: 5000 records, memory: 29,503,992
[2020-11-22 07:44:53] local.INFO: Single Insert: 10000 records, memory: 29,504,184
[2020-11-22 07:45:07] local.INFO: Single Insert: 15000 records, memory: 29,504,376
[2020-11-22 07:45:22] local.INFO: Single Insert: 20000 records, memory: 29,504,568
[2020-11-22 07:45:36] local.INFO: Single Insert: 25000 records, memory: 29,504,760
[2020-11-22 07:45:52] local.INFO: Single Insert: 30000 records, memory: 29,504,952
[2020-11-22 07:46:06] local.INFO: Single Insert: 35000 records, memory: 29,505,144
[2020-11-22 07:46:21] local.INFO: Single Insert: 40000 records, memory: 29,505,976
[2020-11-22 07:46:35] local.INFO: Single Insert: 45000 records, memory: 29,506,168
[2020-11-22 07:46:49] local.INFO: Single Insert: 50000 records, memory: 29,506,360

I've tried adding DB::disableQueryLog() just in case something was enabling it, but that does not seem to make a difference.

You might refer to #35095

This issue was happening on a new clean Laravel 8.15.0 install.
Thanks for the heads-up #crynobone,  As per https://github.com/laravel/framework/issues/35095#issuecomment-722398532 I removed facade/ignition and the memory leak has stopped.
I've tried the same thing on my actual code base and I can confirm that removing Ignition has stopped the leak there also. So this seems to be an issue with Ignition when using inserting arrays of arrays. Strange, why would Ignition care if we performed 50K single inserts vs 10 inserts of 5K records...
I'll close here and open an issue on facade/ignition, thanks!

I've submitted a PR for this on the ignition repo.


Disable ESC when level complete screen shows up

After the timer runs out and the level complete screen shows up you can press ESC and save and quit to menu. When resuming, the game softlocks.

While UI2 is planned and the level complete dialog box is planned to be removed once UI2 resumes development, will be keeping this issue open as that would technically resolve this issue.


Rewrite PCL agent

I rewrote the PCL agent to avoid memory issues when saving Variables inside list / replay buffer. I didn't compare the training curve with the old one, but it seems to learn (the average_value increases and R gets bigger) on Catpole under the new parameters and there is no memory issue when run with large network / reasonably long trajectories.
Main methods are the following:
update: take a loss (as an array), log the result as usual and call optimizer (the backprop is done before this function is called)
update_on_policy and update_from_replay: sample a list of trajectories (from replay or the current one), clear grads and compute loss
compute_loss: take a list of trajectories, perform batch computation (batch size is the number of episodes, which may not be efficient when there is one single episode for on-policy update). This function will call backward immediately and only return an array for logging
_compute_path_consistency: compute path consistency, this part of code is almost unchanged
The new underlying data structure is a list of dict to store the current episode, then a replay buffer that only stores (s,a,r) pairs. The old mu (action_distrib) is removed since it can be recomputed again from other items.
I also added a unified model in the example script and changed a couple of parameters.
Issues addressed: #109 #236 #240
I am not sure if the parameters are used correctly, but if they are correct, this PR also addresses #238

Thank you for the improvements on PCL. I haven't checked the implementation details yet, but I think solving the memory issue is great as long as it won't make training slow.
Can you show the training curves and computation speeds before and after this PR?


Please consider two additional methods for UserManager for claims

I can easily get a enumerated list of claims for a user from the UserManager but when writing a user interface for claims CRUD, I find that the following two methods would be very useful.
Consider adding Bool UserManager.HasClaimAsync()
Consider adding IdentityUserClaim UserManager.GetClaimAsync()  //see below extension
public static async Task<IdentityUserClaim> GetClaimAsync(this UserManager<ApplicationUser> usermanager, int claimId, ApplicationDbContext applicationDbContext)
   {
       IdentityUserClaim<string> IdentityUserClaimString
           = await applicationDbContext.UserClaims.SingleAsync(c => c.Id == claimId) ;

       IdentityUserClaim identityUserClaim = new IdentityUserClaim();

       identityUserClaim.Id = IdentityUserClaimString.Id;
       identityUserClaim.ClaimType = IdentityUserClaimString.ClaimType;
       identityUserClaim.ClaimValue = IdentityUserClaimString.ClaimValue;
       identityUserClaim.UserId = IdentityUserClaimString.UserId;

       return (identityUserClaim);
   }

Why are you trying to get the store specific POCO out?  The data inside of it is basically the same as a standard Claim (Type/Value).
wrt to HasClaim, if you are looking for claims on the logged in user, you can simply do something like:
Context.User.HasClaim(c => c.Type == "whatever" && c.Vaue == "value");
If you want it against an arbitrary user, you can still use LINQ .Where against the IEnumerable of claims from GetUserClaims()

Dropping the request for UserManager.HasClaimAsync()
On GetClaimAsync()  POCO or not, for my purposes, i.e. in this instance I am managing claims for users other than the logged on user; building a UI to manage the claims another user has within the application.  The properties identityUserClaim.Id and identityUserClaim.UserId are both important.  System.Security.Claims.Claim does not have Id or UserId properties if that is what you refer to.

wrt GetClaimAsync, you can still just display a view listing all of a user's claims (GetUserClaims(TUser)).  Once the user has decided which claims to add/remove, you can call Add/RemoveUserClaim() with the claim, and the store will remove/add the appropriate instances.  You still don't need to know about the child entity, that's an implementation detail for the store.  You don't need to drop down to that level, and you probably shouldn't unless you really need to

@HaoK ,
I am unsure why I don't need identityUserClaim.Id and identityUserClaim.UserId of a claim.  identityUserClaim.Id is after all, the primary key, identityUserClaim.UserId is the foreign key.  When building MVC for managing a specific claim such as delete, I use the primary key to know which claim to delete.  That follows the pattern for other identity objects, 'users' for example.
In my App, security is important, and will have a relatively small users base.  The security manager needs to be able to quickly review all assigned claims for all users without having to select each user to see what claims are assigned.  That way the security manager can quickly determine no one has claims they should not.  So I plan to give the security manager user a view enumerating all the claims in the app, again identityUserClaim.Id and identityUserClaim.UserId of each claim will be very important for the edit and delete actions or if the user wants to navigate to the details for the related user.
My app uses windows Kerberos authentication, context.User is the IPrincipal (Windows Identity) for the current logged in user. In my case Context.User doesn't have any (Identity Framework) claims.
Context.User has the IPrincipal assigned from the windows Kerberos authentication consequently Context.User object does not contain the claims assigned to the matching Microsoft.AspNet.Identity.IdentityUser that are stored using Entity Framework.
Thus:
•Context.User.HasClaim() can not determine authorizations
•[Authorize("Administrator", "Allow")] attribute on a controller action method to determine authorization at the controller cannot function.
What can I do to get Context.User.HasClaim() and [Authorize("Administrator", "Allow")] to function correctly when using windows Kerberos authentication?

@blowdart

@WillBaldwin
It looks like you're attempting to use AD for auth and then pull in additional claims from an identity database? Is that correct?
This isn't a scenario we've supported for quite a while now. It was fraught with problems and very few, if any implementations got it correct. IdentityUser is meant for forms based authentication, or a smattering of social auth and nothing more.


New release and follow-up tasks

We're close to a new release, and it makes sense to discuss a few thing before that happens:

We now have v2.6.0 and the current master holds code that has removed any support for Python 2.7. I would guess that qualifies for bumping the version number to v3.0.0 (rather than v2.7.0). – Any other opinions?

For what it's worth to think about after the release:

The Piwik module has a deprecation notice for some time. After the new release it probably would be a good move to remove that module from the code base, finally.
We now have 3 Google Analytics implementations (legacy, gtag, js), and we should probably start to deprecate at least the legacy option. Maybe we can consider consolidating all solutions into a single Google Analytics module, as discussed in #28.

The Piwik module was removed via PR #213. :bomb:


refactor: encapsulate user config checks

About this change—what it does

replaces old converter in update for service integration
does not panic if given type does not have a user config schema

tests


Preserve stopping_strategy of shuffled interleaved dataset (random cycling case)

Preserve the stopping_strategy in the RandomlyCyclingMultiSourcesExamplesIterable.shard_data_sources to fix shuffling a dataset interleaved (from multiple sources) with probabilities.
Fix #5812

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Logs are printed in console, but not getting pushed to Splunk Server

Logs are printed correctly in console, but not getting pushed to Splunk Server.
And not even printing any error.
Below is my log4j2.xml configuration file
<?xml version="1.0" encoding="UTF-8"?> <Configuration status="info" name="example" packages="org.example">     <Appenders>         <Console name="console" target="SYSTEM_OUT">             <PatternLayout                     pattern="%style{%d{ISO8601}} %highlight{%-5level }[%style{%t}{bright,blue}] %style{%C{10}}{bright,yellow}: %msg%n%throwable" />         </Console>         <File name="MyFile" fileName="logs/app.log">             <PatternLayout>                 <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>             </PatternLayout>         </File>         <SplunkHttp name="httpconf"                     url="http://localhost:8088"                     token="b489e167-d96d-46ec-922f-6b25fc83f199"                     host="localhost"                     index="spring_dev"                     source="source name"                     sourcetype="log4j"                     messageFormat="text"                     disableCertificateValidation="true">             <PatternLayout pattern="%m" />         </SplunkHttp>     </Appenders>
<Loggers>
    <Root level="info">
        <AppenderRef ref="console" />
        <AppenderRef ref="MyFile"/>
        <AppenderRef ref="httpconf" />
    </Root>
</Loggers>

I'm seeing the same problem. Logs print to console and written to file but nothing in Splunk. I fired up tcpdump and can see packets going to the HEC endpoint. The heavy forwarder shows no errors and nothing in the index. Also verified HEC is working with curl.
Using log4j v2.33.1 and splunk javalogging 1.11.8.


ACM import

Part of #334

[x] aws_acm_list
[x] aws_acm_import
[ ] aws_acm_request

Left a couple of comments. Unfortunately I think this one is gonna be held back for the next release.