Non-blocking Socket and Early Data

OpenSSL version: 1.1.1a
Configuration data
OS Name, Version, Hardware platform: Debian Stretch on x64
Application Details: nginx 1.15.6

Here is a excerpt from nginx log with additional debug messages:
2018/12/04 13:07:13 [debug] 18272#0: *124 SSL ALPN supported by client: h2
2018/12/04 13:07:13 [debug] 18272#0: *124 SSL ALPN supported by client: http/1.1
2018/12/04 13:07:13 [debug] 18272#0: *124 SSL ALPN selected: h2
2018/12/04 13:07:13 [debug] 18272#0: *124 SSL_read_early_data: 1, 1
2018/12/04 13:07:13 [debug] 18272#0: *124 SSL_get_current_cipher()
2018/12/04 13:07:13 [debug] 18272#0: *124 SSL: TLSv1.3, cipher: "TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD"
2018/12/04 13:07:13 [debug] 18272#0: *124 SSL_session_reused()
2018/12/04 13:07:13 [debug] 18272#0: *124 SSL reused session
[...]
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_read_early_data: 1, 529
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_read_early_data: 1, 72
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_read_early_data: 1, 61
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_read_early_data: 1, 68
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_read_early_data: 1, 63
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_read_early_data: 0, 0
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_get_error: 2
[...]
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL to write: 344
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_write_early_data: 1, 344
[...]
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL to write: 16384
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_write_early_data: 1, 16384
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL to write: 16384
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_write_early_data: 0, 0
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_get_error: 3
[...]
2018/12/04 13:07:13 [debug] 18272#0: *125 ssl new session: 915F34E7:32:157
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_read_early_data: 2, 0
2018/12/04 13:07:13 [alert] 18272#0: *125 ignoring stale global SSL error (at 6) (SSL: error:1409E10F:SSL routines:ssl3_write_bytes:bad length:len:257, rlayer.wnum:0, wb->left:2882, rlayer.wpend_tot:16384, early_data_state:12
) while processing HTTP/2 connection
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_read: -1
2018/12/04 13:07:13 [debug] 18272#0: *125 SSL_get_error: 5

The numbers after the SSL_read_early_data is a return value and readbytes. The numbers after the SSL_write_early_data is a return value and written bytes.
My Interpretation
Call to the SSL_write_early_data returns 0 with SSL error SSL_ERROR_WANT_WRITE and so we wait until an underlying event library notifies us that the fd is ready for writing to retry the write. In the meantime a new SSL session is created and ssl3_handshake_write function is called. It calls ssl3_write_bytes, where the following check does not pass, because of shorter length
    /*
     * ensure that if we end up with a smaller value of data to write out
     * than the original len from a write which didn't complete for
     * non-blocking I/O and also somehow ended up avoiding the check for
     * this in ssl3_write_pending/SSL_R_BAD_WRITE_RETRY as it must never be
     * possible to end up with (len-tot) as a large number that will then
     * promptly send beyond the end of the users buffer ... so we trap and
     * report the error in a way the user will notice
     */
    if ((len < s->rlayer.wnum)
        || ((wb->left != 0) && (len < (s->rlayer.wnum + s->rlayer.wpend_tot)))) {
        SSLfatal(s, SSL_AD_INTERNAL_ERROR, SSL_F_SSL3_WRITE_BYTES,
                 SSL_R_BAD_LENGTH);
    }

Than the end of early data record arrives which in the nginx code triggers calling ERR_peek_error and then SSL_read which fails.

I'm thinking that this not should even be a valid thing to try and do, i.e. call SSL_read_early_data() where there is an outstanding unsuccessful SSL_write_early_data() pending.
There is this statement on the SSL_get_error() page which disallows this for SSL_read()/SSL_write():
It is safe to call SSL_read() or SSL_read_ex() when more data is available
even when the call that set this error was an SSL_write() or SSL_write_ex().
However if the call was an SSL_write() or SSL_write_ex(), it should be called
again to continue sending the application data.

Since SSL_read_early_data()/SSL_write_early_data() basically call SSL_read()/SSL_write() internally that statement should probably be extended to cover the early data forms.
So, probably this is a documentation issue in OpenSSL and nginx needs to be fixed to not do this.

Being able to read and write in parallel is quite important to an HTTP/2 implementation. It certainly used to work before for SSL_read/SSL_write.
0-RTT makes this a little bit tricky because of read-triggered handshake data, but I think the TLS stack should make that work. This was a case we made sure to think through when designing our 0-RTT API and implementation. On the server, it's possible to completely avoid this scenario with half-RTT tickets. On the client, the scenario is unavoidable but possible to abstract away.

It certainly used to work before for SSL_read/SSL_write.

AFAIK this has never worked for SSL_read/SSL_write. The restriction was always there - we just documented it. Most of the time it works anyway but there are cases (e.g. with reneg) where it doesn't.

(HTTP/2 implementations are supposed to disable renegotiation. Though it's true most don't since OpenSSL didn't provide an API for it until recently.)
I believe renegotiation would just cause SSL_read and SSL_write to both drive the handshake, which works mostly fine, modulo the error idempotence issue (the mess surrounding CVE-2017-3737). But even then, that "mostly working", renegotiation being uncommon, and concurrent read and write being a very common application need, I do not think it's reasonable restriction for OpenSSL to now impose. Indeed this interaction with 0-RTT and renegotiation was one of the main reasons we decided to fully support error idempotence on our end by latching the errors. Go's crypto/tls similarly latches errors.
But I don't think that's even what the current text forbids. It says:

It is safe to call SSL_read() or SSL_read_ex() when more data is available even when the call that set this error was an SSL_write() or SSL_write_ex().

This says that SSL_read is okay to call when SSL_write wants data.

However if the call was an SSL_write() or SSL_write_ex(), it should be called again to continue sending the application data."

This is a little confusing because "the call" could refer to a couple things. It reads to me that, if SSL_write blocked on write, one must repeat the same SSL_write call again to continue. That is true, I don't think that's what's going on here. I suspect the internal check for the second sentence is breaking the first sentence. (We reworked writes for handshake data to account for this in 0-RTT.)

Are you able to reproduce this error - or was this a one off?

Yes, I am able to reproduce it with 100%-rate on certain website in Firefox, if early data and HTTP/2 is turned on. The result is that no stylesheets or javascript are loaded.

This says that SSL_read is okay to call when SSL_write wants data.

Correct

It reads to me that, if SSL_write blocked on write, one must repeat the same SSL_write call again to continue.

Also correct - that's what it means.

i.e. if SSL_write is blocked on write you can't call SSL_read. You must call SSL_write again.

Sorry, my text was even more confusing than the original text. :-)

i.e. if SSL_write is blocked on write you can't call SSL_read. You must call SSL_write again.

The text only says that it should be called again "to continue sending the application data". It doesn't say you cannot call other stuff. If the intent was indeed to block out everything, I think that was solidly a mistake. (E.g. NGINX does it a lot.) At the very least, you all should enforce it more consistently: make SSL_read fail if there is pending write data, rather than only flakily failing when hitting a read-triggered write. If you do that, I think you will very quickly find that lots of clients break this assumption.
It also contradicts the first sentence, unless you read "this error" to mean SSL_ERROR_WANT_READ. But I don't think the text supports that. If anything, it's in the SSL_ERROR_WANT_WRITE (In general, this text uses "it", "this", and "the call" too much. It's very hard to understand.)

i.e. if SSL_write is blocked on write you can't call SSL_read. You must call SSL_write again.

The text was supposed to make clear that you can, but clearly
the text is not clear enough.
If SSL_write() fails, and you call it again at some point later,
the length should be the same. But you can call SSL_read() between
those SSL_write() calls.
If that's not the case, I think it's a bug in OpenSSL.

Ok. Lets take a step back and describe the problem as I see it with interleaving SSL_read()/SSL_write(). We can then look to see if there is a sane solution.
SSL_write is supposed to be an atomic operation (assuming you don't set the non default option SSL_MODE_ENABLE_PARTIAL_WRITE), i.e. it either completely succeeds with everything written (positive return value) or it completely fails (SSL_ERROR_WANT_WRITE or similar) with nothing written (at least nothing is reported as written to the user application). Technically there is also the option of a fatal error - in which case we are in an indeterminate middle state. In practice of course the underlying network is not atomic so what it actually does is split the data up into records and writes each one in turn. If anything fails (SSL_ERROR_WANT_WRITE) control is passed back to the application which must call SSL_write again with exactly the same data. As far as the user application is concerned nothing has yet been wrriten - but in practice it may have done. The user application has to keep calling SSL_write() until everything has been successfully written.
Consider the scenario of a large write which needs to be broken up into 3 records. Lets say the first record gets written to the network successfully. During processing the 2nd record libssl creates the record, encrypts it and puts it in its buffer ready to write out. The writing fails (SSL_ERROR_WANT_WRITE). At this point we have some data already written to the network, some data already encrypted and buffered but not yet written, and some data still to process. Control is passed back to the user application.
Now lets say a record arrives for reading. Instead of continuing with the in progress write operation the application calls SSL_read. The read packet is a reneg ClientHello so a new handshake is initiated and the SSL_read call will actually attempt to write out a ServerHello. At this point we have a problem because the write buffers are already full and contain an encrypted application data record ready to write out.
In this scenario what is the correct behaviour for SSL_read? Possible options:

Ditch the buffered write data. This would work in that we would then be able to proceed with writing out the ServerHello. However as far as the application is concerned it is still part way through an ongoing SSL_write operation which it will then attempt to continue - and libssl would have to re-encrypt the second record and carry on from where it left off. But we've had a reneg occur since we started the write operation - so that means that the first record we wrote happened before the reneg, and subsequent records will be written after the reneg. This sounds like a really "bad idea". It would be totally unexpected, and (potentially) dangerous.

Flush the buffered write data. Again this would work but suffers from the same flaw as (1) above. In this case we would have written the first 2 records before the reneg and the 3rd record would be after the reneg. Again this sounds like a bad idea.

Write all the pending application data before writing the ServerHello. However we've only buffered the second record - so in order for this option to work we would have to suck in the entire SSL_write data that the application presented to us when it first called SSL_write and buffer it somewhere. This could potentially be very large and means lots of copying data around. We probably don't want to do this.

SSL_read could fail and signal that the user application needs to complete the SSL_write operation first. But we don't have an error code for this. All the SSL_ERROR_WANT_* codes mean "call SSL_read again", not "stop calling SSL_read and instead call SSL_write". We could introduce a new one or redefine SSL_ERROR_WANT_WRITE to mean "call SSL_read again unless there is a pending SSL_write operation which also returned SSL_ERROR_WANT_WRITE in which case call that" - but this option doesn't help existing applications and would require those applications to change to be able to cope with the new code or redefined code.

Where I thought we were at with this is documenting that you shouldn't call SSL_read if there is an ongoing SSL_write operation. If we don't consider that viable then I think the only sane option is (4) - but that will still require applications that are running into this problem to make a change.

SSL_read() should return SSL_ERROR_WANT_WRITE in that case. It's
already documented that SSL_read() can return
SSL_ERROR_WANT_WRITE.
If SSL_read() returns SSL_ERROR_WANT_WRITE, and there is an SSL_write()
that didn't send all data, calling SSL_read() should continue to return
SSL_ERROR_WANT_WRITE as long as the data from the SSL_write and the
renegiotation after it is not written.
That means that we should not try to write internally until at
least all data from the SSL_write() has been encrypted and is
at least queued.

Note that after SSL_write() is done with the user data, it should
pick up were SSL_read() stopped, otherwise applications that use
select() will most likely get stuck since they have no reason to
try and read.

I'm not sure that'll work for the same reason that (3) won't work. SSL_write does not have a way to return the output to the caller.

If we internally send something, we have an internal buffer to do
so, and that's used again if it failed.

@kroeckx @mattcaswell Is this still an issue with currently supported releases? I assume it is.


Console text input wraps to same line beginning when column limit is exceeded

Currently, in git repositories the text input given to terminal which supposed to go to a new line goes straight to the same line's beginning, overriding the previous text (display-wise).
It is caused by "bash-git-completion" extension, and it seems to disappear when theme is changed to default instead of "Crunch".

Not related to Hadouken CMake script. Fixed in :Hadouken container version 1.0.6.


Running llama-stack with 8B llama on an AWS CPU only instance throwing an error

I'm trying to run llama stack on a cpu-only node to try it out, getting the error below, can't find a way to tell it I don't have one:
File "/home/ubuntu/anaconda3/envs/llamastack-my-local-stack/lib/python3.10/site-package
s/torch/distributed/distributed_c10d.py", line 1594, in _new_process_group_helper
backend_class = ProcessGroupNCCL(
ValueError: ProcessGroupNCCL is only supported with GPUs, no GPUs found!

Typically running models without GPUs will be very slow. Partners like ollama are able to offer a way to run with cpus. You can try llama stack with ollama to run on a cpu-only node. See https://github.com/meta-llama/llama-stack/blob/main/llama_stack/distribution/templates/local-ollama-build.yaml

What is the recommended minimum system spec to successfully run llama stack's default model in your quick start? I've been trying different configurations without much success.


Reduce chances of hitting AWS ECS Rate Limits

AWS will rate limit task definition registration for a given task family. There are two factors that make it easy for a relatively low volume of Prefect flows to hit rate limits:

All Prefect Flows are registered under the same task family "prefect" defined here. This is relevant because the error raise is RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family.
Task definitions create a new log stream for each task created here, which in turn means that each flow run has a new task definition.

Solution:
I feel like creating task families per deployment will be the easiest way around this, as this reduces the number of registrations for a given family. However this does not fully resolve the issue. It is nice to see a new log stream for every flow run, however the need to register task definitions introduces a chance for rate limits to be hit.

I do not think infrastructure runs have access to deployment names either at this time. We may be able to add that but it’s not a small change.
I do not think using our CloudWatch logging setup is compatible with using your own task definition without reregistration, I'm not sure there is a way around this.

ok, I tested this out a bit more and it looks like when user provides the task_definition_arn in this format (with family name but without revision), everything works without hitting the ECS API limits: "arn:aws:ecs:us-east-1:338306982838:task-definition/prefect"
my block:
ecs = ECSTask(
    aws_credentials=aws_creds,
    cluster="prefect",
    task_definition_arn="arn:aws:ecs:us-east-1:111111111111111:task-definition/prefect",
)
ecs.save(DEFAULT_BLOCK, overwrite=True)

I triggered a bunch of runs (more than 12 which would otherwise raise the limit) and everything worked fine

and even though I didn't set any custom CloudWatch logging here, all logs are still in the Prefect backend

@madkinsz LMK if we can close the issue for now and advise users running things at scale how they can register task definition themselves and to use it without explicit revision (using the latest active revision): "arn:aws:ecs:us-east-1:338306982838:task-definition/prefect"

I think there is a separate issue here on this, at least one that I saw reported by a user in community, but receiving this error specifically : An error occurred (ClientException) when calling the RegisterTaskDefinition: Too many concurrent attempts to create a new revision of the specified family.
This error was returned in AWS Cloud Watch logs, and it was determined this issue was included for prefect v1 - https://github.com/PrefectHQ/prefect/pull/5059/files , but does not exist for the agent in v2.
This was hot-fixed by the customer in line 949 here:
https://github.com/PrefectHQ/prefect-aws/blob/main/prefect_aws/ecs.py#L949
task_definition.setdefault("family", f"prefect-{self.labels['prefect.io/flow-run-id']}")
task_definition['family'] = f"{task_definition['family']}-{self.labels['prefect.io/flow-run-id']}"

I'm not sure if it's exactly the same core issue, but the error reported is the same, and seems to have resolved it for this customer.
The core is that it's missing the flow run id?

Too many concurrent attempts to create a new revision of the specified family.

I do not want to use a UUID for the family as we'll generate a bunch of families that way. Ideally, it'd be the deployment name. I'll see if I can find a clean way to get the deployment name to the infrastructure.

Too many concurrent attempts to create a new revision of the specified family.

I do not want to use a UUID for the family as we'll generate a bunch of families that way. Ideally, it'd be the deployment name. I'll see if I can find a clean way to get the deployment name to the infrastructure.

what is the downside of too many families? I believe this is how it worked in v1, and we didn't have issues. I'm open to other solutions, just curious.

Further context in the community here.
I am indeed experiencing this issue.
We have started our migration from 1.0 -> 2.0 but this issue is going to have to make us stop that until this is rectified as we are unable to run our production environment at the scale required.
I would also like to understand the downside in multiple families ? This worked for us in prefect 1.0

there are not as many downsides, it's just a little weird because a task definition is supposed to work equivalent to a Docker image - a template from which you run many instances. In the same way as you wouldn't build a Docker image at runtime every time you run a flow, you usually wouldn't want a task definition per run UUID. We can potentially do a bit with caching here or setting the name to flow_name/deployment_name if it's possible to retrieve that on the infrastructure block - Michael mentioned that he'll investigate whether that's possible.
Workaround: For now, I would recommend incorporating building a task definition as part of your CI/CD, updating that task definition ARN on your ECSTask block also from CI, and this way the AWS rate limit issue should be gone.

Thanks @anna-geller .
With your proposed workaround will the infra overrides still work at run time?
It seems a bit careless of me to have to have a work around during a migration and then be forced to fix it down the line. But if it's the only option I guess I'd prefer to delay the migration until a valid solution is in place.

This is a valid solution. In fact, it's how AWS actually wants us to use this service, that's why this rate limiting exists. Which infra overrides do you have in mind? I would recommend setting all infrastructure related attributes as part of your CI e.g. you build an image in CI, you push it to ECR, you build a task definition and tell Prefect on the block which one to use (Prefect will always use the latest) by specifying an explicit task definition ARN

I think what we discuss here is more a convenience enhancement to avoid task definition registration through CI and doing this at flow runtime instead. This shouldn't block migration though. If you have issues figuring out CI, I can point you to resources that can help

This is what I currently do with my CI.

I build an image and push it to ECR
I build the ECSTask block like so:

def build_and_save_ecs_task(repo_name):
    default_params = json.loads(Ssm.get_secret("/prefect/ecs_task_config"))
    account_number = Sts.get_caller_identity()

    ecs = ECSTask(
        name=repo_name,
        image=f"{account_number}.dkr.ecr.ap-southeast-2.amazonaws.com/prefect-{repo_name}:latest",
        cpu=512,
        memory=256,
        **default_params,
    )
    ecs.save(name=repo_name, overwrite=True)

I deploy the flows and deployments from a deployments.yml file that I maintain with the info that is required.

eg
flows:
 - flow_name: betfair_transactions
   flow_path: flows/betfair/betfair_transactions.py
   deployments:
     - name: strategy
       parameters: {"account_identifier": "strategy", "record_count": 5000}
       schedule: {"cron": "0 * * * *", "timezone": "Australia/Brisbane"}
       infra_overrides: {"cpu": 1024, "memory": 8192}
     - name: automated
       parameters: {"account_identifier": "automated", "record_count": 5000}
       schedule: {"cron": "0 * * * *", "timezone": "Australia/Brisbane"}

This yaml file is used to template what I need in the prefect deployments build and prefect deployments apply commands.
That make sense?
Given that I can still configure my memory and cpu, I see no issues with your proposal.
Are you just suggesting instead of supplying all the configurations to the EcsRun I make my own task definition and then just provide this arn?

thx for sharing, step 2 seems unnecessary since you are using latest image and no changes are made to the ECSTask block
you would need a different ECSTask definition for different cpu/memory setting
yes exactly, you would register the task definition from your CI and all you need on your block is to set the task_definition_arn to "arn:aws:ecs:us-east-1:338306982838:task-definition/prefect" and Prefect will take the latest active revision automatically

so both ECSTask block creation and ECSTask registration would actually need to happen only once if you are using the latest ECR image
to make it easier, perhaps you can have 2 ecs task blocks - one configured with minimal cpu/memory and another one more generous? this way, it's trivial to switch when some deployment turns out to need more memory e.g. switch from -ib ecs-task/default to -ib ecs-task/high-memory

Thanks @anna-geller .
Ah yeah that is not an ideal solution for our use case at all then.
We have many specific and different memory/CPU combinations to be used and the entire reason that we'd be against using this option is because it will end up with many many blocks for ecs. Seems like overkill tbh.
Step 2 is more there just for IAC and reproducibility in the repo.

I think for now the hot fix shown in the thread above is probably the best solution for our needs.


Add note about global.json

Fixes https://github.com/dotnet/sdk/issues/23261.
Preview.

Thanks for the reviews!


Update plugin opsgenie yml

@daniel-beck
Description
OpsGenie Plugin GitHub

Submitter checklist for changing permissions

Always

[x] Add link to plugin/component Git repository in description above

For a newly hosted plugin only

[x] Add link to resolved HOSTING issue in description above

For a new permissions file only

[x] Make sure the file is created in permissions/ directory
[x] artifactId (pom.xml) is used for name (permissions YAML file).
[x] groupId / artifactId (pom.xml) are correctly represented in path (permissions YAML file)
[x] Check that the file is named plugin-${artifactId}.yml for plugins

When adding new uploaders (this includes newly created permissions files)

[x] Make sure to @mention an existing maintainer to confirm the permissions request, if applicable
[x] Use the Jenkins community (LDAP) account name in the YAML file, not the GitHub account name
[x] All newly added users have logged in to Artifactory at least once

Given https://github.com/jenkinsci/opsgenie-plugin/blob/master/pom.xml#L11 this change makes no sense. Please explain.

Given group Id deleted. Updated pom :  https://github.com/jenkinsci/opsgenie-plugin/commit/e775fc3af8e8586c1832832ee6879c34ab6e8c3c


ORC-1487: Enable checkstyle on src/test with checkstyle-suppressions.xml

What changes were proposed in this pull request?
This PR aims to enable checkstyle on src/test directories to validate the recent PRs like #1590 .
Why are the changes needed?
To help the recent community work, we permanently declare the following rules as exception in test source code.
     <suppress checks="LineLength" files="src/test/*"/>
     <suppress checks="NewlineAtEndOfFile" files="src/test/*"/>

     <suppress checks="UnusedImports" files="src/test/*"/>
     <suppress checks="AvoidStarImport" files="src/test/*"/>
     <suppress checks="CustomImportOrder" files="src/test/*"/>

The other suppressed rules will be removed after we finish the test code clean-ups.
How was this patch tested?
Pass the CIs.

cc @mystic-lama

@dongjoon-hyun Awesome. I looked at adding this but was not sure if it was good idea. Thank you so much for this.
Do we want to make Line length as change as part of this PR?

Thank you, @mystic-lama . During reviewing your on-going PR, I evaluated our remaining work in the existing code and decided to ignore LineLength and NewlineAtEndOfFile and three Import-related completely in 2.0.0. It will reduce the number of changed line in your PR and future PRs on benchmark module.

Thanks a ton @dongjoon-hyun
We can address them later

I backported this test infra PR to branch-1.9 and adjusted the JIRA issue versions.


When cache is enabled in v2, pipeline returns ResourceNotFoundError

When cache is enabled using DSP v2 manifests (the default value), the pipeline step pod returns ResourceNotFound error. When cache is disabled, the message is suppressed. However, even with cache disabled, we get errors that the PipelineRun cannot be found.
Possible cause is that some endpoints are "hard-coded". They have default values but always consider that kfpv2 is deployed on kubeflow namespace. There might be other hard-coded locations in the code that prevents the pipeline to run successfuly.

It is confirmed that data-science-pipelines-operator#369 fixes that issue. Closing issue.


type: "Python"

pytorch to caffe appears layer：type: "Python"
But caffe read this layer wrong， The error message is shown below：
Unknown layer type: Python (known types: AbsVal, Accuracy, ArgMax, BNLL, BatchNorm, BatchReindex, Bias, Clip, Concat, ContrastiveLoss, Convolution, Crop, Data, Deconvolution, Dropout, DummyData, ELU, Eltwise, Embed, EuclideanLoss, Exp, Filter, Flatten, HDF5Data, HDF5Output, HingeLoss, Im2col, ImageData, InfogainLoss, InnerProduct, Input, LRN, LSTM, LSTMUnit, Log, MVN, MemoryData, MultinomialLogisticLoss, PReLU, Parameter, Pooling, Power, RNN, ReLU, Reduction, Reshape, SPP, Scale, Sigmoid, SigmoidCrossEntropyLoss, Silence, Slice, Softmax, SoftmaxWithLoss, Split, Swish, TanH, Threshold, Tile, WindowData)
That's all error , thanks for your help!

There is no python layer will be export,can you show your model code so that I can check out what is wrong?


improve bulk delete on sampleids

Currently if you have a large dataset and you delete it, the db will lock and the system will be down for a few minutes (which is a lot of time). One option is to do delayed delays other is simply change "visibility" (update is fast). Some more ideas:

Final solution would be a combination of both, visible and not visible, delay deletes a row at a time
Doing the combined suggestion above would be better for users
Doing a row per row deletion within a single web job but with multiple transactions will be implemented and we will see how it works in the test environment

This was solved in patch 55: https://github.com/biocore/qiita/blob/master/qiita_db/support_files/patches/python_patches/55.py


HBASE-28270 Remove xml-maven-plugin on branch-3

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 36s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ branch-3 Compile Tests _

+1 :green_heart:
mvninstall
2m 49s
branch-3 passed

+1 :green_heart:
compile
4m 57s
branch-3 passed

+1 :green_heart:
spotless
0m 42s
branch has no errors when running spotless:check.

_ Patch Compile Tests _

-1 :x:
mvninstall
2m 43s
root in the patch failed.

+1 :green_heart:
compile
4m 49s
the patch passed

+1 :green_heart:
javac
4m 49s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
xml
0m  1s
The patch has no ill-formed XML file.

-1 :x:
hadoopcheck
4m 36s
The patch causes 10 errors with Hadoop v3.2.4.

-1 :x:
hadoopcheck
9m 28s
The patch causes 10 errors with Hadoop v3.3.6.

+1 :green_heart:
spotless
0m 41s
patch has no errors when running spotless:check.

_ Other Tests _

+1 :green_heart:
asflicense
0m 14s
The patch does not generate ASF License warnings.

28m  4s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5587

Optional Tests
dupname asflicense javac hadoopcheck spotless xml compile

uname
Linux 6059983b3d41 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-3 / b7952fcc10

Default Java
Eclipse Adoptium-11.0.17+8

mvninstall
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt

hadoopcheck
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/1/artifact/yetus-general-check/output/patch-javac-3.2.4.txt

hadoopcheck
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/1/artifact/yetus-general-check/output/patch-javac-3.3.6.txt

Max. process+thread count
79 (vs. ulimit of 30000)

modules
C: . U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/1/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 31s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ branch-3 Compile Tests _

+0 :ok:
mvndep
0m 17s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
2m 51s
branch-3 passed

+1 :green_heart:
compile
5m 49s
branch-3 passed

+1 :green_heart:
spotless
0m 56s
branch has no errors when running spotless:check.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 15s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 29s
the patch passed

+1 :green_heart:
compile
6m  6s
the patch passed

+1 :green_heart:
javac
6m  6s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
xml
0m  3s
The patch has no ill-formed XML file.

+1 :green_heart:
hadoopcheck
13m 27s
Patch does not cause any errors with Hadoop 3.2.4 3.3.6.

+1 :green_heart:
spotless
0m 50s
patch has no errors when running spotless:check.

_ Other Tests _

+1 :green_heart:
asflicense
0m 19s
The patch does not generate ASF License warnings.

41m 44s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5587

Optional Tests
dupname asflicense javac hadoopcheck spotless xml compile

uname
Linux 6e2740525fe4 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-3 / b7952fcc10

Default Java
Eclipse Adoptium-11.0.17+8

Max. process+thread count
73 (vs. ulimit of 30000)

modules
C: hbase-archetypes/hbase-archetype-builder . U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/2/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 31s
Docker mode activated.

-0 :warning:
yetus
0m  2s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-3 Compile Tests _

+0 :ok:
mvndep
0m 18s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
2m 50s
branch-3 passed

+1 :green_heart:
compile
1m 52s
branch-3 passed

+1 :green_heart:
shadedjars
5m 12s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
1m 52s
branch-3 passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 14s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 35s
the patch passed

+1 :green_heart:
compile
1m 52s
the patch passed

+1 :green_heart:
javac
1m 52s
the patch passed

+1 :green_heart:
shadedjars
5m 11s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
1m 51s
the patch passed

_ Other Tests _

-1 :x:
unit
242m  6s
root in the patch failed.

273m 44s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5587

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 3caaf4ce4b1e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-3 / b7952fcc10

Default Java
Eclipse Adoptium-11.0.17+8

unit
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/2/testReport/

Max. process+thread count
8509 (vs. ulimit of 30000)

modules
C: hbase-archetypes/hbase-archetype-builder . U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5587/2/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


setting form can not handle multiple mirrors anymore

Bitbucket has probably changed the name of the enclosing div of the hook setting form. The div with the class hook-config does not exist anymore. So the index for new subviews is always 0. Probably for the div the new class name is called html-contents.
We have observed this problem with a Bitbucket server v5.5.1.

Thanks, will be doing a new build shortly and will include this.

we have add an encosing div to the setting dialog to be independent from changes outsite.

v2.3.0 is released to the Atlassian marketplace.


Pass command line argv to playback file

What are you trying to solve here? Playback files can already access arguments using $command_split or  $argv as a global.

Is that so? I don't seem to be able to access the $argv variable in the playback file, it is undefined (I renamed the pfSsh.php file to pfSsh.php.old:
<EMAIL_ADDRESS>./pfSsh.php.old run mytestfile someArg1 someArg2

Starting the pfSense developer shell....

NULL

The playback file (mytestfile):
var_dump($argv);

Oh right, my PHP is a little rusty, I forgot the global $argv 🤦♂


Fix OpenApiStreamReader.ReadAsync doesn't respect "leave open" setting

Fixes #1579

The change also needs to be implemented lines 44 and 89 for consistency. Thanks for addressing this super fast!

Here's the PR: https://github.com/microsoft/OpenAPI.NET/pull/1582


Exercises-w1

Exercises
Name
Olaf Driessen
Class
Tech 3
GitHub username
https://github.com/OlafDs
Bootcamp exercises
https://github.com/OlafDs/js-bootcamp
Bootcamp notes
Add a link to your notes in your repository

👋 Thanks, we'll look at the exercises from exercises-w2 for your final grade!


Add NodeStatus and use with AudioPlayer

This adds the NodeStatus enum and uses it with the AudioPlayer Playback Node.

Thanks!


small models being loaded as large fix

noticed there was a bug where the small models were being downloaded and then renamed as large models, at least on a fresh install. everyone was pretty confused all around even the official bark code. (also added some explicit print logging, which you may want to remove)

Thanks, merged this together with the suno changes.


After upgrade to Zulip 1.5.1, Desktop App Message entry is half-width

After upgrade to Zulip 1.51, the message entry box has shrunk:

It still looks right in the direct web client (at least, in Chrome)

Try switching to the electron app? https://github.com/zulip/zulip-electron/releases ; it's a complete rewrite and will be the only supported desktop app soon.

Yes that works fine


Chats not loading from computer cold start

When I start & restart my computer, keybase starts as it should, however, the chats do not load at all.  I'm only greeted with the attached image.
To get past this, I need to quit out of keybase, then start the app, again.
I've only observed this behavior since installing the Windows 10 Creators Update over the weekend.
Cheers,
James
my log id: 54584d9d7a448abf276b1

21c

this behavior hasn't happened again since reporting.  Closing


Text function: block formats where exponential notation is mixed with scaling modifiers

In an expression such as Text(12345.34567,"0.00E+0%") we have inconsistent behavior between Excel and C#. Since this is not a realistic scenario, we should just block this and return an error instead.

Fixed in https://github.com/microsoft/Power-Fx/pull/2007


formHTML not working

I have found an issue with the formHTML. Hoping someone can spread some light on the issue for me.
To see the example you can go directly to the provided demos
http://w2ui.com/web/demos/#/form/1
When rendering the form, it ignores any html provided and renders all the fields from the JavaScript. ( As seen in the above example when you got to jsfiddle for the example http://jsfiddle.net/api/post/jquery/2.1/ ).
***Needless to say this library has been amazing and I appreciate all the work that has been put into it.

You're right, the current master seems to ignore the HTML.
Current master: http://jsfiddle.net/tvL7493u/1/
w2ui 1.5 final: http://jsfiddle.net/tvL7493u/

This seems to be fixed, at least the fiddles are working fine now.
@lukemcc20 can you please confirm?

Closing issue as Fixed.
The issue can no longer be reproduced with the current master.


[CCI] Update Elastic links to OpenSearch ones (#557)

Description
Update Elastic links to OpenSearch ones
Issues Resolved
https://github.com/opensearch-project/oui/issues/557
Check List

[ ] New functionality includes testing.
[ ] New functionality has been documented.
[X] All tests pass

[X] yarn lint
[X] yarn test-unit

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

This looks good as-is - I can update https://github.com/opensearch-project/oui/pull/560 after this is merged.


CE-139 typography

add portal heading sizes and styles
add link styles
add code and pre styles

Note: build files are included in commit because gitbook references the repo and the style.css file.

@Aaron, I agree and I did discuss this with the designers. It would be much better to maintain this in one place. One solution might be to have a branch of nio-css-theme called "portal" and link to that until we can standardize on one nio-css-theme.


[Publishing API]: rejected publication requests are not removed from the bucket

There is an API /v1/ops/publication/delete to delete publication requests in PENDING status while REJECTED ones remain in the storage.
Need to extend API in order to delete rejected prequests

the issue is not reproduced


Typo in MerateImagees

In the Moderate images section you copy & pasted the function call to ModerateText.  I believe you meant to put in the call to the images function instead.
Replace  ModerateText(clientText, TextFile, TextOutputFile);  with ModerateImages(clientText, ImageUrlFile, ImageOutputFile);

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 54699297-e828-23c5-77ff-267f5d9c2593
Version Independent ID: f591d910-92fe-4099-0ec4-7e17800a68b7
Content: Quickstart: Use the Content Moderator client library - Azure Cognitive Services
Content Source: articles/cognitive-services/Content-Moderator/client-libraries.md
Service: cognitive-services
Sub-service: content-moderator
GitHub Login: @PatrickFarley
Microsoft Alias: pafarley

@jkatebin
Thanks for the feedback! We are currently investigating and will update you shortly.

@PatrickFarley Hi Patrick, as we discussed, I have no access to the repo. Please help at this moment and I will figure out for similar scenario. Thanks.

Thank you, I've fixed that code snippet
#please-close


Site shows no name when it has one

Look at the tooltip and in the site details - it has a name, but in the side panel it shows no name....

Fixed in staging

Thanks @meomancer


[zh] Update encrypt-data.md

remove surplus character

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.
It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address.  Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at<EMAIL_ADDRESS>
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Already signed a CLA!

/check-cla

/retitle [zh] Update encrypt-data.md

Thank you for proposing this fix. However, we will not merge partial fixes as per stated in the guideline. To find out what are the upstream changes to be synced, do this on the main branch:
./scripts/lsync.sh content/zh/docs/tasks/administer-cluster/encrypt-data.md

/hold

@yuanfangdewo  Pls sign the CLA in order for the PR to be reviewed.
 Could I please have you do that? https://github.com/kubernetes/community/blob/master/CLA.md

No response from the author for 2 months. CLA is failing.
/close


Reverse mode for array function not yet implemented

This is a very common idiom in numpy programming. It seems like it would be good to implement it.
def f(x):
    x = np.array(x)
    return x**2

df = tangent.grad(f)

raises: ReverseNotImplementedError: Reverse mode for function "array" is not yet implemented.

Good idea. Fixed in c8ff080. Btw, adding new gradients is pretty straight-forward, check out the diff: https://github.com/google/tangent/commit/c8ff0808d03521b3919ae5eaf960f805c83455ef.
We do have a lot of gradients still to add for NumPy, and hope to get to it over the next week or so.


enable strict deabstraction in control_flow.swift (SR-8303)

It passes without any changes!

@swift-ci please test tensorflow

Nice!


fix(Title): do not render title textbox unless there is a title present

Description

The TitleRow was creating the Title textbox in the template, even if no title was ever passed in. This waits to create the textbox until a title is present.
References

Testing

Check that the row title still works as expected in the TitleRow story.
Automation

Checklist

[ ] all commented code has been removed
[ ] any new console issues have been resolved
[ ] code linter and formatter has been run
[ ] test coverage meets repo requirements
[ ] PR name matches the expected semantic-commit syntax

Test Execution Failed.

Test Execution Failed.

Test Execution Failed.

Test Execution Failed.

Test Execution Failed.

:tada: This PR is included in version @lightningjs/ui-components-v2.20.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fixed the bug issue on toolbar that it will continue to look for tool…

…bar icons even if there is no element node.

Looks good.


PSP deprecation blog article linked to v 1.25 release blog

Fix: #36199
Note: Once the changes get approved, the localization team has to perform the task to update the blog in their respective language as well.

I would use the Disclaimer shortcode here. However since @sftim has proposed the change, I'd like him to opine on this as well.

I would use the Disclaimer shortcode here. However since @sftim has proposed the change, I'd like him to opine on this as well.

I only know of a disclaimer shortcode for "this documentation page mentions third party software". I don't think we need to add that to this article.

/retitle Revise PSP deprecation blog article to note that PSP is now removed


Can I restore a backup from another computer?

Hey farag2, Thanks for the script. I just ran it on a new laptop and would like to know if I can restore a backup from my other one that stopped working without breaking anything.
I am worried the backup would install apps that the script uninstalled.
(I’m not tech savvy, I just know more than enough to get around the computer, so forgive me if it should be obvious)

I got ill. I'll reply to you tomorrow...

I tried it on my own and only restored the files, so there were no problems. Get well soon!

Write to the Telegram group. I'll reply much quicker.

I'm here. Can you ask this question in the Telegram group? I dont know what kind of backup you mean...


Firestores collection.add creates 2 entries

Version info
Angular: 5.2.0
Firebase: 5.0.1
AngularFire: 5.0.0-rc.7
Windows 10
Node LTS v8.9.4
The problem
I use the AngularFirestore to create after a file upload a db entrie with:
this.snapshot = this.task.snapshotChanges().pipe(
    tap(snap => {
        console.log(snap);
        if (snap.bytesTransferred === snap.totalBytes) {
            // Update firestore on completion
            this.db.collection('user-avatars').add({path, size: snap.totalBytes}).then();
        }
    })
);

By the way: You can find the full gude where i get my code from here: Firebase Storage With AngularFire - DropZone File Uploader
this.db.collection('').add(); creates 2 entries with same content data in the database. (Just they randomized key is different) And i cant reproduce / understand why it does. Is this a bug? or is simply my code wrong? If this isnt a bug, feel free to close this post.

I imagine there are two firings with bytesTransferred === totalyBytes as the state is going to change too, I'd suggest either checking the state or use the finalize operator.
this.task.snapshotChanges().pipe(
    filter(s => s.state === TaskState.SUCCESS),
    tap(snap => this.db.collection('user-avatars').add({path, size: snap.totalBytes}))
);

this.task.snapshotChanges().pipe(
   finalize(() => this.db.collection('user-avatars').add(...))
);


Check for extra/extraneous arguments

It would be useful to check for parameters in the docstring that are not in the signature - I couldn't see an issue for this though it seems like an obvious addition. For example:
"""Test function."""

def test(arg):
    """Lorem Ipsum.

    Args:
        arg: description 1
        arg_2: description 2

    """
    return a

pydocstyle --convention google <doc>.py will pass with no errors, despite arg_2 being present in the docstring. Typically this can arise if there's a refactor and an argument gets dropped. If arg is removed then we do get a D417 (no description for argument).
It seems like here all we'd have to do is check the opposite case, e.g.
D417 is the set difference of arguments defined in the function and the docstring arguments. Note, the docstring calls it a list - this should be changed to be explicit that it's a set IMO.
function_args.difference(docstring_args) -> returns the empty set if all function_args are present in docstring_args
docstring_args.difference(function_args) -> returns the empty set if all docstring_args are present in function_args
i.e. the example above:
`{"arg"}.difference({"arg", "arg_2"}) -> empty
`{"arg, arg_2"}.difference({"arg"}) -> {"arg_2"}

So we'd just add a new violation (e.g. D420):
extra_args = docstring_args - set(function_args)
if extra_args:
    yield violations.D420(
        ", ".join(sorted(extra_args)), definition.name
    )

While I'm thinking about it, it wouldn't be a bad idea to check for duplicate docstring entries either, but this would need to be done prior to forming set(docstring_args). Happy to PR this if this seems reasonable, but I wanted to check where the violation should be raised - e.g. within check_missing or in a new check? Violations are quite tightly scoped, but it would be fairly clean to check these at the same time. I see we have other places where we can raise multiple violations within the same check, so I think it'd be fine to do both here.

I'm pretty sure darglint does this already and does it better.

Thanks! Unfortunately darglint has been marked archived/read-only as of last month.

The doc param extension in pylint could also do what you want:
a.py:3:0: W9016: "arg" missing in parameter type documentation (missing-type-doc)
a.py:3:0: W9017: "arg_2" differing in parameter documentation (differing-param-doc)


containers: smart state: fix error handling

COMMIT: containers: smart state: missing attribute in exception handling
Allows getting [1] instead of [2] when for example the network is unplugged since SocketError has no error code
COMMIT: containers: smart state: fix error handling
Persist message and status by calling job.process_abort.
Also solves reporting of errors coming from the infrastructure. These include exceptions thrown by the job and timeouts.
COMMIT: containers: smart state: avoid raising errors during cleanup
Since cleanup is called by the infrastructure upon errors, we should avoid throwing exceptions there.
See for example models/job.timeout
[1]
[----] E, [2015-11-23T23:39:36.036341 #23083:feb988] ERROR -- : Q-task_id([ac48b4e4-922a-11e5-aada-0242f322b913]) MIQ(MiqQueue#deliver) Message id: [7400], Error: [getaddrinfo: No address associated with hostname]
[----] E, [2015-11-23T23:39:36.036716 #23083:feb988] ERROR -- : Q-task_id([ac48b4e4-922a-11e5-aada-0242f322b913]) [SocketError]: getaddrinfo: No address associated with hostname  Method:[rescue in deliver]
[----] E, [2015-11-23T23:39:36.036789 #23083:feb988] ERROR -- : Q-task_id([ac48b4e4-922a-11e5-aada-0242f322b913]) /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:879:in initialize' /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:879:in open'
/home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:879:in block in connect' /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:74:in timeout'
/home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:878:in connect' /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:863:in do_start'
/home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/net/http.rb:852:in start' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/rest-client-2.0.0.rc1/lib/restclient/request.rb:494:in transmit'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/rest-client-2.0.0.rc1/lib/restclient/request.rb:202:in execute' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/rest-client-2.0.0.rc1/lib/restclient/request.rb:52:in execute'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/rest-client-2.0.0.rc1/lib/restclient/resource.rb:51:in get' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/kubeclient-0.8.0/lib/kubeclient/common.rb:171:in block in get_entity'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/kubeclient-0.8.0/lib/kubeclient/common.rb:45:in handle_exception' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/kubeclient-0.8.0/lib/kubeclient/common.rb:169:in get_entity'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/kubeclient-0.8.0/lib/kubeclient/common.rb:88:in block (2 levels) in define_entity_methods' /home/mtayer/dev/manageiq/app/models/manageiq/providers/kubernetes/container_manager/scanning/job.rb:143:in cleanup'
/home/mtayer/dev/manageiq/app/models/job/state_machine.rb:29:in signal' /home/mtayer/dev/manageiq/app/models/miq_queue.rb:346:in block in deliver'
/home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:89:in block in timeout' /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:34:in block in catch'
/home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:34:in catch' /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:34:in catch'
/home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:104:in timeout' /home/mtayer/dev/manageiq/app/models/miq_queue.rb:342:in deliver'
/home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:106:in deliver_queue_message' /home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:134:in deliver_message'
/home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:151:in block in do_work' /home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:145:in loop'
/home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:145:in do_work' /home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:335:in block in do_work_loop'
/home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:332:in loop' /home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:332:in do_work_loop'
/home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:154:in run' /home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:129:in start'
/home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:29:in start_worker' /home/mtayer/dev/manageiq/lib/workers/bin/worker.rb:2:in <top (required)>'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/runner.rb:60:in load' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/runner.rb:60:in <top (required)>'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:123:in require' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:123:in require_command!'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:90:in runner' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:39:in run_command!'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands.rb:17:in <top (required)>' /home/mtayer/dev/manageiq/bin/rails:4:in require'
/home/mtayer/dev/manageiq/bin/rails:4:in `'
[2]
[----] E, [2015-11-23T23:37:37.002684 #22174:d6798c] ERROR -- : MIQ(MiqQueue#deliver) Message id: [7334], Error: [undefined local variable or method error_code' for #<ManageIQ::Providers::Kubernetes::ContainerM anager::Scanning::Job:0x0000000462ba20>] [----] E, [2015-11-23T23:37:37.002879 #22174:d6798c] ERROR -- : [NameError]: undefined local variable or method error_code' for #<ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job:0x0000000462ba2
0>  Method:[rescue in deliver]
[----] E, [2015-11-23T23:37:37.002947 #22174:d6798c] ERROR -- : /home/mtayer/.rvm/gems/ruby-2.2.1/gems/activemodel-4.2.5/lib/active_model/attribute_methods.rb:433:in method_missing' /home/mtayer/dev/manageiq/app/models/manageiq/providers/kubernetes/container_manager/scanning/job.rb:145:in rescue in cleanup'
/home/mtayer/dev/manageiq/app/models/manageiq/providers/kubernetes/container_manager/scanning/job.rb:142:in cleanup' /home/mtayer/dev/manageiq/app/models/job/state_machine.rb:29:in signal'
/home/mtayer/dev/manageiq/app/models/miq_queue.rb:346:in block in deliver' /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:89:in block in timeout'
/home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:34:in block in catch' /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:34:in catch'
/home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:34:in catch' /home/mtayer/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/timeout.rb:104:in timeout'
/home/mtayer/dev/manageiq/app/models/miq_queue.rb:342:in deliver' /home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:106:in deliver_queue_message'
/home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:134:in deliver_message' /home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:151:in block in do_work'
/home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:145:in loop' /home/mtayer/dev/manageiq/app/models/miq_queue_worker_base/runner.rb:145:in do_work'
/home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:335:in block in do_work_loop' /home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:332:in loop'
/home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:332:in do_work_loop' /home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:154:in run'
/home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:129:in start' /home/mtayer/dev/manageiq/app/models/miq_worker/runner.rb:29:in start_worker'
/home/mtayer/dev/manageiq/lib/workers/bin/worker.rb:2:in <top (required)>' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/runner.rb:60:in load'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/runner.rb:60:in <top (required)>' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:123:in require'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:123:in require_command!' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:90:in runner'
/home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:39:in run_command!' /home/mtayer/.rvm/gems/ruby-2.2.1/gems/railties-4.2.5/lib/rails/commands.rb:17:in <top (required)>'
/home/mtayer/dev/manageiq/bin/rails:4:in require' /home/mtayer/dev/manageiq/bin/rails:4:in '

@miq-bot add_label providers/containers, bug, smart_state

@simon3z please review

address @simon3z comment

LGTM :+1: cc @chessbyte @blomquisg ready for merge, thanks!

@simon3z tested. Seems to be working well. a few nits I need to address now that we have more time:

a better message for success then 'process initiated'
add status of last analysis in container images screen. not sure on the details.
small issue: for this and other jobs, 'started' time is not saved in the database and the displayed column is always empty. I will open an issue.

@miq-bot add_label wip
Still testing

Tests:

flow tested
result

positive
status: ok, state: finished, message: "image analysis completed successfully"

expected exception during job (cleanup called), no exception during cleanup[1]
status: error, state: finished, message: original expected exception

unexpected exception during job[2]  no exception during cleanup[1]
job finished after timeout, message: ''job timed out after ...". the causing exception in the log.

positive with an unexpected cleanup exception[2]
state: 'finished', statue: 'ok' message: "process initiated" cleanup exception

@simon3z looks ok to me.
[1] "pod X not found, skipping delete"
[2] raise 'unexpected'

@miq-bot remove_label wip

LGTM :+1: cc @chessbyte @blomquisg ready for merge, thanks!

@moolitayer actually let's try to improve this part with spec tests.

@simon3z @blomquisg could you please review? thanks

@moolitayer actually let's try to improve this part with spec tests.

added tests as requested. please review

LGTM :+1: cc @matthewd @blomquisg @chessbyte please review/merge


sap_general_preconfigure: Revert to ignoring DNS failures

Solves issue #784.

@rhmk Is not available this week, so I removed him from the list of reviewers.
@rob0d Please have a look at this PR and comment.

@berndfinger  yes, it would make it safer and more flexible to make the strict check optional and use something like sap_general_preconfigure_dns_check_strict: false as a default.
We can possibly also enhance the PTR check to search through all returned records in a hope that one will match as mentioned in #784 as it will possibly help with the Azure behaviour that @sean-freeman  mentioned. I'm not sure if it's worth the effort though.
I can have a look at this, but I am away until 15th July.

I'm going to merge this PR now to get back the previous default behavior while also implementing an option to enforce strict DNS checking. We can continue discussing further options in the issue and see if we want or can improve the DNS checking later.


[WIP] Undo fixes to resolvconf and dhclient hooks for rhel

For reasons we're yet to know, the recent fix for dhclient
hooks to the resolvconf works only for debian family. Undo
the fix for rhel family, unless RCA finished and new fix is
submitted.
Signed-off-by: Bogdan Dobrelya<EMAIL_ADDRESS>

We will revert fully. I will resubmit the reverted patch plus this fix.


When publish creates Identity folder in wwwroot

Hello, sorry for my bad English, when I publish my project I create a folder called "Identity" in the wwwroot folder.
I use my own layout in /Areas/Identity/Pages/_ViewStart.cshtml
@ {
    Layout = "/Views/Shared/_MyLayout.cshtml";
}

I don't want the "Identity" folder to be created automatically at wwwroot, thx.

.NET Core 3.0

@RetsamSoft thanks for contacting us.
This is already possible if you modify the list of static web assets.
<PropertyGroup>
<ResolveStaticWebAssetsInputsDependsOn>RemoveIdentityAssets</ResolveStaticWebAssetsInputsDependsOn>
</PropertyGroup>

<Target Name="RemoveIdentityAssets">
    <ItemGroup>
        <StaticWebAsset Remove="@(StaticWebAsset)" Condition="%(SourceId) == 'Microsoft.AspNetCore.Identity.UI'" />
    </ItemGroup>
  </Target>

@javiercn
1. Any plans to simplify this by adding support for something like:
    <PackageReference Include="Microsoft.AspNetCore.Identity.UI" Version="3.1.*">
      <ExcludeAssets>staticwebassets</ExcludeAssets>
    </PackageReference>

?
2. dotnet sdk has templates which contain wwwroot folder with bootstrap etc. So after the following commands (for example):
dotnet new mvc --auth Individual
dotnet publish -o Distrib

we end up with wwwroot folder inside Distrib which contains application web assets and another copy of assets under Identity subfolder:
PS> ls .\Distrib\wwwroot\

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       10.01.2020     10:48                css
d-----       10.01.2020     10:48                Identity
d-----       10.01.2020     10:48                js
d-----       10.01.2020     10:48                lib
-a----       10.01.2020     10:48          32038 favicon.ico

PS> ls .\Distrib\wwwroot\Identity\

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       10.01.2020     10:48                css
d-----       10.01.2020     10:48                js
d-----       10.01.2020     10:48                lib
-a----       16.11.2019     11:05          32038 favicon.ico

Any plans to solve this somehow? Maybe dotnet sdk templates must contain Identity assets exclusion in project configuration by default.
BTW dotnet sdk templates actually have more common problem with assets exclusion, for example after the commands above I have the following package references:
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore" Version="3.1.0" />
    <PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="3.1.0" />
    <PackageReference Include="Microsoft.AspNetCore.Identity.UI" Version="3.1.0" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="3.1.0" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="3.1.0" />
  </ItemGroup>

but it should be more like:
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore" Version="3.1.0" />
    <PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="3.1.0" />
    <PackageReference Include="Microsoft.AspNetCore.Identity.UI" Version="3.1.0" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="3.1.0" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="3.1.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>
  </ItemGroup>

As I can see all templates lack design-time assets exclusion for now. I may create separate issue for that if you don't have one already.

@mmarinchenko I don't think we plan to do this.
At best we would offer a property in the identity UI package to turn off adding these assets, but we don't plan to build generic functionality for this, as there is already an extensibility point to do this.
We don't plan to extend the PackageReference itemgroup for this feature.

Ok, thanks.


Definition - Stress catalogues and repos

Stress the catalogues and repos inserting, updating and deleting entries using the API. Try to get numbers about how many request per second can support

For both Catalogue and Repositories, the stress tests will be splitted in entry type and the stress:

NSD: POST descriptors with random ID, PUT (update) with random VERSION, DELETE, increasing the requests time intervals
VNFD: POST descriptors with random ID, PUT (update) with random VERSION, DELETE, increasing the requests time intervals
PD: POST descriptors with random ID, PUT (update) with random VERSION, DELETE, increasing the requests time intervals
Package file??
NSR: POST, increasing the requests time intervals
VNFR: POST, increasing the requests time intervals

@dang03
Package files have to be generated according to the descriptors.

On 22 Mar 2017, at 10:05, Daniel<EMAIL_ADDRESS>wrote:
For both Catalogue and Repositories, the stress tests will be splitted in entry type and the stress:
NSD: POST descriptors with random ID, PUT (update) with random VERSION, DELETE, increasing the requests time intervals
VNFD: POST descriptors with random ID, PUT (update) with random VERSION, DELETE, increasing the requests time intervals
PD: POST descriptors with random ID, PUT (update) with random VERSION, DELETE, increasing the requests time intervals
Package file??
NSR: POST, increasing the requests time intervals
VNFR: POST, increasing the requests time intervals
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/sonata-nfv/son-tests/issues/231#issuecomment-288351865, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAuQ7STHaDMrzc50zLGNhUzrGjhCKllks5roPJWgaJpZM4Mc2-9.


Stricter Lockstep Check

Right now, we consider it lockstep as long as the nIters are fixed and it is not a stream controller.  Really, we want it to be considered lockstep also if the start/stop/step come from RegReads not enclosed by a relative control. For example
'LOOP1.Foreach(M by 1 par 2){i =>
  val x = reg.value
  'LOOP2.Foreach(N by 1 par 2){j =>
    'LOCKSTEP.Foreach(x by 1){k => ...}
  }
}

LOCKSTEP should be true if we look relative to LOOP2 but false if we look relative to LOOP1
Working on this now

has been made stricter.  Now it will consider lockstep if the controller is not a stream or FSM and if one of the following is true

fixed number of ctr iterations
Relative to some outer controller to be unrolled, all of the start, step, or stop are either constants, reads of an ArgIn, or are RegReads that happen outside the scope of that controller


Scrolling large data set of 1.000 rows, causes flickering

Using release candidate 3.0.0-rc.12. The grid has drawing issues when using the scrollbar in combination with large data set.
ui-grid-canvas, has a small percentage of hidden rows. 5% top and 5% at the bottom, so the viewpoint is showing 90% of the rows. Is it possible to increase the size of the canvas? Say 80% hidden. This will probably solve my case.

Is there something that changed for ui-grid to struggle to keep up while ng-grid is fine? The ng-grid example seems to work fine so there's surely something that's causing the issue in ui-grid rather than just acceleration settings?

Should I open a new issue about this or is this already known?

I think that's an unfair comparison, the ng-grid plunker has only 4 columns and no additional features being used (such as edit etc).  Compare the large dataset tutorial: http://ui-grid.info/docs/#/tutorial/404_large_data_sets_and_performance, which flickers a little, but is generally good.
I you look at the performance monitoring in chrome the time is all disappearing into angularjs, mostly directive compilation.  There's not a huge amount we can do about that.

I am seeing similar performance in the large dataset tutorial to the ui-grid plunker - especially on mobile.
The ui-grid plunker is also only using 4 columns and no additional features unless I'm missing something that's making the comparison unfair.

Ah, then I was probably comparing to the similar discussion on another issue which had 100 columns x 100 rows and all features enabled.
I suspect some of the lack of interest comes from "Hopefully it is not architecture mistake in the core design." which isn't the kind of comment that makes people who've spent 1-2 years building this happy.

Hello, I'm experiencing the same problem on a v3.1.1 grid of about 20 columns and about 1700 rows, edit enabled on the last four columns, cell navigation.
I see the issue has been closed but working it around with either excessRows or virtualThreeshold makes the initial rendering very slow.
Is there something else I could try?
Thank you,
Salvo


[MM-10394] Make hidden town-square (on read-only) be available to view by joining in more channels

Summary
Make hidden town-square (on read-only) be available to view by joining in more channels
Related PRs:

https://github.com/mattermost/mattermost-server/pull/8751
https://github.com/mattermost/mattermost-redux/pull/493
https://github.com/mattermost/mattermost-webapp/pull/1194

Note: will update this PR once server and redux changes are merged.
Ticket Link
Jira ticket: MM-10394, also related to MM-10430
Checklist

[x] Added or updated unit tests (required for all new features)

Device Information
This PR was tested on: [iOS simulator and Android emulator]
Screen captures
Android:

iPhone:

Thanks @enahum for the guidance!
I'll update this and also other PRs (redux, webapp) based from our discussion and the revised ticket.

@saturninoabril where are we on this one?

I'll prioritize this (and other PRs) today and tomorrow.

@saturninoabril PR looks good, but I'll wait until the redux PR is merged and this PR updates the reference to the mattermost-redux repo.
sounds good?


Add support for multiple ShadowUI stylesheets (close #155)

/cc @churkin @miherlosev
@VasilyStrelyaev Please check test names

\r-

lgtm

Add a test for a case when ShadowUIStylesheed doesn't exist.

@miherlosev I think that a test for a case when ShadowUIStylesheed doesn't exist is redundant, because all actions with ShadowUIStylesheed is produced using documentFragment

ping @churkin @miherlosev

I disagree with @LavrovArtem
ShadowUIStylesheed doesn't exist - is a test case.
all actions with ShadowUIStylesheed is produced using documentFragment - implementation
waiting for @churkin resolution

@LavrovArtem  It would not be superfluous, add please.

FPR

@VasilyStrelyaev please check test name ShadowUIStylesheet doesn't exist

Hmm... It doesn't make much sense
When shouldn't it exist? Probably by default or after something happens?

if ShadowUIStylesheet doesn't exist then nothing happens

Do nothing if ShadowUIStylesheet doesn't exist

FPR

\r-

FPR

\r-

FPR

lgtm

lgtm


Fix set_up_gitlab no such option error in docker-compose

If the GITLAB_REMOTE_LOGIN_PAT happens to start with a dash / hyphen,
it's treated as an option for dotenv:
Write the python-gitlab configuration file to: /data/var/.python-gitlab.cfg
Create the 'houston' user
Create a PAT for the 'houston' user
Append 'houston' user PAT info to the python-gitlab configuration file
Create the 'test' group
Write 'houston' PAT to /data/var/.env
Usage: dotenv set [OPTIONS] KEY VALUE
Try 'dotenv set --help' for help.

Error: no such option: -4

It is possible to reproduce this error on the command line:
root@86b973f9000a:/data/var# dotenv -f .env set GITLAB_REMOTE_LOGIN_PAT "-abcd"
Usage: dotenv set [OPTIONS] KEY VALUE
Try 'dotenv set --help' for help.

Error: no such option: -a

It can be fixed by adding -- to say that anything after that isn't an option:
root@86b973f9000a:/data/var# dotenv -f .env set GITLAB_REMOTE_LOGIN_PAT -- "-abcd"
GITLAB_REMOTE_LOGIN_PAT=-abcd

Pull Request Checklist

[x] Ensure that the PR is properly formatted

Example: All lint checks are passing

[x] Ensure that the PR is properly rebased

Example: The PR is rebased on develop (commit: <insert develop commit hash>)

[x] Ensure that the PR uses a consolidated database migration

Example: One database migration is proposed (revision <insert develop revision> -> <insert new revision>)

[x] Ensure that the PR is properly tested

Example: All automated tests are passing

[x] Ensure that the PR is properly covered

Example: The percentage of the code covered by tests has not decreased

[x] Ensure that the PR is properly sanitized

Example: No sensitive data or large content was added to this PR

[x] Ensure that the PR is properly reviewed

I'm not sure = is a valid token character, but I verified that works with python-dotenv just in case.

The error this resolves was occasionally observable in CI and/or on developer machines.


Tabs inside RB Sheet not scrollable

Hi,
I am using Tab navigator inside RB sheet. But I can't swipe between tabs. Tabs swipe is not working, inside sheet. Can you please suggest on this.

  <RBSheet
    ref={taxSheetRef}
    height={300}
    openDuration={250}
    dragFromTopOnly={true} />

Add drag from top only.


Fix broken link to HRRR archive

Description

In WeatherModels.md, the link to the HRRR archive is missing the protocol bit, causing GitHub web to think it goes to https://github.com/dbekaert/RAiDER/blob/dev/home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_FAQ.html instead of https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_FAQ.html. Adding https:// to the beginning of the URL fixes this.
Type of change

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[X] I have added an explanation of what your changes do and why you'd like us to include them.
[ ] I have written new tests for your core changes, as applicable.
[ ] I have successfully ran tests with your changes locally.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

@the-garlic-os we probably should add to this PR to fix the problem with the HRRR API as well. @piyushrpt may also look at this, but for now you could take a look at implementing a solution. Did some quick searching and this page looks like a good place to start, and also found this package that may help.

@jlmaurer I dont think this is needed anymore. We can close PR as its addressed in PR#379

Yep agree!


Licence checking

We should include a link to license_finder in the guide (or turing way).
Here is an example of how it is used.

This is best done in The Turing Way


Task and GitHub action to handle releases

I have submitted a PR for this:
https://github.com/jesusprubio/up/pull/64


Tags are not recognized on HEIC images

The Database contains no tags for images with the Apple heic format. I double checked that there are some images which do contain tags but they do not land in the database.

Appearantly, digikam does not write Tags to HEIC files but keeps them in the database only (as per https://exiftool.org/forum/index.php?topic=12876.0). That would explain something, I will investigate further...


[EDS-556] Return student search results when requester is authenticated

This is a larger PR that closes out EDS-556. It is broken into 3 commits. I recommend reading each individual commit message for more details of what this changes.
The broad strokes:

Updates the backend models to be cognizant of the new "populations" layer
Updates the frontend to use the new backend models
Updates (and adds) tests for the new behavior

Updated, please see latest commit message

Cool, I'll start back on this in the A.M with a fresh brain.

hmm, there is a test for the sign in link persistence. let me take a look, i don't want to commit a regression to something that i've already fixed 3 times.

Ok, I can repro and identified why my the test didn't catch it. Thanks again!

You're welcome and thank you for rolling with it :)

Ok, fixed again! And I added a test to catch henceforth.

Fixed, and as mentioned offline (but noting here for visibility), it'd be great if you could create a jira with these manual test cases that you'd like to automate.

create a jira with these manual test cases that you'd like to automate.
Great idea! Hopefully this will reduce the back and forth and check for common bugs.
https://jira.cac.washington.edu/browse/EDS-565

Are there any other test cases you need to run before I make another change? I would prefer not to keep finding them one at a time.

There a likely several smaller test cases that I will document as I think of them.
I understand your concern.
How about we get this last one fixed and if the cases I've been running all pass, we merge.
If anything outside of those discovers a bug, I'll file a JIRA bug for it and add to the test cases needed.

In that case I would prefer to just defer other fixes. This isn't getting us anywhere, and i don't think the fact that we have incomplete test cases should continue blocking this PR.
I fulfilled the contract of expanding tests for additional functionality, all of which passed, and expanding tests for new test cases that emerged. I think at this point, a better option is to merge this PR, and then work on automating the test cases as a separate jira, which you've already created. That will be responsible for fixing every little thing. The fact that there were test cases of which I was never made aware shoudln't be a punishment to the effort to return student search results.


Add more examples of using of adding products in cart

I need more examples of work with cart and REST framework to DjangoShop docs. I don't use angular and I need extend the logic of adding products in cart.
I have cloth shop. One product has one or more sizes and colors.
Firstly, I need store more atributes for CartItem. I extended default CartItem:
class CartItem(cart.BaseCartItem):
    quantity = models.IntegerField(validators=[MinValueValidator(0)])
    color = models.CharField('Color', null=True, blank=True)
    size = models.CharField('Size', max_length=5, null=True, blank=True)

In product-detail.html I use next code to add product in cart:
$(function(){
    var d = {
        'quantity': 1,
        'unit_price': '120',
        'subtotal': '120',
        'product': 1,
        'color': 'white',
        'size': 'XL'
    };
    $('.buy').click(function(){
        $.post('{% url 'shop:cart-list' %}', d, function() {
            alert('product is added to cart')
        });
    });
});

But color and size don't save in my CartItem.
Secondly, I want see my cart as like:
name           color   size   quantity
product_1   white   XL   1
product_1   black   S   2

product_1 has same id. Now if I add product_1 product in cart then I get:
name           color   size   quantity
product_1   white   XL   3

How I add product to cart separately if size or color different?
Please, help me. I think I need customize serializer but I can't it bacause I get errors.
I think DjangoShop docs need more examples to developers could be understand how they can extend their shop logic.

First of all, I would store product variations inside the extra field, otherwise you also must redefine your own OrderItem. But if you want to do it as explicit fields, then you have to create your own AddToCartSerializer for the additional fields color  and size. Have a look at how AddSmartCardToCartSerializer is implemented in example/myshop/serializers.py.
For adding product variants, override the method is_in_cart(...). This should return the cart item, if the product to add is considered the same, or None, if it should be added. This is documented though.
You should also override the method get_instance(...) so that you can deliver the initial data, say size and color choices, to the product's detail view which renders the AddToCart form.

Thank you very much! I try to realized your hints and will say about my result. Also I try to improve using django shop in russian community through my blog. I think it system is the most flexly and powerful. I like its concepts. Thank you for django shop!

Thank you, Jacob for your help. I have finished my aim.

hello vivazzi, I am also trying to implement a clothline cart which has color ans size but size and color are not being saved to the database. do you mind sharing your  solution. Thank you


HTTPS?

Not really an issue with the software, but any plans on getting a certificate for the site to prevent MITM tampering with the converted extensions?

No plans at the current address. We are looking for a new host for the converter and other related stuff so if that works out and the host turns out to have a free https option then it will be possible.

@lemon-juice If you want, I could set up the add-on converter at nearlyfreespeech.net and get it an HTTPS certificate.
It would probably only be about $20/year (which I'm fine with paying for myself) and I could give you or anyone else access to the site as well.
By the way, does the converter use SQL for anything besides making a log?

It turns out my host has started recently offering free auto-renewing Let's Encrypt certificates so I've turned it on for the addon converter, no need to pay $20 :). Will it be sufficient? Considering the popularity of SeaMonkey I don't think any hacker would care about eavesdropping even unencrypted addon conversions :).
You can change converter links to https in AMO Browsing for SeaMonkey.
Yes, SQL is only used there for saving logs.

Thanks! I think it's nice to have https working, just in case someone tries
it.
On Oct 30, 2017 6:26 PM, "lemon-juice"<EMAIL_ADDRESS>wrote:

It turns out my host has started recently offering free auto-renewing
Let's Encrypt certificates so I've turned it on for the addon converter, no
need to pay $20 :). Will it be sufficient? Considering the popularity of
SeaMonkey I don't think any hacker would care about eavesdropping even
unencrypted addon conversions :).
You can change converter links to https in AMO Browsing for SeaMonkey.
Yes, SQL is only used there for saving logs.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/lemon-juice/addonconverter/issues/1#issuecomment-340614791,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABZGKzrkYU7sG8H1O4p2-IQ6RHDalCX1ks5sxls0gaJpZM4FRxuW
.


Actions in components contained in sortable-item do not always work

I'm having intermittent problems with actions when I'm using a sortable list. I have a list of items, each rendered with a component. That component has a "delete" action on it, activated by clicking the 'x' next to the item name.
The following code should give an idea what I'm talking about.
{{!-- templates/items.hbs --}}
{{#sortable-group onChange="itemsReordered" as |group|}}
  {{#each model.items as |item|}}
    {{#sortable-item model=item group=group handle=".item-list-item"}}
      {{item-list-item item=item deleteItem="deleteItem"}}
    {{/sortable-item}}
  {{/each}}
{{/sortable-group}}

The item-list-item component has the following template:
{{!-- templates/components/item-list-item.hbs --}}
{{item.name}}x

and the accompanying script file
// components/item-list-item.js
import Ember from 'ember';

export default Ember.Component.extend({
    classNames: ["item-list-item"],
    item: null,

    actions: {
        deleteItem: function () {
            this.sendAction("deleteItem", this.getAttr("item"));
        }
    }
});

Until today, I've found that this has worked flawlessly (or possibly almost flawlessly, and I just assumed that I'd missed the 'x' when it didn't work), as have my two colleagues who have been using the code. As of today, both myself and my colleague are finding that more often than not (> 90% of the time), the deleteItem action doesn't get called. Looking at the DOM inspector, we can see that when this is the case, the .item-list-item div gets the is-dragging class, whilst when deleteItem gets called, it does not. My guess is that there is a race condition between which handler gets called first, and the second handler does not get called.
For now, I can work around this by making the draggable part not include the 'x', but I'd like to avoid doing this if possible.

Thanks for the detailed report.
The solution you’ve gone with is what I’d recommend. I do agree, however, that it should work out of the box. There is some code in ember-sortable to ignore clicks that don’t become drags but evidently it’s not working 100% of the time.
I think the best course of action will be to add an example of this to the demo page and an acceptance test to go with it.

The strange thing is that it did seem to be working straight out of the box. It's a tricky one to test because it's intermittent.
I won't be able to add the example for a while since we're at the beginning of a hectic patch :-(

@AndyPyeSC no problem. Thanks for the detailed. Should provide the basis for a test case.

I'm encountering a similar problem, but totally reproducible.
I'm using this template:
{{#sortable-group classNames='list' as |group|}}
  {{#each models as |model|}}
    {{#sortable-item model=model group=group handle='.handle'}}

 Move

         Action

    {{/sortable-item}}
  {{/each}}
{{/sortable-group}}

To reproduce:

Drag an item to PAST the top of the sortable group
Release it there
Wait for it to slide back to the position it used to be in.
Click on the item you just dragged.

When I follow those steps, the action is never sent.
As long as I never drag the sortable-item past the top of the sortable-group, the action is sent just fine.

Also of note, this issue interacts negatively with ember's link-to helper:
If I put a link-to helper inside the sortable-item, then these repro steps cause the entire page to refresh.

Brilliant repro. Sounds like an event handler is not getting torn down correctly.

Hmmm kind of a deal breaker for me right now with the way i have inputs/components inside the sortables. Going to look into a workaround or a fix. Would love any other input too if someone has found something

@alexparker best of luck finding an alternative solution. Sorry we couldn't get to this one in time.

@jgwhite I found its commonly used UX to have a user toggle a active "sorting" behavior, so I decided to go that route with this library. So it actually renders alternate template for sorting, then renders the interactive component items once the user clicks "done sorting" - it was a suitable workaround. Thanks for your apology, I hope I can contribute something in the near future.

+1 for this one, would love to see a fix. The behavior I am experiencing is: say you have a button within a sortable item. Clicking on the button before sorting works fine. If you sort that item, then it will take two clicks to trigger the action on the button, the first one is ignored.

for a fix, could we not trigger drag until a certain amount of threshold dragged?

@NullVoxPopuli that was added in #137. I need to release a new version with that fix added.

oh, cool. @acburdine until you do that, I'm going to test out the master branch.

nah, bug that I'm experiencing is still there.
I have a sortable-item that contains a link-to. when I let go of the mouse, the whole page navigates.
I'm not using a handle.
actions within a sortable-item don't fire

@koreiklein did you ever find a solution?

Is there any progress on this bug? After dragging the action on the draggable item isn't fired. It does work on a second click.

Any progress? I installed the latest version of the package, but the bug is still exist.

@nagytezsolt I’ve not taken a look into this issue lately, though I was under the impression that a previous PR had resolved it (forgive my haziness). Are you encountering the issue exactly as described above? If you can share a little bit of code, it would be great to learn a little bit about exactly how you’re using it.

{{#sortable-group tagName="ul" onChange="reorderItems" as |group|}}
{{#each fields as |field|}}
{{#sortable-item tagName="li" model=field group=group classNames="sortbox__li sortbox__li--small" handle=".sortbox__li"}}

{{field/field-autoselect class="sortbox__select" content=sources 						 valuePath="id"   labelPath="name" value=field.collection_name modal=true}}
{{field/field-autoselect class="sortbox__select" content=field.sourceOptions valuePath="name" labelPath="name" value=field.column          modal=true}}

{{#if isMobile}}

p

{{else}}

p

{{/if}}
{{/sortable-item}}
{{/each}}
{{/sortable-group}}

As figured out (thanks to @rygor-monterosa), _preventClick function sets click handler _preventClickHandler on sortable item right before drop. This is to prevent unintended button clicks.
_preventClick() {
    $(this.element).one(elementClickAction, this._preventClickHandler);
  },

But for some reason, this handler is not fired during drop, as a result - first intended user click is handled by _preventClickHandler.
Hope this information will help to properly fix the bug.
One possible work-around that we used:
{{#sortable-item
  ...
  onDragStop="onDragStop"}}

onDragStop: ->
      @$("li").each((_idx, item) => $(item).off("click.emberSortable"))

Using the onDragStop() function resolves the problem. Thank you!

This project is now being maintained by the team at @heroku.
To get the project healthy and maintainable again, we're doing a bit of spring cleaning of older issues and pull requests. This issue seems more-or-less resolved so we're closing it.
Please feel free to re-open if you'd like to resume the conversation.


Streaming output for language models

Large language models typically stream tokens as they are running.
We've called this "progressive" output in the past in Cog, but "streaming" is the more common term, so perhaps we switch to that.
Prior art

OpenAI's API returns server-side events as JSON messages in the format {"data": "..."} with each token.

Proposed design
Models return an iterator of strings that are concatenated together. There is a special type called Stream[str] that works the same way as Iterator[str], except it implies that the outputs should be concatenated together to get the final result.
For example:
from cog import BasePredictor, Stream

class Predictor(BasePredictor):
    def predict() -> Stream[str]:
        yield "The "
        yield "quick "
        yield "brown "
        yield "fox "

The output type looks something like:
{
    "type": "array",
    "items": {
        "type": "string",
    },
    "x-cog-array-type": "stream",
}

Other designs considered

We just use Iterator[str]. The problem with this is that the current semantics of this is in-progress snapshots of the output (e.g. diffusion models diffusing), not parts of the output that need concatenating together. You could also imagine situations where you return distinct strings as the output that you don't want concatenated together. So, we need a bit of annotation to hint that the output needs concatenating for things like Replicate's web interface.
We return an iterator of output in the the format ["The", "The quick", "The quick brown"]. The advantage of this is you don't have to concatenate output together to get the completed output (it's just output[-1]) but this returns a lot of data in each response and makes it harder to process each token as its coming in (you have to keep track of your position in the string).
We modify Cog to output a string that can change over time. E.g. {"output": "The"}, {"output": "The quick"} etc. This means that the default case where you just want to get the complete output is really simple (no concatenation required) but it'll require some rearchitecting of Cog and makes it harder to process each token as its coming in (you have to keep track of your position in the string).
We have a similar server-side events API to OpenAI. This is a big change.

See also

https://github.com/replicate/cog-llama/pull/4

@bfirsh I like this. I think the one outstanding thread here is handling the case where a user generates more than 1 completion for a given prompt. OpenAI returns a stream of jsons, each of which are basically a (completion_id, token, metadata) tuple, where completion_id is between 0 and the number of requests generated. Quick example:
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "text-davinci-003",
    "prompt": "Say this is a test",
    "max_tokens": 7,
    "temperature": 0,
    "n": 3,
    "stream": true
  }'
data: {"id": "cmpl-6xmTm0BgCMDjPIYUalSbIb1ZosJro", "object": "text_completion", "created":<PHONE_NUMBER>, "choices": [{"text": "\n", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-003"}

data: {"id": "cmpl-6xmTm0BgCMDjPIYUalSbIb1ZosJro", "object": "text_completion", "created":<PHONE_NUMBER>, "choices": [{"text": "\n", "index": 1, "logprobs": null, "finish_reason": null}], "model": "text-davinci-003"}

data: {"id": "cmpl-6xmTm0BgCMDjPIYUalSbIb1ZosJro", "object": "text_completion", "created":<PHONE_NUMBER>, "choices": [{"text": "\n", "index": 2, "logprobs": null, "finish_reason": null}], "model": "text-davinci-003"}

data: {"id": "cmpl-6xmTm0BgCMDjPIYUalSbIb1ZosJro", "object": "text_completion", "created":<PHONE_NUMBER>, "choices": [{"text": "\n", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-003"}

data: {"id": "cmpl-6xmTm0BgCMDjPIYUalSbIb1ZosJro", "object": "text_completion", "created":<PHONE_NUMBER>, "choices": [{"text": "This", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-003"}```

Interesting to note that these don't necessarily come in order, implying that the tokens aren't all being generated on the same GPU. We're not set up to do that presently, though once we have native batching under the hood, this just becomes a batch prediction instead of a single prediction where `n>1`.

Potential approaches:
* Return a `Stream[List[str]]`. Kind of clunky for the majority use case where we're only returning one prediction.
* Follow OpenAI's pattern. In the instance where there are multiple sequences being generated, we return a `Stream[LLMOutput]`, where `LLMOutput` contains `index`, `text`, and potential other metadata. Great flexibility, a bit more parsing required on the downstream end.
* Intentionally punt on this now, and add it when we get to implementing batch prediction. Either remove the `n` option for predictions, or indicate that `n>1` isn't compatible with streaming.

This is a good point, but considering we haven't figured how we want batching to work yet, I reckon we punt on this for now. The point of the Stream type hint is that the output can be concatenated together, and the other two designs break that.
Ref #612

@bfirsh fair. I think it'd be good to have the ability to toggle streaming on & off for folks who want to return n>1 completions for a given prompt (not how GPT-4 works but useful for some sampling applications) - or for folks who just want to get all their text all at once.
The only way I can think of to do this in cog atm, since predict requires a consistent return type, is toyield an entire generation at once if stream is off. Easy enough to implement, kinda hacky. What's your take?