feat: mark memoized node as cached

Fixes https://github.com/argoproj/argo-workflows/issues/13848
Motivation
when using memoization, its much easier to debug workflows when you can see that nodes have been cached. currently in large workflows you'd have to go over all the nodes one by one to understand if they are cached, or you could go to the timeline tab.
both of those options offer a bad user experience, that becomes worse as the workflows get larger.

Modifications
changed the icon of successful cached nodes to a database icon (from font awesome) to mark them as cached.

Verification
tested in the UI that it works using the examples/memoized-simple.yaml example
screenshots:
first workflow (no cache hit)

second workflow (cache hit)

I like the choice of icon - for the abstract concept of "cached" it seems to align with the rest of the world https://www.google.com/search?q=icon+to+represent+cache
But replacing the check mark within the node itself might be confusing for existing and new users alike - it looks like it's a step related to database access, and two workflows running the same thing will look quite different. I would suggest an additive approach e.g. an overlaid database symbol - quick sketch from a google slide

I've added the requested change, I'm not much of a UI UX designer so if you have any changes you'd like to have to the colors and what not, please share

this is the new icon


WIP: support redis versioning

Stores a new key "version" in redis db which contains the git hash
souper was built with (or timestamp if git hash is unavailable). If git
hash of souper matches value of "version" in redis db during connection,
all is good. If it doesn't matches, we error out. If no "version" key
matches, we put current souper version in the redis database and warn
the user that results might be incorrect.

okay. above TODO is fixed now. It works with ninja now. I think I tried this approach earlier also but it was stymied by the pthread bug that I pushed a fix for above. This time I was careful and tracked it down; so, one shot, two bugs ;)

ok, is this ready to merge? it looks fine to me. you've tested it out I assume?
if all is ready, please squash

i need to add a couple of things you mentioned above (redis tests, etc.). Will let you know when done.

what's status of this pr?

this turned out to be much more complicated than i anticipated. It works on my desktop computer and fails intermittently on my laptop. There are some CI failures as well where it doesn't work. Turns out i need to give it another look (i am just avoiding this problem for now to have a look at it from fresh perspective in coming weeks).
the problem remaining is the same: ninja doesn't like the way i am doing header generation in cmake and compiles the C++ file, which depend on the generated header, before generating the header.

ok thanks
no big rush on this

Okay. So I changed this and now we set the new key in the redis database equal to the "argument names" passed to the command line. Eg: if you invoke souper with following command line:
./souper -z3-path=/usr/bin/z3  -solver-timeout=15  -souper-external-cache -souper-debug-level=3 a.ll
we will construct the following key out of it (let's call it A).
solver-timeout,souper-external-cache,z3-path, (yes, this will be sorted and debug-levels are excluded).
and then we will check if key named cachetype exist in the redis db or not. If it doesn't exist, we insert a new key called cachetype and set its value to A. If it does exist and is not equal to A, we error out and don't proceed. And obviously if it does exist and is equal to A, we continue like normal.
Note that A doesn't contain the "values" of the command line (things after "=") -- there's just no way you can get these values from LLVM's CommandLine library with current API. I am hoping that this could be sufficient for our use-case. After discussion with @zhengyangl , the dataflow stuff is itself a command line options; so, if you already have a cache tagged with say infer-known-bits,infer-power-two, you can't use the same cache when command lines arguments doesn't involve these.
Let me know if this works.

 there's just no way you can get these values from LLVM's CommandLine library with current API
how sure are you about this?

Pretty sure. The CommandLine library exposes getRegisteredCommands that give us access to the Option *. Now the problem is that values for option is part of the cl::opt class which is a subclass of Option class. In a normal situation, I would do a dynamic_cast and get a cl::opt * but in this case, I can't do that because LLVM doesn't use RTTI. My other option is to use LLVM's dyn_cast but I can't use that as well because in order to be able to use dyn_cast without RTTI, classes needs to have a method called classof in their body but these classes doesn't have that.

so you're saying you can't do the getValue() thing suggested here?
https://stackoverflow.com/questions/43336602/where-to-find-o-output-argument-in-clang-llvm

if it's not practical to make that work, I think you'll need to make a list of options that have interesting stuff on the RHS and look for those specifically. like the number of synthesis components, this totally matters, that sort of thing is the whole reason for the cache tags, it really doesn't make sense to do this and leave those out.

cl::opt is a template class with 3 parameters. So for any given type T, there are many possible instances of the class. I can't do static_cast as suggested in the link you pasted because I don't know what template class a given option is going to use unless I bloat the code and put all this information in a huge function (which doesn't sound like a good solution)

like the number of synthesis components, this totally matters,

yeah, that's true. I will think of something. Since it's clear now that the current patch is not sufficient, I will think more about it

like the number of synthesis components, this totally matters,

as we discussed in the last meeting, i went ahead with whitelisting some of the synthesis options. Now it includes values of the options a well. It should be easy to add new whitelisting components.

what's the status of this?


DeprecationWarning when creating a new Configuration object

This is the line of code generating the warning:
const apiConfig = new Configuration()
The warning received:
(node:22088) [DEP0044] DeprecationWarning: The 'util.isArray' API is deprecated. Please use 'Array.isArray()' instead. (Use 'node --trace-deprecation ...' to show where the warning was created) DeprecationWarning: The 'util.isArray' API is deprecated. Please use 'Array.isArray()' instead.     at FormData.append (.\node_modules\form-data\lib\form_data.js:64:12)     at new Configuration (.\node_modules\sailpoint-api-client\configuration.ts:140:22)

Hey @mostafa-helmy-sp, I dug into this and we rely on a form-data package where this deprecation is thrown. They have a PR out for fixing this, once it is merged in we can update to the latest version. I will keep an eye out on it. https://github.com/form-data/form-data/issues/563
Thank you for bringing it to our attention!


Jitterer

Adds a time.jitter function to pam.samplers.
def apply_jitter_to_plan(
    plan: Plan,
    jitter: timedelta,
    min_duration: timedelta
    ):

Apply time jitter to activity durations in a plan, leg durations are kept the same. Activity durations are jittered in sequence order. At each step the activity is jittered according to the maximum jitter and minimum duration of all activities in the plan.

Hard to test but experience has been good so far.
@alex-kaye @ana-kop I share with you because this might be useful in a random walk implementation, eg:
def sort_of_random_walk(plan, steps=n):
    best_score = scorer.score(plan)
    for i in range(n):
        proposed_plan = deepcopy(plan)
        apply_jitter_to_plan(proposed_plan, ...)
        score = scorer.score(proposed_plan)
        if score > best_score:
            plan = proposed_plan
            best_score = score

Fixed a bug - now ready for quick review @ana-kop @alex-kaye


client sockets arising from connection to a server socket do not report disconnection

Easily verifiable with the echo server sample in uproxy-lib:
https://github.com/uProxy/uproxy-lib#echo-server
When I telnet to the echo server and kill the telnet process, nothing appears in the echo server logs (this works fine in Chrome).

I just performed some testing with the echo server and it looks like this has been broken since at lesat freedom-for-firefox 0.6.13.

Look at this code...the onDisconnect handler is never set for "incoming" client connections. onDisconnect only works if you create a new socket and call connect. This explains some of the weirdness I'm seeing in the SOCKS proxy.


hubot does not wake up on Heroku when I ping it

I installed hubot for slack from win 10. Everything works well until the bot was put in sleep mode. I can not ping it to wake it up even if I set URL var in Heroku Config Vars. I accessed it via url and it woke up. How can I solve my problem? Thank you.

+1, I've been having this issue too.

I'd recommend checking out the main heroku keepalive repo here https://github.com/hubot-scripts/hubot-heroku-keepalive

I'm currently on it, but it is not working for some (yet) unknown reason.

Sorry to hear that. That's definitely the best place to figure it out.
Just to clarify one important thing, it's not really the bot that's sleeping, it's the server (technically the "dyno"). It doesn't really have anything to do with Hubot or a particular adapter, it just so happens that a lot of people use Heroku to run Hubot instances and Heroku uses this sleep concept to manage their resources.

This is an issue with the hubot-heroku-keepalive, not hubot-slack. Can you (or anyone else) running into problems keeping Heroku alive file an issue there please?


[ADAM-2224] Complete refactoring of compressionCodec for named parameter.

Fixes #2224

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3069/
Test PASSed.


How to use supplied SQL statement verbatim

Received via email:

Hi, now i'm using logstash-output-jdbc, my data:
{
"reason" => [
[0] "1",
[1] "2",
[2] "3",
[3] "4"
],
"classid" => "31186082",
"@timestamp" => 2017-07-26T09:41:41.851Z,
"@version" => "1",
"host" => "ip-10-0-10-39",
"statement" => "insert into qos_finish_classid_reason(class_id,reason_id) values (31186082, 1),(31186082, 2),(31186082, 3),(31186082, 4);"
}
and, i want to use the filed statement :
jdbc {
statement => ['?', "statement"]
}
but, it always fail. Can you help me?? Please.

By default logstash-output-jdbc uses prepared statements, and your attempt isn't valid.
However, if you're super sure your statement is safe, you can do the following:
jdbc {
   unsafe_statement => true,
   statement => ["%{statement}"]
}


Unable to get mesh on click event

Hi @deltakosh,
I am trying to get mesh on which I am clicking. For some models, it works fine. But for the attached model it is not getting detected. It returns undefined for the clicked mesh.
Attaching Babylon model for reference.
ge_model.zip
Following is the log of pickedMesh got from this model click

But it works fine for the following model
cube.zip
and the log of picked mesh for this model is:

Please suggest the same.
Thanks,
Regards,
Varsha

Hello can you save the model somewhere and create a repro in the playground?

Also please post the question on the forum so people will be able to help you quickly :)
http://www.html5gamedevs.com/forum/16-babylonjs/


Suggestion : add a parameter to generate a project for a specific version of openshift/kubernetes

When we execute the command operator-sdk new to create an API and operator, then the go project is created using the go dependencies defined within the upstream sdk project
I would like to suggest to add a parameter to tell the version to be used for openshift/kubernets clients 3.10 to 3.11 to be populated within the gopkg.toml file
e.g
operator-sdk new spring-boot-operator --version=3.11 --api-version=springboot.snowdrop.me/v1alpha1 --kind=SpringBoot

For 3.11, then we should get such versions
[[override]]
  name = "k8s.io/code-generator"
  # revision for tag "kubernetes-1.11.2"
  revision = "6702109cc68eb6fe6350b83e14407c8d7309fd1a"

[[override]]
  name = "k8s.io/api"
  # revision for tag "kubernetes-1.11.2"
  revision = "2d6f90ab1293a1fb871cf149423ebb72aa7423aa"

[[override]]
  name = "k8s.io/apiextensions-apiserver"
  # revision for tag "kubernetes-1.11.2"
  revision = "408db4a50408e2149acbd657bceb2480c13cb0a4"

[[override]]
  name = "k8s.io/apimachinery"
  # revision for tag "kubernetes-1.11.2"
  revision = "103fd098999dc9c0c88536f5c9ad2e5da39373ae"

[[override]]
  name = "k8s.io/client-go"
  # revision for tag "kubernetes-1.11.2"
  revision = "1f13a808da65775f22cbf47862c4e5898d8f4ca1"

[[override]]
  name="github.com/openshift/client-go"
  branch="release-3.11"

@cmoulliard the idea is sound but it is not simple to just generate a gopkg.toml that works out of box. Wha't the mapping between 3.11 to the version tag/ revision of the k8s.io/code-generator, k8s.io/apimachinery, and etc. if we do figure out how to do that, can operator-sdk guarantee that that k8s version works with operator-sdk api.  For now, the default  gopkg.toml is guarantee to work because we run e2e test for it. Hence, I am not in favor to add the --version flag because I don't think the operator-sdk can make any guarantee.

I agree with @fanminshi, I don't think that it is a trivial change.
I think that this might be something interesting to add to the release doc. Is this something that we should consider as apart of the release? Should instead we let the X.X.X version of the binary only works with Y.Y.Y version of kubernetes.
I don't think we should have any specific logic relating to openshift IMO.

If you provide such a table within the README.md page mentioning which SDK version has been tested according to openshift (okd, ocp) version x.y.z, that will be already a great plus
Something like this : https://github.com/openshift/kubernetes-client-go#compatibility-matrix


Get the main tech stack used for each repository

Proposed Method: get_techstack_repo()
Description
with the help of get_techstack_repo(), one can easily know main tech stack used for each repository of given user, in which it helps to pick repositories to contribute based on Tech Stack we are interested in.
To what class it belongs?: Users
Example Usage
from scrape_up import github
user =  github.Users(username="PalaVenkiReddy")

# Call the techstack function
print(user.get_techstack_repo())

# Output - {repo_name : tech_stack_used , . . . . . . . }

Hey @PalaVenkiReddy, we will include this in the Repo class, not in the users class.


Kiota does not support array of errors

Let's say your OpenAPI operation error looks like that:
"404" : {
  "description" : "Status Code 404",
  "content" : {
    "application/json" : {
      "schema" : {
        "type" : "array",
        "items" : {
          "$ref" : "#/components/schemas/RestApiError"
        }
      }
    }
  }
}

And the RestApiError schema looks like that:
"RestApiError" : {
  "type" : "object",
  "properties" : {
    "errorCode" : {
      "type" : "string"
    },
    "message" : {
      "type" : "string"
    }
  }
}

The generated RestApiError class (which inherits from ApiException) can't be deserialized properly.
Here's a sample (actual) 404 error response:
[{"errorCode":"NOT_FOUND","message":"Some error message"}]

When thrown, the ErrorCode is null and the Message is Exception of type 'MySdk.Models.RestApiError' was thrown.
I'm not sure how Kiota should handle this situation where the error response is an array instead of an object and how the error class should be generated but currently the useful information (errorCode + message) is lost.
I was able to see the problem in the AssignFieldValues method of Microsoft.Kiota.Serialization.Json.JsonParseNode (which seems to be source generated)
private void AssignFieldValues<T>(T item) where T : IParsable
{
    if(_jsonNode.ValueKind != JsonValueKind.Object) return; // 👈 _jsonNode.ValueKind == JsonValueKind.Array ⇒ no deserialization occurs at all
    IDictionary<string, object>? itemAdditionalData = null;
    if(item is IAdditionalDataHolder holder)

It turns out that even though there could be several errors, the API only ever returns a single error in the array. So I was able to workaround this limitation by tweaking the CreateFromDiscriminatorValue method after the code is generated by kiota.
Generated code:
public static RestApiError CreateFromDiscriminatorValue(IParseNode parseNode) {
    _ = parseNode ?? throw new ArgumentNullException(nameof(parseNode));
    return new RestApiError();
}

Tweaked code:
public static RestApiError CreateFromDiscriminatorValue(IParseNode parseNode) {
    _ = parseNode ?? throw new ArgumentNullException(nameof(parseNode));
    var errors = parseNode.GetCollectionOfObjectValues<RestApiError>(_ => new RestApiError());
    return errors.FirstOrDefault() ?? new RestApiError();
}

I also added an override to the Message property in order to have a meaningful exception.
public override string Message => MessageEscaped ?? base.Message;

If the API ever returns multiple errors then at least I'll get the first error, which is already better than no error at all.


fix: Hyper window freezes when I try to close it

I found the windows freezes because of node-pty can not be killed, and the new version 0.8.1 of node-pty fix it. and I also add a config useConpty. the related issues #3736.
another related issues #3758 happened because of the warmed-up session exit(I don't know why). If set config useConpty to false, it will open correctly according to the test on my windows 10  version 1903.

Thank you for your help @ivanwonder on these issues!
Multiple things are mixed in this PR. And I want to be sure that they are related.
Upgrading node-pty is a really good catch. Isn't it enough to fix #3736 ? Or do we need your additions about initialSession too?
Why do you need useConpty config? Can you explain the issue your encountered without it?

I cannot say it is enough to fix #3736.
According to the error info of #3736. the problem is the code fire event of session exit without uid.
https://github.com/zeit/hyper/blob/758e93c21e98556dd50acd81e8dc27615eebe787/lib/index.js#L61
https://github.com/zeit/hyper/blob/758e93c21e98556dd50acd81e8dc27615eebe787/app/ui/window.js#L130
https://github.com/zeit/hyper/blob/758e93c21e98556dd50acd81e8dc27615eebe787/app/ui/window.js#L136
so my thought of the additions about initialSession is clean it and creates a new one.
https://github.com/zeit/hyper/blob/758e93c21e98556dd50acd81e8dc27615eebe787/app/ui/window.js#L146
but the new one still exit. so the main problem is why session exit when open hyper with the hyper cli.
I discovered that if set the useConpty to false, the issues #3736 solved. But I still cannot explain the main problem why session exit and find info about it until now. and the vscode have the setting Windows Enable Conpty.

I think many other windows user think is annoying. Do you think it's worth for a new release? I caught this error too.

It is, yeah. But we don't have anyone doing releases currently.


[WIP] - manager as proxy for controllers

This PR

Adds persistence and HA to stateless controllers such as ingress and enrollment controllers.

Signed-off-by: David Chung<EMAIL_ADDRESS>

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "stack-api-2" git@github.com:chungers/infrakit.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354497248
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.


Provide encryption key for DATA volume as well, to copy/migrate volume.

Description
This PR provides encryption key (passkey) for DATA volume as well, to copy/migrate volume, as the ScaleIO/PowerFlex plugin fails to migrate encrypted DATA Volume between clusters with below error:
Unable to convert/copy from <REDACTED> to <REDACTED>, due to: qemu-img: Could not open 'file.filename=/dev/disk/by-id/emc-vol-<REDACTED>': Parameter 'encrypt.key-secret' is required for cipher].
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolume(VolumeOrchestrator.java:1382)
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.storageMigration(VolumeOrchestrator.java:1493)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI
[ ] test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[x] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

How did you try to break this feature and the system with this change?

@blueorangutan package

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✖ el8 ✖ el9 ✖ debian ✖ suse15. SL-JID 10908

@blueorangutan package

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 10912

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[SF] Trillian test result (tid-11292)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 45664 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9622-t11292-kvm-ol8.zip
Smoke tests completed. 132 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

test_01_secure_vm_migration
Error
134.40
test_vm_life_cycle.py

test_01_secure_vm_migration
Error
134.40
test_vm_life_cycle.py


Start testing peer-calls

Germany , Leipzig 12.03.2020 07:36
Hello,
we will today start the testing of peer-calls.
After the test, i will get an information about the results.
Greetings Stephan

Thanks for letting me know. I'm curious to hear the results.
In case you experience connectivity issues, you might need to install and
configure a TURN server. I use coturn and can share the configuration files
privately if you need them.
On Thu, Mar 12, 2020, 07:36 Stephan Krauß<EMAIL_ADDRESS>wrote:

Germany , Leipzig 12.03.2020 07:36
Hello,
we will today start the testing of peer-calls.
After the test, i will get an information about the results.
Greetings Stephan
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/jeremija/peer-calls/issues/72, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALLUVJ4RUUL5D7IO6LQYRDRHB7G7ANCNFSM4LGFLM5Q
.

Hello,
we did a first test. This test was carried out on our intranet.
It was not possible to make a connection. The error message appeared in the Firefox web tool: 'TypeError: navigator.mediaDevices is undefined'
Sincerely, Stephan

Hello,
is there a public demo ?
Your sincerly Stephan

Which version of Firefox are you using? My guess is that it is not an up-to-date version.
Public demo is available here: https://peercalls.com

Also it could be this (from MDN):

Encryption based security
The getUserMedia() method is only available in secure contexts. A secure context is one the browser is reasonably confident contains a document which was loaded securely, using HTTPS/TLS, and has limited exposure to insecure contexts. If a document isn't loaded in a secure context, the navigator.mediaDevices property is undefined, making access to getUserMedia() impossible.

Are you using using HTTPS?

Hello, we're testing again tonight.
Our intranet has too many restrictions.
We want to test with the URL :: https://peercalls.com.
Sincerely yours
Stephan


Skip inactivIsh states on stack.stopServices

@ibuildthecloud

https://github.com/rancher/rancher/issues/3944


Problem with building docs using sphinx

Details
Building the docs does not work. It worked before, but at some point it stopped working without changing anything. Also, I can build the docs localy using sphinx.
The problem exists since the build #  3768112 (~3 Months ago)
It does not print a clear error message, but the fourth step fails with:
pickling environment... done
checking consistency... done
preparing documents...

After that there is an endless long Traceback which ends with:
RuntimeError: maximum recursion depth exceeded in cmp

Finaly it does print a 'notification' that it created a full log wich should be submited to the devs of sphinx:
Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/side/envs/latest/local/lib/python2.7/site-packages/sphinx/environment.py", line 1965, in traverse_toctree
    traversed.add(docname)
RuntimeError: maximum recursion depth exceeded in cmp
The full traceback has been saved in /tmp/sphinx-err-xmB9lc.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

I hope that this is enough information for you...
Thanks in advance ;)
If this issue is for a specific project or build, include the following:

Project: SiDE
Build #: 3981536

I have exactly the same problem here :
py3njection build #4013767 - failed

Does pinning a newer Sphinx release resolve this issue? 1.3.6 or 1.4.1 are good targets

How can I change the sphinx Version?

@mame98 try editing your requirements.txt to include:
Sphinx==1.3.6

It still fails with the same error but in the log it says Running Sphinx v1.3.6 so 1.3.6 does not fix the problem...

Same error here: https://readthedocs.org/projects/opty/builds/5513070/
Seems conda pins Sphinx to 1.3.5

Hi! Can you confirm that you still have this issue and provide your project/version/build urls?
Conda shouldn't be pinning packages anymore after https://github.com/rtfd/readthedocs.org/issues/2714

Closing due to inactivity.


[Issue #30] (Team3) importing re2j package and its wrapper classes

Wrapper classes are placed under com.google.re2j package, in textdb-dataflow folder
PublicParser, PublicRE2, PublicSimplify do not inherit from corresponding classes in re2j, because what we need (for now) are all static methods.
PublicRegexp needs careful review. The original class, Regexp, represents the parse tree of a regex. Each Regexp object has an array of sub-regexps, so I decide to write a deep-copy function to convert every Regexp object to PublicRegexp Object.

A major problem for this PR is that it lacks enough comments to explain what this package does, what we want to "import," and why we make those changes.  Examples are also lacking.  Please add them first before we can do a careful review.

@chenlica I have added comments explaining our wrapper classes

@sandeepreddy602 : do you want to have a review pass before I can do it (possibly later tonight)?

@zuozhi Please let me know after you take care of the comments of @sandeepreddy602 .  After that I can do a review pass.

@zuozhi I had a quick pass.  After you take care of our comments, I will do another round.

@chenlica I made some changes based on comments. Please do another round of review.

I had one more review pass.  I gave some high-level comments on your style of writing comments, which can be improved (e.g., write complete sentences, use upper case, and use articles properly).
Also I believe we need test cases for these classes.

I'll add test cases, modify comments, and refine Javadoc formatting. I expect to finish it by tomorrow afternoon.

@chenlica I have finished test cases and improved Javadoc. Please take a look.

@sandeepreddy602 @rajesh9625 and @inkudo can any of you do a careful review first?

I had a review pass.  It looks much better now!  Please take care of those comments, which are minor.  Then we will be done!

@chenlica I've taken care of them. Can I merge this PR?
Next PR will be a basic skeleton of our translator. I expect to open next PR by tomorrow night.

This PR looks good.  Please go ahead to do the merge.


bump bower version

I had an issue with quick-ng-repeat only to find you've fixed it upstream  (which is great well done) however bower won't update because you never update the version number.
Edit: I noticed the bower version is on a different account???
Pretty please!

We now have a bower instructions in readme. Have you noticed?

thanks, no I did not as I originally installed the one from the old account, thanks will point at that instead.


Add Deprication Notice to Documentation pages

If someone goes directly there they won't see this note:
https://github.com/Esri/node-arcgis/commit/8b7796c6fa66cf7dba0735a9a58505a700ac43be#commitcomment-29724642

https://github.com/Esri/node-arcgis/blob/ab62e06688836e0118ffb3d43ec6b62ef9c2a87a/doc/_layout.html#L33-L36


Add a .ci-operator.yaml file

CC @jsafrane

/lgtm

/label qe-approved
/label px-approved
/label docs-approved
This is a purely build-related PR.

/assign @gabemontero
for /approve :)
This is needed for https://github.com/openshift/release/pull/35872

/approve

/cherry-pick release-4.13


Add Getters for device's peripheral

@urholaukkarinen
Please validate the code apply_character_filter at line 503 in file scanner.rs


Adds support for repository to be an output, Adding in test for #133

This adds in a test for #133, with image arguments. It also sets the chart version to match the image version.

I pinned google-protobuf because of this issue: https://github.com/pulumi/pulumi/issues/17421


PDN m4 stripes run through macro (if several macros are placed)

Description
When integrating several hardened macros into caravel then the m4 vertical PDN straps run through some of the macros.
Expected Behavior
The m4 straps are cut where there is a hardened macro.
Environment report
Traceback (most recent call last):
  File "./env.py", line 265, in <module>
    main()
  File "./env.py", line 261, in main
    commands[args[0]]()
  File "./env.py", line 168, in issue_survey
    % get_tag()
  File "/foss/tools/openlane/2022.11/dependencies/get_tag.py", line 67, in get_tag
    raise e
  File "/foss/tools/openlane/2022.11/dependencies/get_tag.py", line 47, in get_tag
    raise NoGitException(
dependencies.get_tag.NoGitException: Cannot determine git branch. Please specify OPENLANE_IMAGE_NAME manually.
Full output: fatal: detected dubious ownership in repository at '/foss/tools/openlane/2022.11'
To add an exception for this directory, call:

	git config --global --add safe.directory /foss/tools/openlane/2022.11

Reproduction material
I am using OpenLane 2022.11.25 and its dependencies.
Relevant log output
The error only pops up during the LVS stage, when there is an LVS mismatch displayed. Reviewing the resulting GDS shows the error of the `m4` PDN straps causing shorts.

This is how it looks like.

No. The OL flow apparently only puts the OBS in where there are tracks, and not across the full macro. See here the relevant section of tempsense.lef (that is the smaller macro placed at the corners):
 OBS
      LAYER li1 ;
        RECT 5.520 10.795 144.440 138.805 ;
      LAYER met1 ;
        RECT 5.520 10.640 144.440 138.960 ;
      LAYER met2 ;
        RECT 7.450 4.280 142.500 141.965 ;
        RECT 7.450 4.000 18.670 4.280 ;
        RECT 19.510 4.000 55.930 4.280 ;
        RECT 56.770 4.000 93.190 4.280 ;
        RECT 94.030 4.000 130.450 4.280 ;
        RECT 131.290 4.000 142.500 4.280 ;
      LAYER met3 ;
        RECT 4.000 141.080 145.600 141.945 ;
        RECT 4.000 136.360 146.000 141.080 ;
        RECT 4.400 134.960 146.000 136.360 ;
        RECT 4.000 130.240 146.000 134.960 ;
        RECT 4.000 128.840 145.600 130.240 ;
        RECT 4.000 118.000 146.000 128.840 ;
        RECT 4.000 116.600 145.600 118.000 ;
        RECT 4.000 111.880 146.000 116.600 ;
        RECT 4.400 110.480 146.000 111.880 ;
        RECT 4.000 105.760 146.000 110.480 ;
        RECT 4.000 104.360 145.600 105.760 ;
        RECT 4.000 93.520 146.000 104.360 ;
        RECT 4.000 92.120 145.600 93.520 ;
        RECT 4.000 87.400 146.000 92.120 ;
        RECT 4.400 86.000 146.000 87.400 ;
        RECT 4.000 81.280 146.000 86.000 ;
        RECT 4.000 79.880 145.600 81.280 ;
        RECT 4.000 69.040 146.000 79.880 ;
        RECT 4.000 67.640 145.600 69.040 ;
        RECT 4.000 62.920 146.000 67.640 ;
        RECT 4.400 61.520 146.000 62.920 ;
        RECT 4.000 56.800 146.000 61.520 ;
        RECT 4.000 55.400 145.600 56.800 ;
        RECT 4.000 44.560 146.000 55.400 ;
        RECT 4.000 43.160 145.600 44.560 ;
        RECT 4.000 38.440 146.000 43.160 ;
        RECT 4.400 37.040 146.000 38.440 ;
        RECT 4.000 32.320 146.000 37.040 ;
        RECT 4.000 30.920 145.600 32.320 ;
        RECT 4.000 20.080 146.000 30.920 ;
        RECT 4.000 18.680 145.600 20.080 ;
        RECT 4.000 13.960 146.000 18.680 ;
        RECT 4.400 12.560 146.000 13.960 ;
        RECT 4.000 7.840 146.000 12.560 ;
        RECT 4.000 6.975 145.600 7.840 ;
  END

If there is no OBS then pdngen is free to put stripes across your macro.
I'm not sure what you mean by "where there are tracks".  Wouldn't you have routing tracks on metal4?  Or do you mean actual routing as opposed to tracks?

Take an example from the other small macro config_reg_mux.lef: There is one small routing on m4, and this gets an OBS. If the intention is that a macro should get an OBS across the whole macro (as is done on li1 and met1) then something needs to be changed. I am using stock OL.
 OBS
      LAYER li1 ;
        RECT 5.520 10.795 294.400 288.405 ;
      LAYER met1 ;
        RECT 3.290 8.880 298.010 289.640 ;
      LAYER met2 ;
        RECT 3.320 295.720 9.010 296.000 ;
        RECT 9.850 295.720 14.990 296.000 ;
        RECT 15.830 295.720 20.970 296.000 ;
        <snip>
  LAYER met3 ;
        RECT 4.000 294.080 295.600 294.945 ;
        RECT 4.000 291.400 296.000 294.080 ;
        RECT 4.000 290.000 295.600 291.400 ;
        <snip>
    LAYER met4 ;
        RECT 283.655 18.535 284.905 200.425 ;

I'm not clear why it is a problem that the rail crosses your block if you aren't using m4.

Good question, so I checked, here the macro tempsense: It has m4 in the local PDN, but nothing on OBS. So this causes shorts with the PDN above:

So here is tempsense.lef in klayout, here the layers are better to see: There are tracks on met4, but no OBS:

On a larger macro, the met4.OBS is put in (same OpenLane, similar config.json). Can it be that there is an issue on smaller macros?

Here is the config.json of the larger macro where met4.OBS is correctly in the LEF:
{
    "DESIGN_NAME": "audiodac",
    "DESIGN_IS_CORE": 0,
    "VERILOG_FILES": ["dir::../../verilog/rtl/defines.v", "dir::../../verilog/rtl/audiodac.v", "dir::../../verilog/rtl/audiodac_dsmod.v", "dir::../../verilog/rtl/audiodac_fifo.v", "dir::../../verilog/rtl/audiodac_sinegen.v"],
    "CLOCK_PERIOD": 50,
    "CLOCK_PORT": "clk_i",
    "CLOCK_NET": "clk_i",
    "FP_SIZING": "relative",
    "ROUTING_CORES": 8,
    "FP_PIN_ORDER_CFG": "dir::pin_order.cfg",
    "PL_BASIC_PLACEMENT": 0,
    "PL_TARGET_DENSITY": 0.30,
    "VDD_NETS": ["vccd1"],
    "GND_NETS": ["vssd1"],
    "DIODE_INSERTION_STRATEGY": 4,
    "RUN_CVC": 1,
    "pdk::sky130*": {
        "FP_CORE_UTIL": 25,
        "RT_MAX_LAYER": "met4",
        "scl::sky130_fd_sc_hd": {
            "CLOCK_PERIOD": 50
        },
        "scl::sky130_fd_sc_hdll": {
            "CLOCK_PERIOD": 50
        },
        "scl::sky130_fd_sc_hs": {
            "CLOCK_PERIOD": 50
        },
        "scl::sky130_fd_sc_ls": {
            "CLOCK_PERIOD": 50,
            "SYNTH_MAX_FANOUT": 5
        },
        "scl::sky130_fd_sc_ms": {
            "CLOCK_PERIOD": 50
        }
    },
    "pdk::gf180mcuC": {
        "STD_CELL_LIBRARY": "gf180mcu_fd_sc_mcu7t5v0",
        "CLOCK_PERIOD": 50,
        "FP_CORE_UTIL": 25,
        "RT_MAX_LAYER": "Metal4",
        "SYNTH_MAX_FANOUT": 4,
        "PL_TARGET_DENSITY": 0.30
    }
}

Here is the config.json of the smaller macro where met4.OBS is missing in the LEF:
{
    "DESIGN_IS_CORE": 1,
    "DESIGN_NAME": "user_project_wrapper",
    "VERILOG_FILES": ["dir::../../verilog/rtl/defines.v", "dir::../../verilog/rtl/user_project_wrapper.v"],
    "CLOCK_PERIOD": 50,
    "CLOCK_PORT": "io_in[6]",
    "CLOCK_NET": ["cfg_reg0.clk_i", "temp0.clk", "adac0.clk_i"],
    "RSZ_DONT_TOUCH_RX": ["analog"],
    "FP_PDN_MACRO_HOOKS": ["cfg_reg0 vccd1 vssd1", "temp0 vccd1 vssd1", "adac0 vccd1 vssd1"],
    "MACRO_PLACEMENT_CFG": "dir::macro.cfg",
    "VERILOG_FILES_BLACKBOX": ["dir::../../verilog/rtl/defines.v", "dir::../../verilog/rtl/config_reg_mux.v", "dir::../../verilog/rtl/audiodac.v", "dir::../../verilog/rtl/tempsense.v"],
    "EXTRA_LEFS": ["dir::../../lef/config_reg_mux.lef", "dir::../../lef/audiodac.lef", "dir::../../lef/tempsense.lef"],
    "EXTRA_GDS_FILES": ["dir::../../gds/config_reg_mux.gds", "dir::../../gds/audiodac.gds", "dir::../../gds/tempsense.gds"],
    "FP_PDN_CHECK_NODES": 0,
    "SYNTH_ELABORATE_ONLY": 1,
    "PL_RANDOM_GLB_PLACEMENT": 1,
    "PL_RESIZER_DESIGN_OPTIMIZATIONS": 0,
    "PL_RESIZER_TIMING_OPTIMIZATIONS": 0,
    "PL_RESIZER_BUFFER_INPUT_PORTS": 0,
    "FP_PDN_ENABLE_RAILS": 0,
    "DIODE_INSERTION_STRATEGY": 0,
    "RUN_FILL_INSERTION": 0,
    "RUN_TAP_DECAP_INSERTION": 0,
    "FP_PDN_VPITCH": 180,
    "FP_PDN_HPITCH": 180,
    "CLOCK_TREE_SYNTH": 0,
    "FP_PDN_VOFFSET": 5,
    "FP_PDN_HOFFSET": 5,
    "MAGIC_ZEROIZE_ORIGIN": 0,
    "FP_SIZING": "absolute",
    "RUN_CVC": 0,
    "ROUTING_CORES": 8,
    "UNIT": "2.4",
    "FP_IO_VEXTEND": "expr::2 * $UNIT",
    "FP_IO_HEXTEND": "expr::2 * $UNIT",
    "FP_IO_VLENGTH": "ref::$UNIT",
    "FP_IO_HLENGTH": "ref::$UNIT",
    "FP_IO_VTHICKNESS_MULT": 4,
    "FP_IO_HTHICKNESS_MULT": 4,
    "FP_PDN_CORE_RING": 1,
    "FP_PDN_CORE_RING_VWIDTH": 3.1,
    "FP_PDN_CORE_RING_HWIDTH": 3.1,
    "FP_PDN_CORE_RING_VOFFSET": 12.45,
    "FP_PDN_CORE_RING_HOFFSET": 12.45,
    "FP_PDN_CORE_RING_VSPACING": 1.7,
    "FP_PDN_CORE_RING_HSPACING": 1.7,
    "FP_PDN_VWIDTH": 3.1,
    "FP_PDN_HWIDTH": 3.1,
    "FP_PDN_VSPACING": "expr::(5 * $FP_PDN_CORE_RING_VWIDTH)",
    "FP_PDN_HSPACING": "expr::(5 * $FP_PDN_CORE_RING_HWIDTH)",
    "VDD_NETS": ["vccd1 vccd2 vdda1 vdda2"],
    "GND_NETS": ["vssd1 vssd2 vssa1 vssa2"],
    "SYNTH_USE_PG_PINS_DEFINES": "USE_POWER_PINS",
    "pdk::sky130*": {
        "RT_MAX_LAYER": "met4",
        "DIE_AREA": "0 0 2920 3520",
        "FP_DEF_TEMPLATE": "dir::fixed_dont_change/user_project_wrapper.def",
        "scl::sky130_fd_sc_hd": {
            "CLOCK_PERIOD": 10
        },
        "scl::sky130_fd_sc_hdll": {
            "CLOCK_PERIOD": 10
        },
        "scl::sky130_fd_sc_hs": {
            "CLOCK_PERIOD": 8
        },
        "scl::sky130_fd_sc_ls": {
            "CLOCK_PERIOD": 10,
            "SYNTH_MAX_FANOUT": 5
        },
        "scl::sky130_fd_sc_ms": {
            "CLOCK_PERIOD": 10
        }
     },
    "pdk::gf180mcuC": {
        "STD_CELL_LIBRARY": "gf180mcu_fd_sc_mcu7t5v0",
        "FP_PDN_CHECK_NODES": 0,
        "FP_PDN_ENABLE_RAILS": 0,
        "RT_MAX_LAYER": "Metal4",
        "DIE_AREA": "0 0 3000 3000",
        "FP_DEF_TEMPLATE": "dir::fixed_dont_change/user_project_wrapper_gf180mcu.def",
        "PL_OPENPHYSYN_OPTIMIZATIONS": 0,
        "DIODE_INSERTION_STRATEGY": 0,
        "FP_PDN_CHECK_NODES": 0,
        "MAGIC_WRITE_FULL_LEF": 0,
        "FP_PDN_ENABLE_RAILS": 0
   }
}

So the problem here is not with pdngen but with whatever is generating your LEF.  Is that OR or magic?

Unfortunately, I don't know. It pops out of the OL flow at the end.

A related question: So pdngen expects an OBS on met4, but why is it not taking any met4 structure in the LEF as an obstruction. Like merging met4 and met4.OBS into met.OBS to avoid routing through it.

Do you have non-OBS met4 in your LEF?

Yes, see example above.

You example previously only showed on shape
LAYER met4 ;
        RECT 283.655 18.535 284.905 200.425 ;

while your image shows many.  It is getting hard to follow with multiple blocks and snippets.

Yeah, sorry for the confusion! In summary:

There are structures on met4.PIN but not on met4.OBS in small blocks.
In larger blocks, there is a large structure on met4.OBS.

@arlpetergadfort pin shapes should be avoided by pdngen, correct?

So the problem here is not with pdngen but with whatever is generating your LEF. Is that OR or magic?

Openlane uses magic to create the LEF. By default it creates an abstract LEF, but you can use MAGIC_WRITE_FULL_LEF to disable this.
With the recent updates to the Openroad LEF generator, perhaps we should switch Openlane to it.

@antonblanchard do you know why the results differ for various blocks?

@antonblanchard do you know why the results differ for various blocks?

Could be a magic bug. @hpretl could you please attach def files for the two examples above (met4 OBS and no OBS)?

@arlpetergadfort pin shapes should be avoided by pdngen, correct?

@maliberty yes it should. Unless it's connected to a net pdngen is trying to connect to.

I manually copied the met1 OBS to met4 in the blocks config_reg_mux and tempsense, and then everything is good (proper PDN generation and connection, and LVS passes).
So the issue is either:

The pdngen should consider met4 PIN together with the met4 OBS to stay clear of these with the power grid.
Or the LEF generation should put proper met4 OBS into the LEF (like copying pin and routes into the OBS section).

What of this two is the root cause and the appropriate solution is beyond my means.

@antonblanchard Here you go, all the LEFs in questions:
ol_1522.tar.gz

Thanks, but we need the defs so we can recreate the lefs

Sorry, my bad! Here are the DEFs:
ol_1522_def.tar.gz

@arlpetergadfort pin shapes should be avoided by pdngen, correct?

I believe this is true too. It is not a lef generation issue as areas with no met4.drawing in it will not generate OBS. @hpretl If this is still happening can you provide the design files to recreate the short ?


clean command

Adds a spacchetti clean command that nukes the .psc-package folder.
I also added a clearer message in case the user runs insdhall without a packages.dhall file.

thx


transactions() relation includes transactions that are not accepted

Should $user->transactions() default to only include transactions that are accepted?
Similar to how Eloquent filters out soft deleted models, but you can use the withTrashed() to include ones that are deleted.
Maybe $user->transactions()->withRejected()->get() if you wanted every transaction or something?

In all of my current use cases I'm always showing the list of all transactions. It's important for the clients & the admins to see the failed transactions as well otherwise it could confuse them.

That's cool.
I added a method to calculate a running balance next to each transaction, which is wrong due to including failed transactions but I will add a where clause to the transactions I am getting :+1:


Stuck at the rainbow splash screen on RPi2

Followed the instructions to the letter -- but stuck at the rainbow splash screen. Looks like
the kernel firmware.img has some problem...
I am building on the Ubuntu sub-system on Windows 10.
I see no error or warnings except this:
source/device/hid/keyboard.c: In function 'KeyboardAttach':
source/device/hid/keyboard.c:163:7: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
if (parse->Report[i]->Fields[j].Usage.Keyboard >= KeyboardLeftControl
^~
source/device/hid/keyboard.c:166:8: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the 'if'
data->KeyFields[(u16)parse->Report[i]->Fields[j].Usage.Keyboard - (u16)KeyboardLeftControl] =
^~~~
LINK build/firmware.elf

@hacksterous thank you for reporting this, I can't state where the problem is for now.
Could you try the precompiled binary here:
http://blog.boochow.com/wp-content/uploads/mp-rpi2-20181228-uskbd.zip
Please add bootcode.bin and start.elf. They are not inclueded in the above zip file.
The related blog post is here (I'm sorry it is in Japanese... a url of google translated version follows).
https://blog.boochow.com/article/mp-rpi2-test-release.html
https://translate.google.co.jp/translate?hl=ja&sl=ja&tl=en&u=https%3A%2F%2Fblog.boochow.com%2Farticle%2Fmp-rpi2-test-release.html

Thanks, that worked!
The USB keyboard does not work -- I see USB keyboard errors still, but anyway that's a good start.
Thanks.


Update number.py

For numbers that are percentages, set the corresponding unit (%).

Thanks for the PR. It took a while...


use service names

switch to service names to make use of go micro registry

with the 3.1.0 AFAICT


Add kube config and kube cluster to arc data controller screens

Description:
This change adds two fields:

Kube Config File Path - an File picker that opens a File Explorer/Finder-like window to allow the user to pick a file. The user can also enter the path directly into the corresponding text input field.
Cluster Context - a radio options selector the allows users to pick the cluster context within that config file. This cluster is the one that is used in all future interactions with the controller being registered.

This change also creates a FilePicker class to enable the file picker functionality mentioned above.
Changed Screen Snapshot:

Tests:

Added new tests for FilePicker
New tests for RadioGroupButton picker for properties added.
Validated all extension tests.

Work items and Design:
This PR is next in a series of changes needed for #12727
Here is the full scope of planned work. MultipleControllerSupportInAdsAzureArc

Pull Request Test Coverage Report for Build 383988494

60 of 79 (75.95%)  changed or added relevant lines in 5 files are covered.
5 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.04%) to 41.3%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/arc/src/ui/components/filePicker.ts
30
33
90.91%

extensions/arc/src/common/kubeUtils.ts
1
9
11.11%

extensions/arc/src/ui/dialogs/connectControllerDialog.ts
17
25
68.0%

Files with Coverage Reduction
New Missed Lines
%

extensions/arc/src/ui/dialogs/connectControllerDialog.ts
1
75.53%

extensions/notebook/src/jupyter/serverInstance.ts
4
73.81%

Totals

Change from base Build 383805018:
0.04%

Covered Lines:
21900

Relevant Lines:
48676

💛  - Coveralls


Updated Shreyasri258.json

Changes proposed

Added Profile Picture and another profile link.

Why do you have two files?

Hey, I deleted the file with the name data/data:shreyasri258.json and also resolved conflicts can you please check it out.


Cannot be opened: didn't find classname for 'isa' key

Project /Users/ronysteel/Desktop/music/MusicPlayer/NewProject.xcodeproj cannot be opened: didn't find classname for 'isa' key

same issue


No python application found

I get this error in the logs. And a Internal Server Error when i go to the HTTP server.

I've tried launching it as a docker container, but when I try and restart the cardinal service it throws "Failed to connect to bus: No such file or directory".

Hey @Lawrence-Hall :wave:

I've tried launching it as a docker container, but when I try and restart the cardinal service it throws "Failed to connect to bus: No such file or directory".

Are you following the documentation in webapp/docker? If so, I moved the container implementation over to a new repository (https://github.com/cardinal-dev/cardinal-docker). cardinal-docker deploys Cardinal in a pod-like setup. cardinal-docker also allows for faster deployment.

Hey @Lawrence-Hall, just following up on your issue report. Did you have the chance to try out cardinal-docker?

Hey @Lawrence-Hall :wave:  I'm going to close this issue out for now. If you have any other questions or difficulties with Cardinal, please feel free to open another issue.


Database provider SQlLite hard-coded

Hi,
it would be nice if the database provider / connection string is read from a configuration file and not hard-code to SqlLite (in Startup and ConduitContext).)
I can imagine that someone would like to use an InMemoryDatabase for testing.
Greetings,
Sergiu

Cool idea, I will try this and submit a PR.

Fixed


❓ Please take the vscode-cmake-tools GitHub 2023 survey

Please tell us about your experience with this GitHub repo (and up to 2 others). It should only take a few minutes.
https://www.surveymonkey.com/r/8KTX9VT?Source=RepoLink

The survey is now closed.


add validators in vmware

If the issue is to do with Azure CLI 2.0 in-particular, create an issue here at Azure/azure-cli

Extension name (the extension in question)
vmware
Description of issue (in as much detail as possible)
Add validators to vmware where it makes sense. See https://github.com/Azure/azure-cli-extensions/pull/2998#discussion_r575158720

follow up from PR reviewing


How to install your own certificates for :6443

k3os [~]$ k3os --version
k3os version v0.11.0
k3os [~]$ uname --kernel-release --kernel-version
5.4.0-37-generic #41 SMP Fri Jun 19 23:11:47 UTC 2020
k3os [~]$ uname --machine
x86_64
Describe the bug
I have my own certificates for many of stuff, managed outside the k3os.
I want to use those as k3os:6443
How can i do that? I couldn't find any docs nor any issues in the github (or i've searched badly).
To Reproduce
point your browser to k3os:6443 and you get the error
Expected behavior
The k3os can use the certificate provided, so web browsers will not "whine".
Actual behavior
Web browsers says that the certificate is not secure.
I have my own CA, internally. All of the computers is having it added to the ca-certificates.
I want to use the certificate created by my CA (or even some common one) instead of self created by the k3os or k3s.

@jabbas I haven't looked into this in some time but I think the most straight-forward approach would be a write_files or run_cmd entry in config.yaml to establish the certificates on disk followed up a run_cmd to concatenate them with /etc/ssl/cert.pem

There is also https://github.com/rancher/k3os/issues/518 to be aware of (with a workaround).

So, I re-read your comment. You will want to see https://github.com/rancher/k3s/issues/1868#issuecomment-639690634

@dweomer Yup that's what i need. But this solution/hack will work for k3s. How to get it working for k3os install?
Or better - maybe it would be nice to have possibility to put base64 encoded certs into cloud.config during install?

Use write files and create the following files does indeed work.  I pre-build ami's and use this during the packer run so I have consistent API keys accross all builds.
write_files:
- encoding: ""
  content: |-
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  owner: root
  path: /var/lib/rancher/k3s/server/tls/client-ca.crt
  permissions: '0644'
- encoding: ""
  content: |-
    -----BEGIN RSA PRIVATE KEY-----
    -----END RSA PRIVATE KEY-----
  owner: root
  path: /var/lib/rancher/k3s/server/tls/client-ca.key
  permissions: '0600'
- encoding: ""
  content: |-
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  owner: root
  path: /var/lib/rancher/k3s/server/tls/request-header-ca.crt
  permissions: '0644'
- encoding: ""
  content: |-
    -----BEGIN RSA PRIVATE KEY-----
    -----END RSA PRIVATE KEY-----
  owner: root
  path: /var/lib/rancher/k3s/server/tls/request-header-ca.key
  permissions: '0600'
- encoding: ""
  content: |-
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  owner: root
  path: /var/lib/rancher/k3s/server/tls/server-ca.crt
  permissions: '0644'
- encoding: ""
  content: |-
    -----BEGIN RSA PRIVATE KEY-----
    -----END RSA PRIVATE KEY-----
  owner: root
  path: /var/lib/rancher/k3s/server/tls/server-ca.key
  permissions: '0600'

Thanks @gbarton - that will work!

Thanks @gbarton - that will work!


Automatic loading of default parameters from config directory

Provide us a use case of the feature
User creates a file under a hard-coded name (e.g. default.txt or config.txt) inside Instaloader's config directory (~/.config/instaloader/ on Linux) and enters here default parameters.
Describe the solution you'd like
Default parameters are loaded automatically.
Describe alternatives you've considered
Currently I use a file with a one-letter filename (a) inside current working directory, so I have to type +a as a part of every command.
If the feature request is accepted, would you be willing to submit a pull request?
Yes

I do not see a need for this feature.
As a convenient alternative to passing +a every call, you could define an alias in your .bashrc, like
alias instaloader='instaloader +~/.config/instaloader/default'

I do not see a need for this feature.
As a convenient alternative to passing +a every call, you could define an alias in your .bashrc, like
alias instaloader='instaloader +~/.config/instaloader/default'

Okay, that actually might be a solution. Feel free to close the issue now or later if it gets stale with no one else submitting another feedback.


Implement equality

What does the pull request do?
Add equality implementations to:

SolidColorBrush & ImmutableSolidColorBrush
RowDefinition & RowDefinitions
ColumnDefinition & ColumnDefinitions

What is the current behavior?
The types use the default (reference) equality
What is the updated/expected behavior with this PR?
The types use value equality
How was the solution implemented (if it's not obvious)?

Checklist

[x] Added unit tests (if possible)?
[x] Added XML documentation to any related classes?
[x] Consider submitting a PR to https://github.com/AvaloniaUI/Avaloniaui.net with user documentation

Breaking changes
Fixed issues

@jmacato @kekekeks could someone review this again ? :)

@JaggerJo  @jmacato we are using them in other places because the code was written before => was a thing. The general policy is not to change the formatting in existing code for the sake of changing formatting, so we keep old files with curly braces instead of =>.
It's perfectly OK to use new language features in newly written code as long as it doesn't hurt performance.

BTW, in https://github.com/AvaloniaUI/Avalonia/pull/2744 I don't use ValueTuple to create hash codes, I used the VS generator.
Do we have a preference as to which to use? Are there pros and cons or are they functionally equivalent?

BTW, in #2744 I don't use ValueTuple to create hash codes, I used the VS generator.
Do we have a preference as to which to use? Are there pros and cons or are they functionally equivalent?

Personally I like to use ValueTuple to compute a hash code. Microsoft also does is this way in their examples
public override int GetHashCode()
{
return Tuple.Create(re, im).GetHashCode();
}
https://docs.microsoft.com/en-us/dotnet/api/system.object.equals?view=netframework-4.8

BTW: I'm really impressed by the code reviews 👍

Tuple.Create(re, im) allocates an object on the heap, don't do that.

@kekekeks I wish we could codify that somewhere, I was under the impression that we enforce code guidelines retroactively repo-wide.

Tuple.Create(re, im) allocates an object on the heap, don't do that.

Sorry, I meant ValueTuple which lives on the stack.

@grokys Let me know if there is still something wrong.. Sorry for the wasted time, my git config was corrupt and the changes were not pushed..

Are we sure that removing GetHashCode on mutable classes is the correct thing to do? Indeed, the compiler gives a warning now:
CS0659 'SolidColorBrush' overrides Object.Equals(object o) but does not override Object.GetHashCode()

Where did you get your advice that "GetHashCode should be only implemented on immutable types / where the hash cant change"? All C# documentation that I've found says it must be overridden (e.g. https://docs.microsoft.com/en-us/dotnet/api/system.object.gethashcode?view=netframework-4.8)

Are we sure that removing GetHashCode on mutable classes is the correct thing to do? Indeed, the compiler gives a warning now:
CS0659 'SolidColorBrush' overrides Object.Equals(object o) but does not override Object.GetHashCode()
Where did you get your advice that "GetHashCode should be only implemented on immutable types / where the hash cant change"? All C# documentation that I've found says it must be overridden (e.g. https://docs.microsoft.com/en-us/dotnet/api/system.object.gethashcode?view=netframework-4.8)

An important caveat - you should only override GetHashCode if your objects are immutable.
If the properties of your Foo class can change and change the GetHashCode result, any collection (List, etc…) will behave unpredictably or throw an exception if the hashcode of an individual Foo instance in the collection changes.
http://www.aaronstannard.com/overriding-equality-in-dotnet/
But in the official docs they mention:
You can override GetHashCode() for immutable reference types. In general, for mutable reference types, you should override GetHashCode() only if:

You can compute the hash code from fields that are not mutable; or
You can ensure that the hash code of a mutable object does not change while the object is contained in a collection that relies on its hash code.

Yeah, makes sense. We should probably suppress CS0659 then.

I think it must be added on all classes except ImmutableSolidColorBrush

@grokys maybe we could ignore the CS0659 warning on a per project basis ?
Because according to specification its not that common that you actually should override it.

Overriding object.Equals might break Dictionary<IBrush, TValue>. Two separate instances of SolidColorBrush will return true for object.Equals and will be treated as the same object while not actually being one.
I think we should use separate naming, like Equivalent instead of Equals.

I'd suggest just putting the #pragma once at the top of the files that we want to suppress the warnings for.
BTW, when looking through the WPF code, I notice that in WPF, DependencyObject-derived controls are prohibited from overriding Equals and GetHashCode:
https://github.com/dotnet/wpf/blob/master/src/Microsoft.DotNet.Wpf/src/WindowsBase/System/Windows/DependencyObject.cs#L148
Given that, are we sure this is a good idea?

Interesting how nearly everyone does it differently, but I get why they do it.

We need Equals on immutable brushes in order to compare immutable brushes to mutable brushes, which means we shouldn't actually need mutable brushes to implement Equals/GetHashCode - only the immutable ones.
What did you need Equals for on row/column definitions?

FuncUI diffs the last and the next values and only updates them if they changed. Because UI is a pure function from state, it creates new brushes each time the view is updated. (this is simplified, there is actually some caching and lazy evaluation going on..)
Someone reported that grid throws (cant reassign row/column definitions)
This happens because (RowDefinitions "1*,1*") != (RowDefinitions "1*,1*")
    let view (state : CounterState) (dispatch) : View =
        Views.grid [
            Attrs.rowDefinitions (RowDefinitions "1*,1*")
            Attrs.columnDefinitions (ColumnDefinitions "1*, 1*")
            Attrs.children [
               ...
            ]
        ]

@kekekeks

Overriding object.Equals might break Dictionary<IBrush, TValue>. Two separate instances of SolidColorBrush will return true for object.Equals and will be treated as the same object while not actually being one.

I think if the Dictionary depends on reference equality it should be explicitly given a IEqualityComparer that checks for reference equality. (https://stackoverflow.com/questions/8946790/how-to-use-an-objects-identity-as-key-for-dictionaryk-v)

I think we should use separate naming, like Equivalent instead of Equals.

Having a.Equals(b as object) should be consistent with 'a.Equals(b as T)' I think. It would be really confusing if there was a difference. You cant relay on Object.Equals for reference equality, so if you do rely on this I think ReferenceEquals should be used.
Am I wrong ?

I think if the Dictionary depends on reference equality it should be explicitly given a IEqualityComparer that checks for reference equality.

I think the problem with this is that people don't do this by default, and then get hard-to-diagnose bugs.

Having a.Equals(b as object) should be consistent with 'a.Equals(b as T)' I think.

Definitely. If not, this would break a load of stuff.

I think we should use separate naming, like Equivalent instead of Equals.

I think what @kekekeks is suggesting is that AvaloniaObject.Equals always does reference equality, as is enforced in WPF. We could do the same as them and seal Equals and GetHashCode.
Then we'd have another method of comparing whether two AvaloniaObjects were "equivalent": i.e. their properties are the same at this moment in time. This wouldn't affect dictionary/hashtable etc.
Thinking about this some more, you say "FuncUI diffs the last and the next values and only updates them if they changed." - what about e.g. attached properties in this case? Do they need to be taken into account?
Your ColumnDefinition.Equals method only compares 3 properties - what if an attached property is set on the ColumnDefinition? Does that make it unequal or equal to another ColumnDefinition with the same 3 property values but a different attached property value?

Yeah, I thinks so too. Bad that this warning exists.

I think if the Dictionary depends on reference equality it should be explicitly given a IEqualityComparer that checks for reference equality.
I think the problem with this is that people don't do this by default, and then get hard-to-diagnose bugs.
Having a.Equals(b as object) should be consistent with 'a.Equals(b as T)' I think.
Definitely. If not, this would break a load of stuff.
I think we should use separate naming, like Equivalent instead of Equals.
I think what @kekekeks is suggesting is that AvaloniaObject.Equals always does reference equality, as is enforced in WPF. We could do the same as them and seal Equals and GetHashCode.
Then we'd have another method of comparing whether two AvaloniaObjects were "equivalent": i.e. their properties are the same at this moment in time. This wouldn't affect dictionary/hashtable etc.
Thinking about this some more, you say "FuncUI diffs the last and the next values and only updates them if they changed." - what about e.g. attached properties in this case? Do they need to be taken into account?
Your ColumnDefinition.Equals method only compares 3 properties - what if an attached property is set on the ColumnDefinition? Does that make it unequal or equal to another ColumnDefinition with the same 3 property values but a different attached property value?

AttachedProperties are added like this and compared seperatly
    let view (state: CounterState) (dispatch): View =
        Views.dockPanel [
            Attrs.children [
                Views.button [
                    Attrs.click (fun sender args -> dispatch Increment)
                    Attrs.content "click to increment"
                    Attrs.dockPanel_dock Dock.Top
                ]
                Views.button [
                    Attrs.click (fun sender args -> dispatch Decrement)
                    Attrs.content "click to decrement"
                    Attrs.dockPanel_dock Dock.Top
                ]
                Views.textBlock [
                    Attrs.dockPanel_dock Dock.Top
                    Attrs.text (sprintf "the count is %i" state.count)
                ]
            ]
        ]

Yeah, I thinks so too. Bad that this warning exists.

Sorry, I'm lost here. Are you saying "I think so to" to agree with disallowing overriding Equals and GetHashCode like in WPF?

Another data point: UWP does not seal Equals and GetHashCode.

Yeah, I thinks so too. Bad that this warning exists.
Sorry, I'm lost here. Are you saying "I think so to" to agree with disallowing overriding Equals and GetHashCode like in WPF?

Sorry I don't actually know where this comes from. Just ignore it.
This really is harder to get right than I would have ever thought..

I think if the Dictionary depends on reference equality it should be explicitly given a IEqualityComparer that checks for reference equality.
I think the problem with this is that people don't do this by default, and then get hard-to-diagnose bugs.

Where exactly is this used ?
It's also really hard to debug if yo have to be sure all object properly implement reference equality. VS. checking one implementation that compares all elements for equality.
https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.dictionary-2?view=netframework-4.8

Where exactly is this used (I mean where are we depending on ref equality) ?

I don't know that we are doing that anywhere ourselves. But by implementing equality as such, we're potentially causing problems for users who want to do that.
This brings to mind another question: what is s special about row/column definitions that they need equality where other classes don't?

This brings to mind another question: what is s special about row/column definitions that they need equality where other classes don't?

When you define a class or struct, you decide whether it makes sense to create a custom definition of value equality (or equivalence) for the type. Typically, you implement value equality when objects of the type are expected to be added to a collection of some sort, or when their primary purpose is to store a set of fields or properties.
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/statements-expressions-operators/how-to-define-value-equality-for-a-type
One could discuss if this applies, Personally I would just assume that I can compare RowDefinitions, ... for equality.

(@JaggerJo I don't know if you saw but I made some edits to https://github.com/AvaloniaUI/Avalonia/pull/2747#issuecomment-513203597 probably while you were writing this reply)

Typically, you implement value equality when objects of the type are expected to be added to a collection of some sort, or when their primary purpose is to store a set of fields or properties.

Button can be added to Panel.Children and stores a set of properties. Why doesn't it need an Equals implementation?

One could discuss if this applies, Personally I would just assume that I can compare RowDefinitions, ... for equality.

As I mentioned in my edited comment above, this PR is overriding Equals with a very narrow definition of "equality". It may seem academic, but thinking more about this, I've seen this cause very real problems in code. For example, using ReactiveUI with your Equals implementation:
    public class TestViewModel : ReactiveObject
    {
        RowDefinition _foo;

        public RowDefinition Foo
        {
            get => _foo;
            set => this.RaiseAndSetIfChanged(ref _foo, value);
        }
    }

var viewModel = new TestViewModel();
var rowDef1 = new RowDefinition(10, GridUnitType.Auto);
var rowDef2 = new RowDefinition(10, GridUnitType.Auto);

rowDef2.SetValue(DockPanel.DockProperty, Dock.Right);

viewModel.Foo = rowDef1;
viewModel.Foo = rowDef2;

var dock = viewModel.Foo.GetValue(DockPanel.DockProperty); // is Left not Right!

Here, we assign rowDef2 to viewModel.Foo but because rowDef1.Equals(rowDef2) == true it doesn't get assigned!

@grokys Yeah, haven't seen that.
Hah, you're right. It's really complicated because they are AvaloniaObjects.
What shall we do ?

Stick to default equality on AvaloniaObjects (and seal Equals & GetHashCode) ?
Think about a proper value like comparison for AvaloniaObjects (incloding SharedSize, AttachedProperties) even thats complicated ?

I'm tempted to seal Equals and GetHashCode as WPF does with a link to this thread, so we don't go through this again ;)

I'm tempted to seal Equals and GetHashCode as WPF does with a link to this thread, so we don't go through this again ;)

Maybe the simplest solution


Remove unused fluidErrorCode from LoggingError

Description
This property hasn't been used since before 1.0, yet it shows up in the logs and can be confusing.

Thank you @alexvy86 for looking at internal telemetry, where we find there are a very small number of lingering clients on super old Loader version (pre-1.0).  I see < 200 sessions with errors from these clients in the last 30 days, and I can't imagine it will hinder any material analysis if some of these errors get mis-annotated.  I'm also following up on our official support matrix to understand if this could be an issue otherwise.

I'm awaiting confirmation that we don't need to support pre-1.0 loaders before merging.


Split travis build into multiple jobs

Our build on the master branch is occaisionally hitting the 50 minute
travis-ci job timeout. To give more headroom, we can split independent
parts of the build into multiple jobs.

Integration tests and unit tests will run in parallel jobs. Coverage
reports are supposed to be merged according to codecov docs.
(https://docs.codecov.io/docs/merging-reports)
Publishing to bintray will run after both test jobs pass on the master
branch.

Codecov Report

Merging #1643 into master will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master    #1643      +/-   ##
============================================
- Coverage     73.60%   73.59%   -0.01%
+ Complexity     8741     8683      -58
============================================
  Files           615      615
  Lines         47653    47652       -1
  Branches       6023     6023
============================================
- Hits          35076    35071       -5
+ Misses        10776    10754      -22
- Partials       1801     1827      +26

Impacted Files
Coverage Δ
Complexity Δ

...c/main/java/com/github/ambry/rest/RequestPath.java
89.15% <100.00%> (-0.13%)
33.00 <1.00> (ø)

...com/github/ambry/utils/NettyByteBufLeakHelper.java
91.42% <0.00%> (-8.58%)
5.00% <0.00%> (-3.00%)

...hub/ambry/protocol/CatchupStatusAdminResponse.java
75.00% <0.00%> (-6.25%)
6.00% <0.00%> (-1.00%)

...a/com/github/ambry/utils/ZeroBytesInputStream.java
80.95% <0.00%> (-4.77%)
8.00% <0.00%> (-1.00%)

...main/java/com/github/ambry/commons/NettyUtils.java
31.11% <0.00%> (-2.23%)
6.00% <0.00%> (-1.00%)

...hub/ambry/messageformat/ValidatingTransformer.java
89.13% <0.00%> (-2.18%)
7.00% <0.00%> (-1.00%)

...in/java/com/github/ambry/network/Transmission.java
91.07% <0.00%> (-1.79%)
21.00% <0.00%> (-1.00%)

...n/java/com/github/ambry/commons/HelixNotifier.java
96.00% <0.00%> (-1.34%)
15.00% <0.00%> (-1.00%)

...stermap/InstanceConfigToDataNodeConfigAdapter.java
91.86% <0.00%> (-1.17%)
9.00% <0.00%> (ø%)

...ithub/ambry/messageformat/BlobStoreHardDelete.java
84.42% <0.00%> (-0.82%)
2.00% <0.00%> (ø%)

... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dbda183...bcebc36. Read the comment docs.

if build fails, can we only resume the failed job, instead of the whole build?

I see that if you click on the job you can restart just the job. I'm not sure if the dependent stages will run or not though. We can try it out the next time a job fails.


refactor: introduce FunctionCallbackResolver interface

Introduces a new FunctionCallbackResolver interface to define the strategy for resolving FunctionCallback instances from the application context.
Renames FunctionCallbackContext to DefaultFunctionCallbackResolver to better reflect its implementation role. Updates all related components to use the new interface.
Update the affected AI model implementations

Replaces FunctionCallbackContext parameter with FunctionCallbackResolver in all model constructors
Updates builder patterns to use functionCallbackResolver() method instead of withFunctionCallbackContext()
Deprecates old withFunctionCallbackContext() methods in builders to guide migration

Updates integration tests to use DefaultFunctionCallbackResolver
Improves documentation to clarify the resolver's role in function callbacks
Moves SchemaType enum from FunctionCallbackContext to FunctionCallback (Braking change)git add .

Resolves #758

Fixed the javadoc, squashed, rebased and merged as 479a0959769b5ff3be9841212b9e483743b671e0

Also, fixed an issue with the AnthropicAutoConfiguration's AnthropicChatModel to use the interface dependent type FunctionCallbackResolver be0f9fbb676627ec6f64fb99ad9c9cf407ba8941


Emojifix

Pull Request Checklist

[x] Pull request is based on the develop branch
[x] Pull request updates CHANGES.rst
[x] Pull request includes a sign off

Signed-off-by: N-Pex<EMAIL_ADDRESS>

@N-Pex thanks for this PR.


Compute block break time

Inputs: assumed blockstate id, current tool item/slot, player
Output: expected break time in game ticks

[ ] fundamentals: block material, tool type
[ ] tool enchantments
[ ] active player effects (haste, slowness, etc.)
[ ] slow-down from not standing on ground
[ ] underwater status vs. helmet enchantments (aqua affinity etc.)

don't read world state in accordance with Civ botting rules

@Gjum just to be clear, is this in terms of botting or in terms of break time for the needs of physics

This is for botting; physics receive all block breaks from the server. I figured I'd create this to keep track of all the different aspects of block break time we will eventually need to handle. Feel free to start working on this by the way.

I'm like halfway through this


Announcement: v3 beta testing

Hey,
I'm happy to say that the illusive v3 is now available for beta testing. I'm sorry it's taken so long but it turned into a full blown re-write in the end. The changelog is massive as you can imagine and there's quite a few breaking changes. The docs  are also updated.
It also includes an ads plugin (in partnership with https://vi.ai) which we are in the final stages of finalising. It's currently undocumented but once we're happy with it then it'll be documented and publicised accordingly.
If you find any bugs then please prefix them with [v3] so I know it's the new version. Once all testing is complete and bugs fixed, then it'll go to master.
There's a preview up here:
https://plyr.io/beta
Also, if you have any questions, feel free to duck into the slack workspace
Cheers
Sam

slack bit.ly link is dead :(

This new version does not seem to work on Internet Explorer without the polyfills.
Maybe this should be added to the documentation?

Sorry, will check the bit.ly link.
Yep, the new version uses ES6 which IE obviously doesn’t support so the polyfills are required if you need to support IE. Will add a section on polyfilling. It’s going to be the case with most libs these days.

@sampotts Yeah, makes sense, but I use your lib on computers that don't have access to the internet. Would it make sense to add a "build with polyfills" option to your build process?

wait, how about babel?

@dacostafilipe, @sampotts, @aolko
I've been thinking about this as well. For example, fetch was added as a plyr dependency after I initially started using it. I'm not using fetch due to the lack of some features XHR handles. It also isn't part of any of the es6/7/8-sets on polyfill.io. So when I updated it broke IE support.
It's beta, so these things should be expected. But it would be nice to ensure that future stable releases don't require additional undocumented polyfills. If I understand it correctly eslint-plugin-compat can be used to make the build fail if the additional polyfills aren't explicitly listed in a whitelist. In that case this could be used to make the dev aware of any additional polyfill requirements introduced since last release.
This could in turn be used to document additional requirements in the changelog, release notes and main page. But also possibly to maintain an always updated set of polyfills needed at polyfill.io. Blissfuljs for example does this. I think this makes it much easier for devs.
Alternatively, building with polyfills included as suggested by @dacostafilipe makes sense too. This should be less complicated with babel, but adds more of an everhead. I'm not sure how well babel and rollup actually handles pruning unused polyfills.
It would make sense if this was done before the stable release.

I've been working a lot with ES6 recently and haven't really found an ideal solution to this. polyfill.io means relying on a third party which is never ideal and babel-polyfill / core-js massively bloat the bundle for 95% of users who don't need the polyfills. I also found babel-polyfill included a fair bit I didn't need even with it set up correctly to apparently include all that's required rather than everything. My plan, as lazy as it sounds, was to let the end users decide since inevitably you're going to have to polyfill for other libs as ES6 and newer become more prevalent. I could probably do a polyfilled bundle somehow with gulp/rollup but it was like 50k gzipped when I tried which is insane.
I can probably roll back fetch to use XHR as I'm not sure I need fetch as such anyway. Polyfill.io does include a polyfill but it's 7k and as I say, I'll work on replacing it with XHR.

Slack link updated ... it's https://bit.ly/plyr-slack 👍

i have Problems with fetch too
-> Safari 10.1 (mobile) doesn't know this function
-> core-js won't like to implement a polyfill
Also there's a Problem with "find" -> IE11 doesn't support this function. But for this problem i can use core-js
v3.0.11-beta

I'll document the polyfills but it's pretty common practice now with ES6. As I mentioned above, you'll notice the demo uses polyfill.io...

OK, just pushed 3.0.0-beta.12 with:

Fetch removed in favour of XHR. One less polyfill. Didn't really need fetch anyway.
Added more notes about polyfilling
Added backwards compatibility for <div> embed placeholders

Next up is to fix captions on IE11.

Hey,
I've just pushed v3.0.0-beta.13 which contains:

A fix for custom controls
Completely rewritten fullscreen class - iOS native fullscreen is WIP since playsinline makes it a little more tricky
A bug fix from @friday

Will look at IE11 captions next.
Cheers 👍

following polyfills could fix the fetch and promise problems:
npm install whatwg-fetch --save-dev
npm install es6-promise --save-dev

@gehaktmolen: Fetch isn't used anymore.

After @sampotts post regarding polyfills, I did some tests of my own.
I saw a 75% overhead when using babel-polyfill, obviously, this is not usable in production. (think about the idea of all libs/frameworks including polyfills ... scary)
For now, we will keep v2 for our "offline" sites and use v3 + polyfill.io on our online sites.
That said, I would add an build option for v3+polyfills for those special cases. I could create a PR if needed.

Yeah I will work out a build that includes polyfills. I'll probably try and use core-js and include just the polyfills that are needed or use babel-polyfill and babel-preset-env to try and limit the polyfills. I'll work something out...

Just pushed v3.0.0-beta.14 which contains:

Fix for pause button not showing while playing
Added polyfilled build (need to document) as plyr.polyfilled.js - this is 20kb (gzipped!) larger than the base build so I'd warn that it's only really suitable for standalone players as really you'd want to manage polyfills globally.
Added unminified builds to /dist - minified is now plyr.min.js (need to document). CDN builds remain minified and use the same filename (plyr.js).

Oh, thanks to @friday for the work on the unminified builds 👍

Glad I could help :)
Forgot to mention, if someone has gotten a [object%20Object] network request error (trying to get the icon svg), then this could be due to uglifyjs. It's not really a plyr bug since the error is introduced later. It has only happened to me using uglifyjs with mangle enabled (the default) and using the minified plyr build as a source. Either using the non-minified build which is now available in v3.0.0-beta.14 as dist/plyr.js, disabling mangle or switching to uglify-es seems to fix the issue.

Awesome job @sampotts !! Really appreciate this player, man! I saw you added a quality switcher for Youtube, any idea when that's coming for HTML5? Thanks again, bro!

Small issue: The progress bar is growing and shrinking as the time changes. This is seemingly because the new font isn't monospace and hence digits 1 and 7 are not as wide as 0 and 8, causing noticeable jumps.
Switching back to the old font fixes it, but obviously doesn't look quite the same:
.plyr--video .plyr__time {
  font: 600 13px 'avenir';
}

This last solution seems like a good one. I'll give it a go. The new font is really only for use in the demo as that's all it's licensed for anyway.

@friday I included your typeface change. I actually set as a global declaration on the whole player as we present timestamps in a few spots and I actually preferred now the numerics looked with it.
3.0.0-beta.18 contained several fixes for issues with .destroy() leaving event bindings on the document causing issues (as you can imagine). Also fixes for trying to set menus when there's no menus in the controls and some improvements to the ad plugin from @friday and @gehaktmolen 👍
3.0.0-beta.19 contains the above fix for the timestamps font spacing, thanks to @friday.

Nice! You've been working hard lately :) I think you have fixed all known v3 issues now? and obviously some that were present in v2 before.

I think so. Closing all the tickets related to bugs fixed in v3 might take me a while. I think I'll push current master to a v2 branch and then merge beta to master tomorrow night and prepare for the backlash 😂

3.0.0-beta.19 still Problems with destroy() on HTMLAudioElement
Uncaught TypeError: Cannot read property 'display' of null
    at e.timeUpdate (plyr.min.js:1)
    at HTMLAudioElement.<anonymous> (plyr.min.js:1)

@Maqsyo Try 3.0.0-beta.20 - I found in some cases I had to delay the garbage collection of old elements.

Thanks! this worked for me :)

Thanks for all your feedback and help. I'm gonna close this for now but if you find anything, please don't hesitate to grab me in Slack or raise an issue if need be! 👍


zero-copy send with IOSQE_CQE_SKIP_SUCCESS is broken

I noticed that zero-copy send can't be used with IOSQE_CQE_SKIP_SUCCESS. It took quite a while to find that out(i thought the problem was on my end), so I think it should be documented at least.

It's rather not yet supported, I need to add and agree that mans should be adjusted in the meanwhile.
Btw, let me ask what kind of behaviour you was expecting from it considering that there are two CQEs including notifications?

I was expecting no CQEs at all. I'm sending static buffer, I don't need notifications.

Agreed with illiliti, if I used IOSQE_CQE_SKIP_SUCCESS with sendzc, I imagine it would only be for buffers that remain "alive" for the entire duration of the program i.e. so I would not be freeing them upon notif CQEs.
More precisely, if IOSQE_CQE_SKIP_SUCCESS  is set:
(first CQE) of sendzc only shows up if failed or it was a short send
(second CQE i.e. notif CQE) should never show up with  IOSQE_CQE_SKIP_SUCCESS.

It doesn't seem viable apart from benchmarks and highly specific cases. The user should not be rewriting buffers before getting a notification, so it needs a way to know that the buffer is freed. My plan is to:

Make IOSQE_CQE_SKIP_SUCCESS to affect only the first CQE according to the generic rules. That gives consistency.
Add a send-zc flag that would copy the return code into the notification CQE and skip the completion (1st) CQE. IOW, you would get it roughly with an ACK for TCP.

And allow to freely mix them. How does it sound?

The buffer is constant. An example of such buffer is a file #embeded at compile time or mmaped at program startup. It never changes, it needs no way to know if it's freed, thus it need no (from application/user perspective)useless notifications about its state.
@bnbajwa proposal is exactly what is needed, basically make IOSQE_CQE_SKIP_SUCCESS work as it work for normal non-zerocopy send + discard second SQE. Your proposal void the whole point of IOSQE_CQE_SKIP_SUCCESS because user would still have to handle CQE.

Add a send-zc flag that would copy the return code into the notification CQE and skip the completion (1st) CQE. IOW, you would get it roughly with an ACK for TCP.

And allow to freely mix them. How does it sound?

If IOSQE_CQE_SKIP_SUCCESS and the send-zc flag can be freely mixed together, why does the send-zc flag need to implicitly skip the completion (1st) CQE? send-zc flag should just copy the return code from completion CQE into the notification CQE, that's it — it should not automatically skip the first CQE. If the user wants  to to skip the completion (1st) CQE, then they can just add the IOSQE_CQE_SKIP_SUCCESS flag too (i.e. IOSQE_CQE_SKIP_SUCCESS + send-zc).
send-zc flag would be useful, it would simplify my TCP sendzc implementation. This is because — currently — when we get a notif CQE, we can't be sure if we should free the buffer or not as we can get notif CQEs even on failures and short sends — in which case I will want to retry sendzc request with same buffer. So knowing the return code in the notif CQE is important. I've implemented this in userspace by simply copying the return code from 1st CQE into allocated memory which the notif CQE then reads from.

The buffer is constant. An example of such buffer is a file #embeded at compile time or mmaped at program startup. It never changes, it needs no way to know if it's freed, thus it need no (from application/user perspective)useless notifications about its state.

Again, that's too limited of a use case and more seems like a benchmark to me. We're not going to cater it to one very specific case.

@bnbajwa proposal is exactly what is needed, basically make IOSQE_CQE_SKIP_SUCCESS work as it work for normal non-zerocopy send + discard second CQE. Your proposal void the whole point of IOSQE_CQE_SKIP_SUCCESS because user would still have to handle CQE.

How so? Would you elaborate? Set two of them and you get a usual IOSQE_CQE_SKIP_SUCCESS behaviour, one CQE on failure or none if succeed, which is, to be honest, pretty useless unless linked with other requests.

Add a send-zc flag that would copy the return code into the notification CQE and skip the completion (1st) CQE. IOW, you would get it roughly with an ACK for TCP.

And allow to freely mix them. How does it sound?

If IOSQE_CQE_SKIP_SUCCESS and the send-zc flag can be freely mixed together, why does the send-zc flag need to implicitly skip the completion (1st) CQE? send-zc flag should just copy the return code from completion CQE into the notification CQE — it should not automatically skip the first CQE. If the user wants to to skip the completion (1st) CQE, then they can just add the IOSQE_CQE_SKIP_SUCCESS flag too (i.e. IOSQE_CQE_SKIP_SUCCESS + send-zc).

It seems you don't understand the idea. You can also think of it as combining two CQEs into one, which will be posted at a time when previously the notification would've been posted.

send-zc flag would be useful, it would simplify my TCP sendzc implementation. This is because — currently — when we get a notif CQE, we can't be sure if we should free the buffer or not as we can get notif CQEs even on failures and short sends — in which case I will want to retry sendzc request with same buffer. So knowing the return code in the notif CQE is important. I've implemented this in userspace by simply copying the return code from 1st CQE into allocated memory which the notif CQE then reads from. So, its simple to implement in userspace, thus, the flag itself is not essential but would be nice.

A notification CQE is posted strictly after the corresponding completion CQEs. If it's not the case, please report a bug.

The user should not be rewriting buffers before getting a notification, so it needs a way to know that the buffer is freed.

As with @illiliti, my imagined use case would also be for constant buffers that are never modified e.g. a loaded file in memory.

Add a send-zc flag that would copy the return code into the notification CQE and skip the completion (1st) CQE. IOW, you would get it roughly with an ACK for TCP.

I've been thinking about this further, is your proposal that the completion (1st) CQE would always be skipped with this flag even for failures/short sends? So essentially, it would turn sendzc from a 2 CQE request into an always 1 CQE request? If so, seems to be reasonable but not sure if it would be widely applicable.

No, the idea is to combine two into one, and then you can apply generic IOSQE_CQE_SKIP_SUCCESS rules on top of that one, i.e. CQE on error or none otherwise.

It doesn't seem viable apart from benchmarks and highly specific cases.

I don't think any of the proposals in this thread for IOSQE_CQE_SKIP_SUCCESS are general enough to be honest. They are maybe applicable to quite specific cases only.

Make IOSQE_CQE_SKIP_SUCCESS to affect only the first CQE according to the generic rules. That gives consistency.

I think IOSQE_CQE_SKIP_SUCCESS affecting only the first CQE seems fine/reasonable. However, I think we do need a flag to allow the 2nd notif CQE to be skipped (always?) as well.
Being able to control exactly which CQEs to skip seems like a more general proposal. i.e. we should be able to skip any of:
* first CQE only

* second CQE only

* both CQEs

For example, suppose I had a constant buffer loaded in memory that I wanted to send in chunks. I would want to skip second CQE because I won't be modifying the buffer. However, I would want to receive the first CQE so I immediately know when I can send the next chunk (high throughput).

In general case the user need to do sth with the buffer afterward (free / reuse) and so should have a way to infer the state of the buffer. otherwise it can't be reused or even normally freed but only munmap()'ed at the end. Just skipping is not going to work.

@isilence Excuse me, how's the progress? Because any error might lead wait(nr) for send_zc into deadlock, a single cqe is still needed to fit wait(nr).

I drafted something similar some time ago, can rebase and merge, but I wonder if it even fits your app, and that depends on your workload pattern. Notifications come much later than main completions, and by merging them together you'll get a big latency for getting the result back. Do you use TCP? Does your program looks like send(); recv(); send(); recv() ... ? Or can you send multiple times before getting anything back?
With how it is now, you can wait (in terms how you choose that "nr") in two steps, first for the main completion and then the notification. Even more, you don't want to eagerly wait for the notifications unless you have a memory pressure situation.
compl_expected = 0;
notif_expected = 0;

while (1) {
	while (send_more()) {
		queue_send_zc();
		compl_expected++;
	}

	to_wait = compl_expected;
	if (memory_pressure)
		to_wait += notif_expected;
	wait_cqes(nr=to_wait);

	while (cqe = peek_cqe()) {
		compl_expected--;

		if (is_notification(cqe)) {
			notif_expected--;
		} else { // sendzc
			if (cqe->flags & F_MORE) {
				compl_expected++;
				notif_expected++;
			}
		}
	}
}

Notifications come much later than main completions, and by merging them together you'll get a big latency for getting the result back. Do you use TCP? Does your program looks like send(); recv(); send(); recv() ... ? Or can you send multiple times before getting anything back?

If non-zc wants to skip the completion cqe, why zc not? Personally notification cqes are just used to reclaim buffers if necessary. Also it is almost a free lunch for UDP. So IMHO it is just a complement, meanwhile it will be another enhanced variant to skip the notification cqe.

With how it is now, you can wait (in terms how you choose that "nr") in two steps, first for the main completion and then the notification. Even more, you don't want to eagerly wait for the notifications unless you have a memory pressure situation.

Still have to collect at least twice. My temporary workaround is io_uring_submit_and_wait_timeout(nr*2), so mostly no need to worry about rare error cases.  However wait(nr) could be preferable for precise reclaim.

Notifications come much later than main completions, and by merging them together you'll get a big latency for getting the result back. Do you use TCP? Does your program looks like send(); recv(); send(); recv() ... ? Or can you send multiple times before getting anything back?

If non-zc wants to skip the completion cqe, why zc not? Personally notification cqes are just used to reclaim buffers if necessary. Also it is almost a free lunch for UDP. So IMHO it is just a complement, meanwhile it will be another enhanced variant to skip the notification cqe.

Right, which is why it's a separate question on your use case and it's for outlining pitfalls.

With how it is now, you can wait (in terms how you choose that "nr") in two steps, first for the main completion and then the notification. Even more, you don't want to eagerly wait for the notifications unless you have a memory pressure situation.

Still have to collect at least twice. My temporary workaround is io_uring_submit_and_wait_timeout(nr*2), so mostly no need to worry about rare error cases. However wait(nr) could be preferable for precise reclaim.

Exactly why I'm asking about your use case. First it's not right as you're not guaranteed that it'll post 2 CQEs, e.g. when something bad happens early on like an allocation failure, so that can wait indefinitely. And also that could be a MASSIVE performance / latency problem depending on the use case. Users generally don't/shouldn't want to eagerly wait for notifications unless you have pressure on the memory front built up, and I've seen people falling into similar traps and finding performance being miserable.
I'd wait for nr, i.e. one CQE per send, but because that wait might have left more than nr CQEs in the ring, I'd try to opportunistically reap more CQEs.
io_uring_wait_cqe(nr);
io_uring_for_each_cqe(ring, head, cqe) {
    handle_cqe();
}
io_uring_cq_advance(ring, nr_cqes_actually_processed);


StealSummary table implementation

StealSummary:
Runner_on_1 = (no runner=null, no steal=0, steal=1, perfect steal=2)
Runner_on_2 = (no runner=null, no steal=0, steal=1, perfect steal=2)
Runner_on_3 = (no runner=null, no steal=0, steal=1, perfect steal=2)
CatcherRosterLoc

TODO move that to definition and stop doing this on mobile


APEXMALHAR-2184: Add documentation for File Input Operator

@yogidevendra   @ashwinchandrap  and @amberarrow  please review.

Please take a look at branch APEXMALHAR-2184-ram at https://github.com/amberarrow/incubator-apex-malhar
I've added more explanatory content and made various other changes. I've also made several additional comments on this PR. Please review both and merge into your PR. This guide needs more work, specifically:
The use of the WindowDataManager needs to be discussed.
The derivative abstract classes need more explanation and also need code fragments illustrating how to make concrete derived classes out of them.

@amberarrow  is it a good idea to discussion possible implementations for scenarios like tracking file name for tuple etc in this document? Or they should go somewhere else like discussion forum etc? Or may be a new section in this document which discuss about implementation possibilities?

@amberarrow  for partitioning, you have mentioned dynamic partitioning can be done by changing value of "partitionCount" parameter, but that doesn't seem to be a dynamic partitioning exactly. Provided this any way of defining our static partitioning can actually be dynamic as we allow changes of variable values on application runtime.

In this instance, tracking the file name comes up often w.r.t this operator, so yes, I think we should provide the information in the context where it is most useful and that is here. Creating a new section in the document is fine as long as the information is present and is accurate. The new section can be titled "Common implementation issues" or even "FAQ" or something similar. Not all operators merit this treatment, but this one does.
We should not lose track of the main purpose of this document: The reader should be able to use the operator in common situations without running into any difficulties that are not discussed in the document.
Regarding dynamic partitioning -- I've explained my rationale in the earlier note. Yes we allow variable changes at runtime and it is quite reasonable and even standard to call them "dynamic" changes. Not all operators support partitioning at runtime based on such changes; it is entirely reasonable to call those that do "dynamically partitionable". I don't quite understand what this means: "Provided this any way of defining our static partitioning can actually be dynamic ".
The design question can be discussed separately.

@amberarrow  I have addressed most of your review comments. Can you take a look and merge PR if looks appropriate?


Unable to use the --docker parameter in CLI due to interpolation format error

Hi,
I have recently started using R2R. I have followed the documentation and set up Postgres + pgvector. While trying to start the R2R server using the following docker command:
r2r --config-name=default serve --docker
I am getting the following exception message:
ERROR: Invalid interpolation format for "r2r" option in service "services": "POSTGRES_USER=${POSTGRES_USER:-postgres}"
It works fine when I run with the following code:
r2r --config-name=default serve

Hi,
Thanks for sharing! Unfortunately, I cannot replicate this from my laptop.
Which environment variables do you have set for POSTGRES_USER? Which os you running on and with which cpu arch? Lastly, which Docker version are you using?
Sorry for the inconvenience - let's figure this one out.

Hi @emrgnt-cmplxty ,
Please check the following information:
POSTGRES_USER=postgres
POSTGRES_HOST=localhost
POSTGRES_PORT=5432

OS: Ubuntu 20.04.6
CPU Arch: x86_64
Docker Version: 27.1.0

Thanks @jais001 - We are going to try to replicate on an ubuntu machine. I believe this is likely the source of some of the other errors you are seeing as well, we will report back.

Could you please run docker-compose --version and report back your findings?

@emrgnt-cmplxty
My docker-compose version is 1.25.0