Controller documentation

Here is all of the documentation for the website on the controller. It will include in the end all of the relevant information on each of the control loops, a guide on tuning the controller and short summary guide to editing and modifying the controller.

One question I have is whether I should add a diagram for the inner loop and outer loop for each of the controllers. It would make things more clear, but perhaps all I have to do is highlight an example loop when briefly discussing successive loop closure as a whole.

One question I have is whether I should add a diagram for the inner loop and outer loop for each of the controllers. It would make things more clear, but perhaps all I have to do is highlight an example loop when briefly discussing successive loop closure as a whole.

I think you could also just draw a box behind the inner loop on each of the diagrams, and label the box "Inner Loop". That might be more clear and take up less space than making inner and outer loop diagrams for everything. Thoughts?

I have finished all of the controller documentation. It looks like other documentation has been put onto this branch as well, I will leave it to you @JMoore5353 to decide when to merge this/make this a full PR rather than a draft.

@JMoore5353 Could we rename the directory "Path Planning" to something like "Navigation"? Path planning referencing both a specific module and the entire path part of the autopilot I think could be confusing.

I have finished all of the controller documentation. It looks like other documentation has been put onto this branch as well, I will leave it to you @JMoore5353 to decide when to merge this/make this a full PR rather than a draft.

Sorry about that. I should have made a new branch. I wasn't thinking about that!


Settings editor: Present extension-specific table of contents when scoped to extension

Related/parent issue: https://github.com/microsoft/vscode/issues/70589
A lot of this is discussed in #70589 but the settings UI with GitLens is completely unusable:

I'd love to get to a point where extensions investing in a custom settings UI is a waste of time. Right now they solve real problems, for the above it's disorienting trying to scroll in such an enormous page and without a proper table of contents. To start tackling this issue I think we should change scoped searches to change the table of contents, so the TOC here:

Looks a lot more like this:

The best way to group/order these is maybe a separate discussion, but it could be broken up by the namespace of the setting initially.
cc @eamodio

@rzhao271 if searching multiple extensions if could just revert to the regular view, I image that's a pretty uncommon case too.

TOC here

hey! the tom would require being able to grouping things in settings first, right? i think the following proposal can help with that:

https://github.com/microsoft/vscode/issues/200686


Fix MB units conversion(spark->k8s)&Set the executor ENV_EXECUTOR_MEMORY to executorMemoryWithOverhead

What changes were proposed in this pull request?

Fix MB units conversion(spark->k8s) error.
Set the executor ENV_EXECUTOR_MEMORY environment variable to executorMemoryWithOverhead in accordance with driver.

How was this patch tested?
Manual tests show that Memory units conversion is correct and ENV_EXECUTOR_MEMORY environment variable is same with executorMemoryWithOverhead.

Dupe of #470

@ash211 i am afraid #470 is not enough to include my PR.


Syntax Error

File "download_archive.py", line 84
def download_descriptions(ids: list, descs_dir: str, num_processes: int) -> list:
^
SyntaxError: invalid syntax

Hmm I can't seem to reproduce it
Can you do 2 things for me?

Make sure you have the latest version of this repository
Tell me what is the result of python --version

Thank you so much for your time. It is working now. I was running it on python 2. Sorry for creating an issue which was not there in the first place.

On the contrary my friend!
At the time you submitted your issue, the project did not ask for python 3 as a necessity.
After you submitted your issue, I suspected that it should have been, which is why I asked you for your python version.
You can even see that I updated the README file to contain the python 3 requirement a couple of days after you submitted your issue.
So you were completely on to an issue! :)
I'm glad it's working well for you now, and thank you for bringing up this issue!

I thought i had missed the python 3 part in the readme. Glad it all worked out.


Fixed grammatical error and added enhancement

Description

Fixed grammatical error and added enhancement to the sentences in the 4 Issue templates
...

Closes #12344

Types of changes

[X] Documentation
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (anything that would change an existing functionality)
[ ] Maintenance (refactor, code cleaning, development tools...)

Checklist

[X] I have read and follow the CONTRIBUTING.md document.
[X] The pull request title and template are correctly filled.
[X] The pull request targets the right branch (develop or develop-v...).
[X] My commits are correctly titled and contain all relevant information.
[X] I have updated the documentation accordingly to my changes (if relevant).
[ ] I have added tests to cover my changes (if relevant).

These are not really grammatical errors. You read the contribution guide once and then follow / understand / apply it.


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/112026

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/112935

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/114060

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/116651


adding the quality module

fixing this https://github.com/rainyl/opencv_dart/issues/216

@abdelaziz-mahdy please also bump the version to 4.10.0+5

oh sorry, i forgot this is opencv.full😥

thx, i will push a new tag later to release it.

<3


Add Response class

This class represents an HTTP response.
public __construct ()
public View loadView ()
public void setHeader (string $str, [, int $responseCode])
public void setResponseCode (int $responseCode)

I'm on it


presentation is too dense

The "Kubernetes Resource Model" slide in the presentation is way too dense. I think it needs to be split into multiple slides so that people can grasp the material.
For the workshop last week I did split it out into multiple slides -- attached in case it's helpful:
Kube-101-all.pptx

I agree, I think the argument for that slide was, "there's a lot of other capability, but we're only going to talk about xyz" I think some of the added expanded slides go into things that aren't even in the material at all.

You're probably right -- depends on the goal of the presentation. Are we giving an detailed overview, or just enough to get through the example? Might need a set of "accordion" slides that can expand or contract depending on length of presentation, whether it's hands-on or lecture, etc. :)

@emckean is there any builtin functionality to powerpoint for this accordion type of operation? We've been looking for just that kind of functionality in lots of tools for a long time.

You can do this with Reveal.js -- they have a "vertical slides" function so
you can go forward and back and "up and down" so you can literally drill
down on a topic. :)
On Mon, May 14, 2018 at 2:29 PM, Morgan Bauer<EMAIL_ADDRESS>wrote:

@emckean https://github.com/emckean is there any builtin functionality
to powerpoint for this accordion type of operation? We've been looking for
just that kind of functionality in lots of tools for a long time.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IBM/kube101/issues/11#issuecomment-388969044, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AANwmVVRZQlEKp9freOuhviNK08P6vyjks5tyfcngaJpZM4TIyYC
.

looks neat. Looks like it can run locally as well, which is necessary. @duglin WDYT?

expired, please recreate issue if this still exists in the current version


[explorer] - add UI for package versions

Description
Describe the changes or additions included in this PR.
Test Plan
How did you test the new or updated feature?

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.
Type of Change (Check all that apply)

[ ] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes

Current dependencies on/for this PR:

main

PR #9150   👈

This comment was auto-generated by Graphite.


Function call with Ollama and LlamaIndex

Hi, I'm looking for a way to add function call to work with Ollama and LlamaIndex.
From my research we have format json in Ollama, so theoretically, there are 2 ways we can support function call:

Enforce the LLM to output json following a schema, and we can call the function based on the json output.

Not sure how reliable it is for this approach, has anyone been able to have a consistent output from the LLM for the exact prompt?
Client side also need to implement a retry mechanism so we will feed the previous output and errors back to LLM and ask it to regenerate
What are schemas and data structure that we should use? Currently, most people seem to go with OpenAI function call schema, but it does not support validation and we probably need to have a pydantic model and keep it up-to-date for LLM response's validation.
Some examples: https://github.com/lgrammel/modelfusion/blob/main/examples/basic/src/model-provider/ollama/ollama-chat-use-tools-or-generator-text-mistral-example.ts

We can also add API in Ollama itself to support function call directly, similar to OpenAI.

I'm not sure how this will work, especially OpenAI is not open source. Do you think it's possible to implement the function call feature directly in Ollama?

I'm not sure will we need to have a specific model that support function call, and we can feed { role: "tool", content: "tool output" } into the LLM
Or it's simply the feature we can add at the API level.

Please let me know what do you guys think and what should be the right approach for this issue going forward.

From personal experience, enforcing the schema is somewhat hit-or-miss, especially depending on the complexity of the schema. I've gotten the best results with both being highly explicit in describing the schema (explaining each property in detail, specifying which properties are required), instructing it to only follow the schema (eg. "only include properties defined in the schema"), and giving some examples.
For my own project I'm currently using a different approach where I instead I defined a custom "line-based protocol" for it to use which allows for both "sending messages" as well as "running commands" which not only reduces the overall response size (since JSON is quite verbose and thus increases the number of tokens per response quite a lot), but also enables my application to make use of streaming as well. The specifics of the protocol are somewhat specific to my application, but the general gist of it is this:
Every response line is either a message or a command.
Empty lines are skipped from processing

A response line is processed as a command if it is prefixed with `{command}:`.
Calling the `a` (action block) command takes in an action and parameters
Calling the `d` (data) command takes in a JSON object to be passed into the current action block
Calling the `e` (end) command ends the current action block
a: insert tasks
d: { "name": "Task name", "completed": false }
e:

Actions can also take in multiple parameters, for example to update a collection we can do
a: update tasks { "name": "Task name" }
d: { "completed": true }
e:

Response lines which are not prefixed with a command are processed as regular messages

Hi @xprnio ,
Thanks a lot for sharing the experience and the detailed write-up. I wonder which schema do you use? Is it following OpenAI function call schema, or is it a custom schema we define ourselves?

@sandangel
You need to define your own schema, which means that the world is your oyster in that regard. Make the schema as complex or as simple as you want, explain it however you want, etc.
For more of how I used it, have a look at this gist. It's quite big though (in terms of tokens) and mainly focuses on explaining it more in natural language than code, but does also incorporate quite a lot of examples to help the LLM understand.
I've also heard that another good way of describing JSON is to use TypeScript (haven't tested, but I think this might be a pretty good approach as well).

My application explains the protocol, the various actions available, and the collections to the model in the system prompt, and by giving some examples for each of them it does do it's job quite well (at least the Mistral and Mixtral models, haven't tested others yet)

I'll have to try using the Mistral and Mixtral models. I've been adapting the Eclipse IDE plug-in called "AI Assist" to work with the Ollama API instead of OpenAI API but so far I've found it excruciatingly hard to get any of the coding specific LLMs to use function calls:

The Deepseek models seem to have been actively fine tuned to refuse to run any functions even though they seem to understand what you are asking them to do!
The codelama models and their derivatives seem to have much more trouble understanding what you are asking them to do, but will everything call a function if you totally spell it out and ask them "please call function X", but otherwise they just won't use them.

I'll be interested to see what you are using to help prompt them into using functions in your code. I agree showing examples of how to call the functions is important. The most success I had was just adding the functions to the system prompt in OpenAI API format (with the parameter descriptions, which parameters are optional, etc) with some examples below of how to use them.
I also found trying to get chat/instruct fine-tuned models to call functions right at the start of their reply (because of the way AI Assist handles streaming and function calls) was near impossible. Had some funny chats along the lines of: "No!!! please use the function at the start of the message" and they would then apologise and then try calling it again - doh.
Overall it's been a huge fail so far.

Hi @sandangel , @xprnio , @jukofyork , thanks for contributing to this issue. For function calling, I have found the best result coming from doing a few things:
First include format: json. Then specify in the system prompt that the model needs to output json. This gets you most of the way there. What makes it perfect in most cases I have tried is to do a few shot prompt. This is easiest with the chat endpoint. So include your system prompt, then an example question, and then the example answer in your schema. repeat that 1 or 2 more times. That has worked well for me.

You're right @technovangelist, the way I used to do it was by putting all of the examples into the system prompt instead of "simulating" the examples through the chat interface itself with pre-made messages showing the expected path.

@xprnio can you please share an example of your code? I wanted to build a bot that asks necessary questions, and when the  requisite information is received, then it calls the api. (imagine a shopping bot).

@sampriti026 what part of the code do you mean exactly? In all honesty, the application I've been using this approach in has been put "into the drawer" for a bit and isn't really that good in terms of quality. But I do plan on open-sourcing the project as soon as I get time to clean up the code a bit, however I guess there's nothing really stopping me from just throwing it all up here and getting to cleaning it up whenever I have the time to.
But yeah, let me know what exactly you want an example of, I'll try to get that project up here on Git some time this week, and I'll give you a ping with the appropriate part of it. For context, the project itself is written in Go, just so you know

I read on twitter - one user was getting good mileage making 2 calls - rather than forcing chatgpt 3.5 to return json in addition to prompt - just get the results - then ask api to format result into a json response. was 100% hit rate.

for function calling you can try this model  https://huggingface.co/TheBloke/gorilla-openfunctions-v1-GGUF and then format the prompt template as :
FROM ./gorilla-openfunctions-v1.Q4_K_M.gguf

TEMPLATE """

### User:
{{.Prompt }}
### System:
{{.System}}

### Response:
"""

SYSTEM """<<function>> functions = [
    {
        "name": "Uber Carpool",
        "api_name": "uber.ride",
        "description": "Find suitable ride for customers given the location, type of ride, and the a>
        "parameters":  [
            {"name": "loc", "description": "Location of the starting place of the Uber ride"},
            {"name": "type", "enum": ["plus", "comfort", "black"], "description": "Types of Uber rid>
            {"name": "time", "description": "The amount of time in minutes the customer is willing t>
        ]
    }
]\n
ASSISTANT:"""

PARAMETER stop "<|system|>"
PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"
PARAMETER stop "</s>"

./ollama run gorilla_test "USER: <<question>>"Call me an Uber ride type \"Plus\" in Berkeley at zipcode 94704 in 10 minutes ""
uber.ride(USER="plus", LOC="94704", TIME=10)

and append ""USER: <" before the user request.

how about this blog:https://www.lepton.ai/blog/structural-decoding-function-calling-for-all-open-llms？

how about this blog:https://www.lepton.ai/blog/structural-decoding-function-calling-for-all-open-llms？

I get 404 for this url

sorry for that, here is the link:https://www.lepton.ai/blog/structural-decoding-function-calling-for-all-open-llms

for function calling you can try this model https://huggingface.co/TheBloke/gorilla-openfunctions-v1-GGUF and then format the prompt template as :
FROM ./gorilla-openfunctions-v1.Q4_K_M.gguf

TEMPLATE """

### User:
{{.Prompt }}
### System:
{{.System}}

### Response:
"""

SYSTEM """<<function>> functions = [
    {
        "name": "Uber Carpool",
        "api_name": "uber.ride",
        "description": "Find suitable ride for customers given the location, type of ride, and the a>
        "parameters":  [
            {"name": "loc", "description": "Location of the starting place of the Uber ride"},
            {"name": "type", "enum": ["plus", "comfort", "black"], "description": "Types of Uber rid>
            {"name": "time", "description": "The amount of time in minutes the customer is willing t>
        ]
    }
]\n
ASSISTANT:"""

PARAMETER stop "<|system|>"
PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"
PARAMETER stop "</s>"

./ollama run gorilla_test "USER: <<question>>"Call me an Uber ride type \"Plus\" in Berkeley at zipcode 94704 in 10 minutes ""
uber.ride(USER="plus", LOC="94704", TIME=10)

and append "USER: <" before the user request.

i cant   understand

Hi there! Tools are now supported in Ollama. See https://ollama.com/blog/tool-support. After some preliminary testing, it does work with LlamaIndex's OpenAI tooling and I know they're working on some amazing tool calling improvements to their Ollama integration


Remove usages of async-unsafe lifecycle methods

Fixes #897

I think my changes in #898 supersede #856 .
We might want to consider how these would fit together.

I'm testing with this fork and components subscribed to redux state using mapStateToProps don't trigger componentDidUpdate on state changes.  Is there something else I need to set up?

@peacechen it works OK on my project. Can you share some parts of your setup (maybe as a reproduction repo), so that I could debug it?

It's a pretty large app.  I'll try to create a small sample project to recreate the behavior.

Yeah, I suspect there's a lot of overlap between the two.  I assume this would probably merit a v6.0 major version?  So, notionally v6 would be "compat with React 16.3+", and v7 would be "super-duper React async rendering compat" .

Anyone else want to second this? I can push a 5.1.0-beta.1 to give it a whirl elsewhere.

I could try to integrate it into a small project of mine and leave some feedback, once the beta is out on npm.

OK, I'm going to merge this in and push a 5.1.0-test.1 release. Please give this a whirl! We can look into some of the BC-breaking stuff next.

Found a big bug with the gDSFP change in 16.4: https://reactjs.org/blog/2018/05/23/react-v-16-4.html#bugfix-for-getderivedstatefromprops
Because it's getting called for any render, it's getting double-called for most updates. That leads to it setting the sCU flag to true and then false on the 2nd call.
I've got to do a deeper dive, but I don't have time at the moment.

Well, I think (hope) it's just an issue with our test suite. So, react-redux@next is now on npm to try out.


kubectl config set: parameter name split with space in --help

kubectl v1.10.0
/kind bug
/sig cli
See the space in PROPERTY _NAME and PROPERTY _VALUE occurrences below:
$ kubectl config set --help
Sets an individual value in a kubeconfig file

PROPERTY _NAME is a dot delimited name where each token represents either an attribute name or a map key.  Map keys may
not contain dots.

PROPERTY _VALUE is the new value you wish to set. Binary fields such as 'certificate-authority-data' expect a base64
encoded string unless the --set-raw-bytes flag is used.

Options:
      --set-raw-bytes=false: When writing a []byte PROPERTY_VALUE, write the given string directly without base64
decoding.

Usage:
  kubectl config set PROPERTY_NAME PROPERTY_VALUE [options]

I took a run at this and failed to find a good resolution.    Anywhere templates.LongDesc is called like https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/config/set.go#L48 goes through the markdown processor here https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/templates/normalizers.go#L34.   The characters *, _, & and [ (possibly others) get prepended by a space.
The following test run in the https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/templates directory demonstrate the problem.
`
package templates

import (
	"fmt"
	"testing"
)

func TestLongDesc(t *testing.T) {
	fmt.Println("testing testy tester")
	testData := []struct {
		n        string
		expected string
	}{
		{"A_underscore", "A_underscore"},
		{"A[bracket", "A[bracket"},
		{"A&ampersand", "A&ampersand"},
		{"A*asterisk", "A*asterisk"},
	}

	for _, d := range testData {
		actual := LongDesc(d.n)
		if actual != d.expected {
			t.Errorf("LongDesc: expected %s, actual %s", d.expected, actual)
		}
	}
}

`
I traced the problem to the private callback methods here https://github.com/russross/blackfriday/blob/master/markdown.go#L366-L381 but could not pry any deeper.    It seems like the proper solution would be to set different per character callback methods but I couldn't figure out how to do that.

This is still relevant.
/remove-lifecycle stale
But this does not meet the criteria for help-wanted because the general direction for the solution is not yet defined.
/remove-help

/area kubectl
/priority P3


Add support for 7th gen vm types

Adds support for general purpose and RAM optimized 7th gen vm types.
https://aws.amazon.com/about-aws/whats-new/2023/08/amazon-ec2-m7i-flex-m7i-instances/
https://aws.amazon.com/about-aws/whats-new/2023/09/amazon-ec2-r7iz-instances/

are there vm types that are/can be removed as far as you know?


Kernel not loading on Spyder startup

File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/spyder/plugins/ipythonconsole.py", line 1345, in close_client
self.create_new_client()
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/spyder/plugins/ipythonconsole.py", line 1038, in create_new_client
self.connect_client_to_kernel(client)
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/spyder/plugins/ipythonconsole.py", line 1073, in connect_client_to_kernel
kc.start_channels(shell=True, iopub=True)
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/qtconsole/kernel_mixins.py", line 43, in start_channels
super(QtKernelClientMixin, self).start_channels(*args, **kw)
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/jupyter_client/threaded.py", line 212, in start_channels
self.shell_channel._inspect = self._check_kernel_info_reply
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/jupyter_client/client.py", line 143, in shell_channel
socket, self.session, self.ioloop
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/jupyter_client/threaded.py", line 205, in ioloop
self._ioloop = ioloop.IOLoop()
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/tornado/util.py", line 312, in new
instance.initialize(*args, **init_kwargs)
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 139, in initialize
super(ZMQIOLoop, self).initialize(impl=impl, **kwargs)
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/tornado/ioloop.py", line 845, in initialize
super(PollIOLoop, self).initialize(**kwargs)
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/tornado/ioloop.py", line 357, in initialize
self.make_current()
File "/home/deepak/.conda/envs/dlgpu_py3/lib/python3.6/site-packages/tornado/ioloop.py", line 315, in make_current
assert asyncio is None
AssertionError

Thanks for reporting. This is a duplicate of #6627 ; and in any case outside of Spyder's control. To fix it, run conda update pyzmq from the Anaconda Prompt/Terminal/shell (Windows/Mac/Linux) in the conda environment that Spyder is installed in.


Web Font Load Issue

@aristath
Issue description:
hey Ather Here,
i am working on custom theme i want to load webfont in backend admin (Gutenberg Editor) area and also the kirki generated css file. Both are loading fine however in windows the styles are not applying but in mac its work fine. Any idea.
There is one other issue also. after i make any change to customizer i will have to manually refresh the kirki generated css file in browser then it take effect (But only working in Mack for now.)
add_filter( 'kirki/dynamic_css/method', function() {
return 'file';
});
function theme_styles() {
wp_enqueue_style( 'kirki-style', content_url() . '/uploads/kirki-css/styles.css', false, '@@pkg.version', 'all' );
wp_enqueue_script( 'webfont-script', 'https://ajax.googleapis.com/ajax/libs/webfont/1.6.26/webfont.js?ver=3.0.22');
}
add_action( 'enqueue_block_editor_assets', 'theme_styles');
Version used:
3.0.22
Using theme_mods or options?
Kirki::add_config( 'start', array(
'capability'    => 'edit_theme_options',
'option_type'   => 'theme_mod',
) );

Closing due to inactivity. If this is still an issue then please feel free to reopen this and post an update on your current status.


CentOS 7 and systemd support, test validation

Add support for RedHat based systems, specifically CentOS 7. Also add support for systemd. Complete testing playbook by checking mesos metrics endpoint.

My reviews will be slightly delayed, do you require immediate review? Otherwise it won't be until tomorrow morning, unless any of the other members here can do a once over.
I also think you contributions merit a membership.
-Jason

On Aug 2, 2015, at 11:09 PM, Benoit Carriere<EMAIL_ADDRESS>wrote:
Add support for RedHat based systems, specifically CentOS 7. Also add support for systemd. Complete testing playbook by checking mesos metrics endpoint.
You can view, comment on, or merge this pull request online at:
https://github.com/AnsibleShipyard/ansible-mesos/pull/25
Commit Summary
CentOS/EL 7 and systemd support, test validation.
File Changes
M Vagrantfile (10)
M tasks/RedHat.yml (5)
M tasks/mesos.yml (38)
M templates/conf-mesos-slave.j2 (1)
A templates/mesos-master.service.j2 (14)
A templates/mesos-slave.service.j2 (14)
M tests/main.yml (18)
M tests/playbook.yml (2)
M vars/RedHat.yml (17)
Patch Links:
https://github.com/AnsibleShipyard/ansible-mesos/pull/25.patch
https://github.com/AnsibleShipyard/ansible-mesos/pull/25.diff
—
Reply to this email directly or view it on GitHub.

Hi Jason,
No I don't require immediate review, tomorrow morning is quite good.
And thank you for considering me for a membership, I was not expecting that but I will be happy to help :)
/B
From: Jason Giedymin <notifications<EMAIL_ADDRESS>Reply-To: AnsibleShipyard/ansible-mesos <reply<EMAIL_ADDRESS>Date: Monday, August 3, 2015 at 7:31 AM
To: AnsibleShipyard/ansible-mesos <ansible-mesos<EMAIL_ADDRESS>Cc: Benoit Carrière <bcarriere<EMAIL_ADDRESS>Subject: Re: [ansible-mesos] CentOS 7 and systemd support, test validation (#25)
My reviews will be slightly delayed, do you require immediate review? Otherwise it won't be until tomorrow morning, unless any of the other members here can do a once over.
I also think you contributions merit a membership.
-Jason

On Aug 2, 2015, at 11:09 PM, Benoit Carriere <notifications<EMAIL_ADDRESS>wrote:
Add support for RedHat based systems, specifically CentOS 7. Also add support for systemd. Complete testing playbook by checking mesos metrics endpoint.
You can view, comment on, or merge this pull request online at:
https://github.com/AnsibleShipyard/ansible-mesos/pull/25
Commit Summary
CentOS/EL 7 and systemd support, test validation.
File Changes
M Vagrantfile (10)
M tasks/RedHat.yml (5)
M tasks/mesos.yml (38)
M templates/conf-mesos-slave.j2 (1)
A templates/mesos-master.service.j2 (14)
A templates/mesos-slave.service.j2 (14)
M tests/main.yml (18)
M tests/playbook.yml (2)
M vars/RedHat.yml (17)
Patch Links:
https://github.com/AnsibleShipyard/ansible-mesos/pull/25.patch
https://github.com/AnsibleShipyard/ansible-mesos/pull/25.diff
Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHubhttps://github.com/AnsibleShipyard/ansible-mesos/pull/25#issuecomment-127200881.

Hey Jason,
I was wondering if you had a chance to look at the PR?
Thanks


Add support for any UPlayerInput class + retrocompatibility

This solve the problem from https://github.com/IDI-Systems/UnrealImGui/pull/21
Tested on 4.26, 4.27 and 5.3

Neat!
Any reason this should only work for 4.27 or higher? Documentation suggests that 4.26 also supports this.

I didn't see the function referenced there: https://docs.unrealengine.com/4.26/en-US/API/Runtime/Engine/GameFramework/UInputSettings/ that's why 4.26 isn't used in my code but if i look at github it's seems to be added in 4.26 https://github.com/EpicGames/UnrealEngine/blob/4.26/Engine/Source/Runtime/Engine/Classes/GameFramework/InputSettings.h

Also does this actually fully replace #17 or is it an addition to it?

I think it works to replace the iteration on every derived classes. Because why iterate on everything when we can know what will be used.
For the forcing to rebind OnEditorInitialized i don't know. I did'nt see any problem.

@Herhangi Any chance you could test this and see if it also fixes the issues fixed in #17?


CP: nested objects are not supported on c.sys.CUD

try to insert:
{
	"cuds": [
		{
			"fields": {
				"sys.QName": "app1pkg.air_table_plan",
				"sys.ID": 1,
				"name": "test",
				"air_table_plan_item": [
					{
						"form": 42
					}
				]
			}
		}
	]
}

error: status 500: cuds[0] INSERT app1pkg.air_table_plan: field air_table_plan_item: unsupported value type []interface {}{map[string]interface {}{"form":"42"}}
whereas inserting untill.order with untill.order_item works well because untill.order is not CUD in c.air.Order, it is written by IRowWriter.FillFromJSON() that supports nested objects.

that is the behaviour by design. Nested objects are inserted as separate objects linked by rawID


Fixes the hit detection does not work well at Retina display

The hit detection in styles with custom rendering at Retina display does not work well #13714

Thanks for this attempt to fix the problem. It looks like this breaks hit detection for standard image styles, see e.g. https://deploy-preview-13722--ol-site.netlify.app/examples/icon.html.

Thanks for reply
I find another way to fix the problem, change the custom renderer function. So I close the pull request and issue.
feature.setStyle(
  new Style({
    renderer(coordinates, state) {
      const scalePixel = function (value) {
        if (
          state.context.canvas.width === 1 &&
          state.context.canvas.height === 1
        ) {
          return value;
        } else {
          return state.pixelRatio * value;
        }
      };
      const [x, y] = coordinates;
      const ctx = state.context;
      ctx.beginPath();
      ctx.arc(x, y, scalePixel(30), 0, 2 * Math.PI, true);
      ctx.fillStyle = '#FFFF00';
      ctx.fill();
      ctx.strokeStyle = 'rgba(255,0,0,1)';
      ctx.stroke();
    },
  })
);


Account for rubocop split

We'll need to account to the decoupling of rubocop into separate gems
For instance, currently rubocop-performance is not found

I ran into this error trying to use rubocop-rspec

We'll need to account to the decoupling of rubocop into separate gems
For instance, currently rubocop-performance is not found

Can we read from .rubocop.yml and install the needed gems according to the values in require:?


Cluster Autoscaler (0.4.0) - Excessive calls to describeautoscalinggroup

We're still on 1.5.6 here and thus suck on 0.4.0
We are also in a corporate shared AWS account for just a few more weeks but we've been responsible for hammering the describe asg endpoint. At one count we had about 6200 calls an hour while watching just 2 autoscaling groups with a scan interval of 15s.
After reviewing the logs at verbosity 4 I'm seeing what seems like excessive cache regeneration log inc below.
I0411 00:18:03.743910       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196
I0411 00:18:03.767672       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-workers-asg
I0411 00:18:03.814842       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-subnet_workers-asg
I0411 00:18:03.847489       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-flink_workers-asg
I0411 00:18:03.878911       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196
I0411 00:18:03.909796       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-workers-asg
I0411 00:18:03.948900       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-subnet_workers-asg
I0411 00:18:03.971444       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-flink_workers-asg
I0411 00:18:04.014617       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196
I0411 00:18:04.062654       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-workers-asg
I0411 00:18:04.112767       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-subnet_workers-asg
I0411 00:18:04.135767       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-flink_workers-asg
I0411 00:18:04.179638       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196
I0411 00:18:04.208361       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-workers-asg
I0411 00:18:04.255125       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-subnet_workers-asg
I0411 00:18:04.285420       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-flink_workers-asg
I0411 00:18:04.315456       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196
I0411 00:18:04.338964       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-workers-asg
I0411 00:18:04.391533       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-subnet_workers-asg
I0411 00:18:04.414932       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-flink_workers-asg
I0411 00:18:04.439831       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196
I0411 00:18:04.475670       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-workers-asg
I0411 00:18:04.532452       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-subnet_workers-asg
I0411 00:18:04.565761       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-flink_workers-asg
I0411 00:18:04.596577       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196
I0411 00:18:04.628497       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-workers-asg
I0411 00:18:04.689757       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-subnet_workers-asg
I0411 00:18:04.711811       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-flink_workers-asg
I0411 00:18:04.741555       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196
I0411 00:18:04.767574       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-workers-asg
I0411 00:18:04.809850       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-subnet_workers-asg
I0411 00:18:04.849322       1 aws_manager.go:186] Regenerating ASG information for a0098-p18-flink_workers-asg
I0411 00:18:04.878316       1 aws_manager.go:186] Regenerating ASG information for tf-asg-00451965f3d6dfa3c713f1f196

I'm digging through the code trying to track down what's invalidating the cache so rapidly.

It looks like the cause here is that we have 7 node pools and only 4 under management. So every time it tries to evaluate a node that is outside of the four managed asgs it regenerates the entire cache.
I might suggest keeping another pool of nodes that aren't found in the cache after a regeneration and handle those as a separate 'early-out' type case.
For now I think I'll pull all asgs under management but the other 3 in a fixed (same min as max) size. Though this makes me nervous.

I guess we might have the similar problem with GCE/GKE. Need to take a look.

The issue is here:
https://github.com/kubernetes/contrib/blob/master/cluster-autoscaler/cloudprovider/aws/aws_manager.go#L173
But is not present in GCE:
https://github.com/kubernetes/contrib/blob/master/cluster-autoscaler/cloudprovider/gce/gce_manager.go

It is now:
https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/aws_manager.go#L173
And for GKE:
https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/gce/gce_manager.go#L191


[Feature][Task] Implement Task Template Feature

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
I'd like to propose the implementation of a Task Template feature,it would allow users to save an existing task node as a template and reuse it when creating new tasks.
The Task Template would include all the configuration and execution parameters of a task node, except for the scheduling parameters and custom variables. This feature would enable users to quickly create tasks based on a pre-defined template, abstracting the logic of the desired business code.
I believe that the implementation of this feature would enhance the user experience and increase the efficiency of task creation in Apache DolphinScheduler.
Thanks for considering this proposal.
Use case
No response
Related issues
No response
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi, @wendongdi , any updates?

I don't think this is a needed feature, right now if you use UI to create workflow, when you drag a task, there is already a task template, you only need to inject parameters in the form.


only initialize the framer once

Codecov Report

Merging #1678 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1678      +/-   ##
==========================================
- Coverage   86.99%   86.99%   -<.01%
==========================================
  Files          97       97
  Lines        6281     6280       -1
==========================================
- Hits         5464     5463       -1
  Misses        581      581
  Partials      236      236

Impacted Files
Coverage Δ

session.go
73.44% <ø> (-0.04%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd9baf9...d535340. Read the comment docs.


Update docs & fix #1036

Fixes #1036

Codecov Report
Base: 4.14% // Head: 6.86% // Increases project coverage by +2.71% :tada:

Coverage data is based on head (69f497c) compared to base (f3163b4).
Patch coverage: 23.03% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff            @@
##           master   #1048      +/-   ##
=========================================
+ Coverage    4.14%   6.86%   +2.71%
=========================================
  Files          24      26       +2
  Lines        7140    7255     +115
=========================================
+ Hits          296     498     +202
+ Misses       6844    6757      -87

Impacted Files
Coverage Δ

nanomq/apps/broker.c
23.50% <ø> (+5.52%)
:arrow_up:

nanomq_cli/client.c
0.00% <0.00%> (ø)

nanomq_cli/main.c
0.00% <0.00%> (ø)

nanomq_cli/nng_proxy.c
0.00% <0.00%> (ø)

nanomq_cli/nngcat.c
0.00% <0.00%> (ø)

nanomq_cli/proxy.c
0.00% <0.00%> (ø)

nanomq_cli/rest_api.c
0.00% <0.00%> (ø)

nanomq_cli/web_server.c
0.00% <0.00%> (ø)

nanomq/unsub_handler.c
52.94% <100.00%> (+52.94%)
:arrow_up:

nanomq/unsub_handler_test.c
100.00% <100.00%> (ø)

... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


isAdjIn should be set for both pre and post policy RIB-IN messages

https://github.com/OpenBMP/obmp-collector/blob/3663bf9d9203350980b742e27a9e9b6251d69428/Server/src/bmp/parseBMP.cpp#L393
This attribute should be getting set for both pre and post policy RIB-IN messages .
Thanks
Girish

Fixed in https://github.com/OpenBMP/obmp-collector/pull/4


Improved Swing/JavaFX interop

This PR includes fixes -- also from the scenery side -- that improve the interop between JavaFX and Swing, particularly:

dialog hangs (#138)
a race condition leading to invisible volumes (#139)

Hmm, using a JFrame for the main window? Didn't realize that was feasible. Probably makes the sc.iview.javafx.JavaFXMenuCreator obsolete... at least, we could switch to JMenuBar and then the menus will look like they do everywhere else. What do you guys think?

@ctrueden I'd like to stick with the JavaFX solution there, as it seems the better and more future-proof way. Also, all the GUI code on the scenery side is JavaFX, and I don't want to have more Swing there.

Additionally, the context menus need to be JavaFX, so it seems only consistent to have the menubar in JavaFX as well.


Can't install Iris in Python3 AWS Sagemaker environment

🐛 Bug Report
Install fails for Iris in the Python3 env on Sagemaker with The environment is inconsistent, please check the package plan carefully
Not sure if source of issue is AWS or Iris.
How To Reproduce
Steps to reproduce the behaviour:

Create a new Notebook under Sagemaker
Open with JupyterHub and choose conda_python3 kernel
Open a terminal and 'source activate python3`
Do usual conda install -c conda-forge iris
(grinds away and eventuall gives up with "The environment is inconsistent...")

Expected behaviour
Iris gets installed.
Screenshots
dump of session:
sh-4.2$ source activate python3
(python3) sh-4.2$ conda install -c conda-forge iris
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: |
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - conda-forge/noarch::nbclient==0.5.2=pyhd8ed1ab_0
  - conda-forge/linux-64::matplotlib==3.3.4=py36h5fab9bb_0
  - conda-forge/noarch::qdarkstyle==2.8.1=pyhd8ed1ab_2
  - conda-forge/linux-64::scikit-image==0.16.2=py36hb3f55d8_0
  - conda-forge/noarch::python-language-server==0.36.2=pyhd8ed1ab_0
  - conda-forge/linux-64::widgetsnbextension==3.5.1=py36h5fab9bb_4
  - conda-forge/noarch::flake8==3.8.4=py_0
  - conda-forge/noarch::ipywidgets==7.6.3=pyhd3deb0d_0
  - conda-forge/noarch::typing-extensions==<IP_ADDRESS>=0
  - conda-forge/noarch::path.py==12.5.0=0
  - conda-forge/noarch::dask==2021.2.0=pyhd8ed1ab_0
  - conda-forge/noarch::nbformat==5.1.2=pyhd8ed1ab_1
  - conda-forge/linux-64::path==15.1.2=py36h5fab9bb_0
  - conda-forge/linux-64::nbconvert==6.0.7=py36h5fab9bb_3
  - conda-forge/linux-64::distributed==2021.2.0=py36h5fab9bb_0
  - conda-forge/noarch::anaconda-client==1.7.2=py_0
  - conda-forge/noarch::aioitertools==0.7.1=pyhd8ed1ab_0
  - conda-forge/linux-64::matplotlib-base==3.3.4=py36hd391965_0
  - conda-forge/linux-64::pluggy==0.13.1=py36h5fab9bb_4
  - conda-forge/noarch::black==20.8b1=py_1
  - conda-forge/linux-64::blaze==0.11.3=py36_0
  - conda-forge/noarch::pyls-spyder==0.3.2=pyhd8ed1ab_0
  - conda-forge/noarch::odo==0.5.1=py_1
  - conda-forge/linux-64::keyring==22.0.1=py36h5fab9bb_0
  - conda-forge/noarch::anaconda-project==0.9.1=pyhd8ed1ab_0
  - conda-forge/noarch::importlib_metadata==3.7.0=hd8ed1ab_0
  - conda-forge/linux-64::jupyter==1.0.0=py36h5fab9bb_6
  - conda-forge/noarch::jupyterlab_server==2.3.0=pyhd8ed1ab_0
  - conda-forge/noarch::seaborn-base==0.11.1=pyhd8ed1ab_1
  - conda-forge/noarch::imageio==2.9.0=py_0
  - conda-forge/noarch::numpydoc==1.1.0=py_1
  - conda-forge/linux-64::yarl==1.6.3=py36h8f6f2f9_1
  - conda-forge/noarch::jsonschema==3.2.0=py_2
  - conda-forge/noarch::flask==1.1.2=pyh9f0ad1d_0
  - conda-forge/noarch::seaborn==0.11.1=hd8ed1ab_1
  - conda-forge/noarch::helpdev==0.7.1=pyhd8ed1ab_0
  - conda-forge/linux-64::nb_conda==2.2.1=py36h5fab9bb_4
  - conda-forge/noarch::nbclassic==0.2.6=pyhd8ed1ab_0
  - conda-forge/noarch::sphinx==3.5.1=pyhd8ed1ab_0
  - conda-forge/noarch::jupyterlab_launcher==0.13.1=py_2
  - conda-forge/linux-64::spyder==4.2.0=py36h5fab9bb_0
  - conda-forge/linux-64::importlib-metadata==3.7.0=py36h5fab9bb_0
  - conda-forge/linux-64::pytest==6.2.2=py36h5fab9bb_0
  - conda-forge/noarch::pyls-black==0.4.6=pyh9f0ad1d_0
| ^Cfailed with initial frozen solve. Retrying with flexible solve.

Keeps retrying and eventually gives up.
Environment

OS & Version: Linux ip-....-.amzn1.x86_64 #1 SMP Mon Nov 1 20:48:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Iris Version: Latest from conda-forge (didn't install)

Additional context

I note that there is a lot of JupyterLab in the inconsistency list. Was that in previous packaging of Iris? (particualrly the jupyterlab_server)

Click to expand this section...
Please add additional verbose information in this section e.g., code, output, tracebacks, screenshots etc

I've not used sagemaker before, but I can tell you that conda doesn't handle the situation of "install this big new set of dependencies into an existing environment" very well.  Do you have the ability to create new conda environments in sagemaker? Something like:
conda create -n iris_n_friends -c conda-forge iris ipykernel
That would help nail down if this is an issue specific to the existant environment, or an incompat with sagemaker in general.
FYI, these are the named dependencies of iris when adding to a conda env:
- python >=3.6
- cartopy >=0.18
- cf-units
- cftime >=1.5.0
- dask-core >=2
- matplotlib-base
- netcdf4 >=1.4
- numpy >=1.14,!=1.15.2
- scipy
- python-xxhash

I can create new envs per your line above - but interestingly pyhton then fails import iris once activated.
Sagemaker uses full anaconda rather than minconda per the recommended installation intructions. Might this be a factor ?
(I am creating a clean standard env and trying to do the line by line installs. WLYK)

Anaconda is essentially miniconda + 100s of packages installed by default from pkgs/main. So if that is the env you are activating it might not be surprising that it struggles to install iris from conda-forge.
conda create ... should make a completely new environment, isolated from the anaconda provided envs.

OK - sorted. Partly PICNIC, partly conda-forge incompatabiltities with installation in standard Sagemaker kernels.

I was using legacy source activate env rather than **conda** activate env which doesn't resolve all the environment links and executables, so that didn't help (and I'm not alone)
There are many inconsistencies being thrown between standard AWS envs and the conda-forge packages

Resolution

Create seperate env per above command: conda create -n iris_n_friends -c conda-forge iris ipykernel
Register as Jupyter kernel: python -m ipykernel install --user --name iris_n_friends --display-name "iris"
Select new kernel for notebook

I was doing this as a qucik/temporary thing. If you want the kernel to persist you need to create the env on the EBS volume for the Sagemaker instance and then configure the instance at start up to recognise and use it. See this set of instructions for example.
An alternative is to create a seperate containerised image (for Studio). See: https://aws.amazon.com/blogs/machine-learning/bringing-your-own-custom-container-image-to-amazon-sagemaker-studio-notebooks/


Move package to @mapbox namespace.

@mourner - you're the principal author. do you want to adopt this or are you okay with moving to @mapbox npm org?

Validated that this repository is managed under the organization in-place. Closing and whitelisting.


Update Azure Marketplace Wizard URL

Update README's URL for the Azure Marketplace Wizard to https://portal.azure.com/?pub_source=email&pub_status=success#create/microsoft-avere.vfxt-templateavere-vfxt-arm
Prior URL was to the preview version.
PS: My editor got rid of some trailing spaces, too.

Works for me. :)


How can I make video player responsive?

Hi. How can I make video player responsive?
I couldn't find anything about this in documentations and examples.

playerOptions: {         fluid: true, }


Nature of detectSignature method

I am trying to rewrite the detectSignature method from javascript in actionscript.  However, some of the fields used for this signature, are sometimes unavailable in the flash environment.  It the best practice, in this case, to return null for the signature or to try to build one based on those pieces of data that I am still able to get?

Hey @natilivni - that's a good question! I would say we should build a signature from as many pieces of data as are available. I think the question then is whether we use the existing signature field (confusing as it's not the same signature as the JS Tracker's) or create a new field in the flash_context? @yalisassoon thoughts?

For now, I am just implementing two separate signatures, one for cookies and the other for shared objects.

Please notice the following three new subject params that I have added in commit 8715a620301dc4b3d0509efd348a0c3a52ade8f9 :
https://github.com/snowplow/snowplow-actionscript3-tracker/blob/master/snowplow-as3-tracker/src/com/snowplowanalytics/snowplow/tracker/Parameter.as#L46-49
They are the flash shared object storage equivalent of their corresponding cookie based parameters.  @alexanderdean, please look at the naming convention and let me know if any changes should be made.

Hi @natilivni - these three fields look great, however we have a freeze on adding new fields to the Tracker Protocol (preferring custom contexts instead), so these fields will need to go into the flashContext instead. Suggested names:

so_vid -> domainSessionIndex
so_duid -> domainUserId
so_fp -> userFingerprint

These are based on the names found in the atomic.events, tweaked to make them a bit more JSON-idiomatic. @yalisassoon can you review these names and make any changes?
Also: I wasn't 100% sure about referring to domain in the property names but a quick read of the documentation suggests that Flash's shared object(s) are domain-specific, so quite analogous to first-party cookies...

Thanks @alexanderdean, I have changed the field names and moved them to the flash_context payload in commit 8787b36922814894cef27147ca9473b75fcce8a4.
@ephraimtabackman, can you please update the schema and push the patch to iglu?

I updated the schema and pushed to feature/flash_context in iglu.

I updated the schema and pushed to branch feature/flash_context in my fork. Please pull.

Done, https://github.com/snowplow/iglu-central/pull/120 This will be released later today...

Follow up question, can someone please provide me with a signature that is know to be correct in the javascript environment and the data conditions that are used to produce it.  I want to write some unit tests for the new code.  Same question for domainUserId.  Thanks!

Hey @fblundun - do we have any unit tests that check known browser fingerprints in known browser environments (e.g. in Selenium/SauceLabs)?
A separate point I forgot to mention, apologies @natilivni - we are planning on moving the JS Tracker's domainUserId to be a UUID. It's just simpler than creating a custom ID. The ticket is here: https://github.com/snowplow/snowplow-javascript-tracker/issues/274
We might as well start the Flash domain user ID as a UUID too...

There are some browser fingerprints used for SauceLabs tests here: https://github.com/snowplow/snowplow-javascript-tracker/blob/master/tests/functional/detectors.js#L49-L61

@alexanderdean, I think we should coordinate that for the same milestone.  It is important for users that the javascript and actionscript trackers produce the same results for shared fields, since often times, they are both part of the same web application.  Therefore, until the javascript tracker actually moves to uuid, we should keep this one the same.  It the timetable for that milestone short enough so that it can be release in conjunction with the actionscript tracker?

Hey @natilivni - the JavaScript Tracker is blocked on moving to UUID because the domain_userid field in Redshift is too short. The ActionScript Tracker's domainUserid in the flash_context doesn't have the same limitation (because it will go into a new flash_context table that we create).
However I totally see your point - if you are able to set both the duid (i.e. cookies are functional) and the flash_context.domainUserid from the Flash Tracker, we should want to set them to the same value. So I've created a ticket here: https://github.com/snowplow/snowplow-actionscript3-tracker/issues/21

@fblundun, do the comments in the signature test represent the full userAgent that is used in creating that signature?  For example when the code says:
1627396823, // Firefox 27.0 Mac
972065378,  // Firefox 27.0 XP
Does that mean that the value<PHONE_NUMBER> should be generated by the userAgent "Firefox 27.0 Mac" ?

That means that the value<PHONE_NUMBER> is the user fingerprint generated when SauceLabs runs the tests in Firefox 27.0 on Mac. The value will be influenced by which browser plugins are present. You can see the tests here: https://saucelabs.com/u/snowplow

I see.  in that case, it is still difficult for me to test the actionascript algorithm I wrote.  Since me local environment is difference than the one on saucelabs.  maybe we should create a task to run the actionscript tests in saucelabs?  @alexanderdean, what do you think?

Hey @natilivni - I think it would be cool to run a subset of the ActionScript tests (ones that care about the DOM I guess) in SauceLabs. It might be quite fiddly, but definitely cool. Ticket is here:
https://github.com/snowplow/snowplow-actionscript3-tracker/issues/22
Definitely non-essential though...

I have run the tests a few times, and I see that things are collecting in the "bad" bucket.  Can you guys help me make sense of the results?  Also, can you think of any other tests we can run for this new functionality?

Hey @natilivni - okay so you are getting some failures like this:
{"schema":"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-0","data":[{"test":"testMaxBuffer","mehh":["somebar","somebar"]}]}

this is an easy fix - just send through valid events even rather than these dummy payloads. You need to make sure to set these properties: "e", "p", "tv".
More concerning is this payload:
{
  "data": [
    {
      "eid": "e500c9fc-cb53-1210-3b69-a73e6098cfdb",
      "ti_qu": "1",
      "e": "ti",
      "schema": "iglu:com.snowplowanalytics.snowplow\/contexts\/jsonschema\/1-0-0",
      "ti_sk": "no_sku",
      "ti_cu": "USD",
      "ti_id": "order-8",
      "hasScriptAccess": false,
      "p": "pc",
      "aid": "cloudfront",
      "dtm": "1421072735697",
      "tv": "as3-0.1.0",
      "data": [
        {
          "data": {
            "someContextKey": "testTrackPageView2"
          },
          "schema": "iglu:com.snowplowanalytics.snowplow\/example\/jsonschema\/1-0-0"
        }
      ],
      "tna": "AF003",
      "hasLocalStorage": true,
      "version": "MAC 11,7,700,225",
      "ti_pr": "34",
      "ti_nm": "Big Order",
      "isDebugger": true,
      "playerType": "StandAlone",
      "ti_ca": "Food",
      "stageSize": "500x375"
    }
  ],
  "schema": "iglu:com.snowplowanalytics.snowplow\/payload_data\/jsonschema\/1-0-0"
}

This has a few things wrong with it:

The "schema" field following "e" and the "data" field following "tv" should be in a JSON object {} envelope, stringified, and either base64 encoded or URI-encoded. If base64-encoded, they should be assigned to a field "cx"; if URI-encoded, they should be assigned to a field "co".
The fields "hasLocalStorage", "isDebugger" etc should live inside a self-describing JSON, flash_context, which in turn lives inside a stringified JSON inside the array contained within co or cx

Think of the POST payload as containing an array of events which are each identical to a GET payload, except that every &name=value pair becomes a "name": "value" property inside the JSON.
You can see a valid POST payload here:
https://github.com/snowplow/snowplow/blob/master/3-enrich/scala-hadoop-enrich/src/test/scala/com.snowplowanalytics.snowplow.enrich.hadoop/good/CljTomcatTp2MultiEventsSpec.scala#L39
Here it is Base64-decoded:
{
  "data": [
    {
      "e": "pv",
      "tv": "py-0.5.0",
      "p": "pc",
      "dtm": "1410184736783",
      "url": "http:\/\/www.example.com",
      "cx": "eyJkYXRhIjogW3siZGF0YSI6IHsib3NUeXBlIjogIk9TWCIsICJhcHBsZUlkZnYiOiAic29tZV9hcHBsZUlkZnYiLCAib3BlbklkZmEiOiAic29tZV9JZGZhIiwgImNhcnJpZXIiOiAic29tZV9jYXJyaWVyIiwgImRldmljZU1vZGVsIjogImxhcmdlIiwgIm9zVmVyc2lvbiI6ICIzLjAuMCIsICJhcHBsZUlkZmEiOiAic29tZV9hcHBsZUlkZmEiLCAiYW5kcm9pZElkZmEiOiAic29tZV9hbmRyb2lkSWRmYSIsICJkZXZpY2VNYW51ZmFjdHVyZXIiOiAiQW1zdHJhZCJ9LCAic2NoZW1hIjogImlnbHU6Y29tLnNub3dwbG93YW5hbHl0aWNzLnNub3dwbG93L21vYmlsZV9jb250ZXh0L2pzb25zY2hlbWEvMS0wLTAifSwgeyJkYXRhIjogeyJsb25naXR1ZGUiOiAxMCwgImJlYXJpbmciOiA1MCwgInNwZWVkIjogMTYsICJhbHRpdHVkZSI6IDIwLCAiYWx0aXR1ZGVBY2N1cmFjeSI6IDAuMywgImxhdGl0dWRlTG9uZ2l0dWRlQWNjdXJhY3kiOiAwLjUsICJsYXRpdHVkZSI6IDd9LCAic2NoZW1hIjogImlnbHU6Y29tLnNub3dwbG93YW5hbHl0aWNzLnNub3dwbG93L2dlb2xvY2F0aW9uX2NvbnRleHQvanNvbnNjaGVtYS8xLTAtMCJ9XSwgInNjaGVtYSI6ICJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIn0=",
      "eid": "a91a31bc-593b-481f-869a-9d822000ee86"
    },
    {
      "se_ca": "my_category",
      "se_ac": "my_action",
      "e": "se",
      "tv": "py-0.5.0",
      "p": "pc",
      "dtm": "1410184736784",
      "cx": "eyJkYXRhIjogW3siZGF0YSI6IHsib3NUeXBlIjogIk9TWCIsICJhcHBsZUlkZnYiOiAic29tZV9hcHBsZUlkZnYiLCAib3BlbklkZmEiOiAic29tZV9JZGZhIiwgImNhcnJpZXIiOiAic29tZV9jYXJyaWVyIiwgImRldmljZU1vZGVsIjogImxhcmdlIiwgIm9zVmVyc2lvbiI6ICIzLjAuMCIsICJhcHBsZUlkZmEiOiAic29tZV9hcHBsZUlkZmEiLCAiYW5kcm9pZElkZmEiOiAic29tZV9hbmRyb2lkSWRmYSIsICJkZXZpY2VNYW51ZmFjdHVyZXIiOiAiQW1zdHJhZCJ9LCAic2NoZW1hIjogImlnbHU6Y29tLnNub3dwbG93YW5hbHl0aWNzLnNub3dwbG93L21vYmlsZV9jb250ZXh0L2pzb25zY2hlbWEvMS0wLTAifSwgeyJkYXRhIjogeyJsb25naXR1ZGUiOiAxMCwgImJlYXJpbmciOiA1MCwgInNwZWVkIjogMTYsICJhbHRpdHVkZSI6IDIwLCAiYWx0aXR1ZGVBY2N1cmFjeSI6IDAuMywgImxhdGl0dWRlTG9uZ2l0dWRlQWNjdXJhY3kiOiAwLjUsICJsYXRpdHVkZSI6IDd9LCAic2NoZW1hIjogImlnbHU6Y29tLnNub3dwbG93YW5hbHl0aWNzLnNub3dwbG93L2dlb2xvY2F0aW9uX2NvbnRleHQvanNvbnNjaGVtYS8xLTAtMCJ9XSwgInNjaGVtYSI6ICJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIn0=",
      "eid": "00da9e62-8a1c-4ffc-92a5-bc0d4fb797a9"
    },
    {
      "se_ca": "another_category",
      "se_ac": "another_action",
      "e": "se",
      "tv": "py-0.5.0",
      "p": "pc",
      "dtm": "1410184736784",
      "cx": "eyJkYXRhIjogW3siZGF0YSI6IHsib3NUeXBlIjogIk9TWCIsICJhcHBsZUlkZnYiOiAic29tZV9hcHBsZUlkZnYiLCAib3BlbklkZmEiOiAic29tZV9JZGZhIiwgImNhcnJpZXIiOiAic29tZV9jYXJyaWVyIiwgImRldmljZU1vZGVsIjogImxhcmdlIiwgIm9zVmVyc2lvbiI6ICIzLjAuMCIsICJhcHBsZUlkZmEiOiAic29tZV9hcHBsZUlkZmEiLCAiYW5kcm9pZElkZmEiOiAic29tZV9hbmRyb2lkSWRmYSIsICJkZXZpY2VNYW51ZmFjdHVyZXIiOiAiQW1zdHJhZCJ9LCAic2NoZW1hIjogImlnbHU6Y29tLnNub3dwbG93YW5hbHl0aWNzLnNub3dwbG93L21vYmlsZV9jb250ZXh0L2pzb25zY2hlbWEvMS0wLTAifSwgeyJkYXRhIjogeyJsb25naXR1ZGUiOiAxMCwgImJlYXJpbmciOiA1MCwgInNwZWVkIjogMTYsICJhbHRpdHVkZSI6IDIwLCAiYWx0aXR1ZGVBY2N1cmFjeSI6IDAuMywgImxhdGl0dWRlTG9uZ2l0dWRlQWNjdXJhY3kiOiAwLjUsICJsYXRpdHVkZSI6IDd9LCAic2NoZW1hIjogImlnbHU6Y29tLnNub3dwbG93YW5hbHl0aWNzLnNub3dwbG93L2dlb2xvY2F0aW9uX2NvbnRleHQvanNvbnNjaGVtYS8xLTAtMCJ9XSwgInNjaGVtYSI6ICJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIn0=",
      "eid": "115d56c5-ce0a-4fbe-a64f-830bc21effb8"
    }
  ],
  "schema": "iglu:com.snowplowanalytics.snowplow\/payload_data\/jsonschema\/1-0-0"
}

Thanks @alexanderdean !
I think that those are old entries.  None of the tests send the example schema any more.  The entries that we were curious about are from this week or the end of last week.

Ah! Sorry for confusion. I've taken a look:

In shredded/bad, I can't see any failures - so the JSON aspects seem to be working fine
In enriched/bad, I can't find any bad rows sent by the ActionScript Tracker. All the bad rows in there are just regular spam and hacking attempts

To turn the question around: what are you expecting to see in Redshift that you are missing?

Closed


Use another library to generate our documentation

We currently use keras-autodoc https://github.com/keras-team/keras-autodoc to generate the markdown for our documentation. We then use mkdocs to convert that to html.
Keras-autodoc isn't supported anymore, we need a remplacement before the CI starts breaking (shouldn't be urgent but we need to do it at some point). We should look into a replacement like https://github.com/NiklasRosenstein/pydoc-markdown or similar https://github.com/mkdocstrings/mkdocstrings
It doesn't matter if the documentation is different afterwards

Hey I could take a look at it! Will prepare the PR for this.

That would be great, thanks! Don't hesitate to ask if you need help or guidance!

It looks really good! Thanks @kclamalex !


👷 Actions: Add Docker build and push

Builds a Docker image and push to Docker Hub every time a tag is pushed.

Just for reference, was this publish script taken from another project where it's already working?

Just for reference, was this publish script taken from another project where it's already working?

Yeah, got it from here.
If you open the actions on my forks, you'll see it works (except last commit as I replaced the repo name by yours).
Forgot to specify you need to set the DOCKER_USERNAME and DOCKER_PASSWORD (which can be an access token), of course

Yeah, got it from here.
If you open the actions on my forks, you'll see it works (except last commit as I replaced the repo name by yours).
Forgot to specify you need to set the DOCKER_USERNAME and DOCKER_PASSWORD (which can be an access token), of course

Hmm, just looking at other usages we have where we are publishing and this came up: https://github.com/ppy/osu-notification-server/blob/master/.github/workflows/pack.yml
Probably worth either copying that for consistency, or if there's reason, applying the version you have here to all existing deployments.

Hmm, just looking at other usages we have where we are publishing and this came up: https://github.com/ppy/osu-notification-server/blob/master/.github/workflows/pack.yml
Probably worth either copying that for consistency, or if there's reason, applying the version you have here to all existing deployments.

Yeah, that's the first thing I looked into, then realized it wasn't up-to-date. I'll PR this to the other repos then.

Yeah, that's the first thing I looked into, then realized it wasn't up-to-date. I'll PR this to the other repos then.


Replace other-modules with exposed-modules for Hackage packages

Replace other-modules in the library section:
$ sdiff old new
library								library
  hs-source-dirs:						  hs-source-dirs:
    src								    src
  ghc-options: -Wall						  ghc-options: -Wall
  build-depends:						  build-depends:
    base >= 4.3 && < 5,						    base >= 4.3 && < 5,
    ghc-prim							    ghc-prim
  exposed-modules:						  exposed-modules:
    Data.Orphans						    Data.Orphans
  other-modules:					      |	  exposed-modules:
    Data.Orphans.Prelude					    Data.Orphans.Prelude
  default-language: Haskell2010					  default-language: Haskell2010

test-suite spec							test-suite spec
  type: exitcode-stdio-1.0					  type: exitcode-stdio-1.0
  main-is: Spec.hs						  main-is: Spec.hs
  hs-source-dirs:						  hs-source-dirs:
    test							    test
  ghc-options: -Wall						  ghc-options: -Wall
  build-depends:						  build-depends:
    base >= 4.3 && < 5,						    base >= 4.3 && < 5,
    base-orphans,						    base-orphans,
    hspec == 2.*,						    hspec == 2.*,
    QuickCheck							    QuickCheck
  other-modules:						  other-modules:
    Control.Applicative.OrphansSpec				    Control.Applicative.OrphansSpec
    Control.Exception.OrphansSpec				    Control.Exception.OrphansSpec
...

I'm sorry.
! @jystic
/jury approved @amosr

:100:

we probably want a way to disable it just in case?

yeah, except then some projects only builds with/without a particular mafia option, which might not be a good thing for boris.

As a user of mafia outside Ambiata, I don't think this is a good default. I have no problem with it as a --savage option, but its a really bad default.

One issue with making this optional is that it affects the packages which are kept in the global cache, which will mean that builds stop being reproducible between machines.

Ok, its its not optional, then I am pretty firmly against this.

As amusing, and potentially useful as this is, this feels like a large deviation from just being just a reliable cabal. I imagine at some point we would become transparently reliant on this behaviour in some places. I don't feel as strongly as Erik, but it does feel like we should be very clear about the consequences of exposing modules we don't normally have access to, and what that means long term, before we make this change.

I don't mind, this is more of an experiment than anything. Making it an option might help with quickly trying out things in dev but probably not much more beyond that.

This is quite challenging to make optional, I'm going to close this until we figure out a way.
I've renamed this branch to wip/sed.


new(driver): unify support for some new syscalls between drivers 1/n

What type of PR is this?
/kind bug
/kind feature
Any specific area of the project related to this PR?
/area driver-kmod
/area driver-bpf
/area driver-modern-bpf
/area tests
Does this PR require a change in the driver versions?
No
What this PR does / why we need it:
On one side this PR supports 3 missing syscalls in the modern bpf probe https://github.com/falcosecurity/libs/issues/723:

poll
ppoll
mmap2

On the other side, it unifies the behavior of this syscall between our 3 drivers:

send event even when the pointer is invalid

Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
new(driver): unify support for some new syscalls between drivers

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 4d22c2f2d17bae349e4236c2214a19dfe07a1dcc


Add metrics support

Fix the relevant metrics configs
Add manifests to deploy with prometheus
Add E2Es for metrics

One nit, overall looks good.
Would it make sense to add statistics related to the config status / config events as metallb does?

Would it make sense to add statistics related to the config status / config events as metallb does?

LGTM


InvalidOperationException while using AzureSignalRService with MultiTenant class in Hub

I've created a sample application which uses multi tenancy using Saaskit Multitenancy. and inject the multi tenant class as dependency injection in Hub.
Project works fine when Azure SignalR Service isn't used. but while using Azure SignalR Service it doesn't work. and gives following error in few seconds after signalR service gets connected.
signalr.js:1881 Error: Connection disconnected with error 'Error: Server returned an error on close: Connection closed with an error. InvalidOperationException: Unable to resolve service for type 'WiredBrain.MultiTenant.AppTenant' while attempting to activate 'WiredBrain.Hubs.CoffeeHub'.'.
Git Hub Repro
You need to replace {AzureServiceEndPointConnectionString} with the Azure SignalR Service EndPoint Connection String in ConfigureServices method of Statup Class.
There is another branch "multitenantwithoutAzureService". and SignalR works fine in there.

I am afraid this third-party package SaasKit.Multitenancy to handle multi-tenancy is not supported when using Azure SDK with its current way dealing with DI.
Under the hood, this SaasKit.Multitenancy actually registers a HTTP middleware, and when a new HTTPrequest comes in, it registers the object to DI if necessary, according to https://andrewlock.net/loading-tenants-from-the-database-with-saaskit-in-asp-net-core/#configuring-the-services-and-middleware
However when it comes to Azure SDK, there is no new http request invoked. All the connections from client goes to service and then goes into an already-established server-service connection and then directly create the required Hub from DI dynamically. There is nowhere for the middleware of SaasKit.Multitenancy to be invoked.
One alternative way is to add singleton AppTenantResolver when ConfigureServices, and resolve the AppTenant inside the Hub. remember to remove the UseMultitenancy line in Configure. Do notice that the Context.GetHttpContext().Request.Host.Value is the ASRS endpoint "sgtest.service.signalr.net" because the client is actually talking to ASRS.
If you want to pass app server url, e.g. localhost:60907, through ASRS to the Hub, you can set the value from client in the URL query string and fetch it from the Hub through: connectionBuilder.WithUrl("/coffeehub?url=localhost:60907")

Closing this as the solution is provided.


WSN Transport Bug : shouldEchoBinaryFrameWithPayloadLength0 and shouldEchoTextFrameWithPayloadLength0

Test shouldEchoBinaryFrameWithPayloadLength0 uses this K3PO script.
Test shouldEchoTextFrameWithPayloadLength0 uses this K3PO script
Tests are added in gateway pull #236
File: BaseFramingIT.java & @Ignore has this issue's title.
Error is that script is expecting 'read [0x82 0x00]', actually getting 'Disconnected'. The GATEWAY is not echoing 0 length payload.

This issue will address many ignores in pull #236, besides just these two. 4 tests in FragmentationIT.java also fail because the gateway throws an exception when echoing an empty payload.

Changed bug title from the original "WSN Transport Bug : shouldEchoBinaryFrameWithPayloadLength0 andshouldEchoTextFrameWithPayloadLength0" because we get the same issue with the WSE spec tests (for example: BinaryIT.shouldEchoFrameWithPayloadLength0).
Debugging shows the underlying cause is that the AbstractIoSession.write method so as of IllegalArgumentException if the input data buffer has zero length, i.e. !hasRemaining().

I reproduced the issue. I'm trying to figure out the impact of removing the throw statement from the AbstractIoSession.write method.

@cmebarrow Looking at the gateway project, I can't seem to understand why there's a mina.core module and a mina.build  one. Could you please shed some light?
Also, if I were to change the write() method to no longer throw IllegalArgumentException, I would need to do it in both mina.core and mina,netty, correct?
Thanks!

@cmebarrow I opened this PR, so you can see my changes and discuss whether this is the way to go. The mina modules are used all over the gateway, so I'm not sure of the impact that this change might have. Thanks!

@jitsni Jitu, could you take a look at this and answer @danibusu 's questions? I am very busy with BPCE right now. I think his approach is ok except I am not sure about the lowest level (nio/tcp). Allowing a write of a zero length buffer might not make sense for that.


return correction in ...plotter/plot_chart()

With this commit and merge request I am testing my repository access rights, please reject the pull request, in case this correction doesn't make sense.
This is related to the issue reported in DPF-Post repository - Issue # 15.
Thanks!

With this commit and merge request I am testing my repository access rights

PRs by anyone are permitted, but to merge you'll need to be part of the developers group, the CI tests need to be passing, and you'll need at least one approving review.

With this commit and merge request I am testing my repository access rights

PRs by anyone are permitted, but to merge you'll need to be part of the developers group, the CI tests need to be passing, and you'll need at least one approving review.

@rlagha, when you get a chance, could you please look over this?


Feature request: Highlight jmp addresses in the address list in the left part of the disassembly

00007FF849E4E25D  jmp         DialogBox2+10Ch (07FF849E4E26Bh )
00007FF849E4E25F  lea         rcx,[rsp+30h]    ^^^^^^^^^^^^^^  <--- selected
00007FF849E4E264  call        TranslateMessage (07FF849E26190h)
00007FF849E4E269  xor         edx,edx
00007FF849E4E26B  lea         rcx,[rsp+30h]
    ^^^^^^^^^^^^  <- highlighted

Love the asm-dude!

Yes, good idea, should not be very complicated to add that. With a bit of luck this weekend.

Thanks for pointing this out. Is indeed handy. Version <IP_ADDRESS> has this features.

It works so well!
Great job!


REST API does not appear to be thread safe

Which version are you using?
v0.17.2
Which operating system are you using?
OS

[X] Linux
[ ] Windows
[ ] macOS

Architecture

[X] amd64
[X] arm64
[X] arm7
[ ] arm6

Describe the issue
First of all, let me just say thanks to you for this great project.  I had started down the path of using an RTSP library in Node to add streaming support to my project and then I found out about rtsp-simple-server and I believe it has saved me a significant amount of work implementing capabilities that are built in here.
One issue I did run into, I'm using the REST API to dynamically add paths for cameras during startup and it works well, however, because my project is in Javascript and the initial creation of objects uses async methods, if a user has more than a few cameras the API can get hit with quite a few calls that are nearly at the same instant.  I've found that if you do two or three calls at nearly the same instant you will sometimes see one of the paths being "created" and then immediately "destroyed" because it appears the configuration reload for one API call somehow interferes with the call in progress from another.  The more calls made concurrently the more likely the problem is to occur.
For now I've worked around the issue on by moving the path creation logic to a synchronous function and adding a 100ms delay between API calls (perhaps could be significantly less, I haven't tried to see how low it could go, my guess is probably just a few ms is enough).  After that, path creation works 100% of the time.
Describe how to replicate the issue
Just try to create 3-5 paths via the API at nearly the same instant, I can whip up some sample Javascript if you really need it, but the problem is pretty easy to reproduce although it was a little hard to troubleshoot as it exhibited as just random paths being not available for some users, but looking at the logs the issue was clear.

Thanks for the issue, i've found a way to replicate and a fix is coming soon.
for i in $(seq 1 50); do (curl -d '{}' http://localhost:9997/v1/config/paths/add/testing$i &); done

This is fixed in main and will be added to the next release.

added in v0.16.3


Amit Kumar Garg

Inauguration of the CircleCI app Was did by myself .
Deserve for CircleCI development or not.

Expectation  !

Hopefully !

Issue .

Design .

For.

Progress .

OK .Alright .

Celebrate .

Together .

When .

In free time .

Leave it .today

Whoa what just happened?


LLM: define common/system-specific event structure

Area(s)
llm
Is your change request related to a problem? Please describe.
See https://github.com/open-telemetry/semantic-conventions/pull/825#discussion_r1533040974
#825 is going to introduce LLM events and (currently) requires to put prompts and completions as json string in OpenAI format. Some LLM vendors might not use json and instrumentation would be forced to use specific structure.
We should decide if:

we define a common format and all LLM vendor instrumentations are forced to use it.
we want one event with OpenAI JSON messages format (or another common format) and other vendors are forced to use it.
vendor-specific events. E.g. gen_ai.{vendor}.prompt and then vendors  could document their own structure or document that they emit OpenAI-formatted events
we make format dynamic with additional attributes representing mime type.

This is closely coupled with https://github.com/open-telemetry/semantic-conventions/issues/829
Describe the solution you'd like
TBD

Based on the discussion in this thread:

we should consider recording individual choices as events rather than completions
finish_reason should become a string rather than array and appear on events - this way it matches openai response structure
we can still add more attributes or event payload fields to describe the structure of choice


modules: hal_nordic: Fix inclusions in nrfx glue layer

Update the hal_nordic module revision so that the following commit
is applied:
nrfx_glue: Move #include directives outside of extern "C" blocks
To prevent compilation errors when some C++ elements, like a
template definition, happen to appear in those included files or files
included by them.
Fixes #23850.
https://github.com/zephyrproject-rtos/hal_nordic/pull/33 needs to go in first.

The required update of the nordic_hal module will be brought by #24062.


How to deblur a video?

Thanks for your excellent work.
I have found that the demo is capable of deblurring images using a set of three images.
However, I am wondering if you could offer some guidance on how to deblur a video instead.
Would it be appropriate to deblur each frame using information from the previous and next frames?
I appreciate your help with this matter.

Would it be appropriate to deblur each frame using information from the previous and next frames?
Yes, it should be easy.
You can extract each frame from the video and modify the inference code slightly, e.g., just using a loop to infer with three frames in order.

Thanks for your help!


druid集成clickhouse

druid.version：1.2.8
clickhouse-jdbc-version：0.3.1
Cause: java.sql.SQLException: sql injection violation, dbType clickhouse, , druid-version 1.2.8, syntax error: syntax error, expect AS, actual , pos 66, line 1, column 66, token , : SELECT COUNT() FROM (SELECT month AS month_date, cast(month_date, 'String') AS monthStr, concat(substring(monthStr, 1, 4), '-', substring(monthStr, 5, 6)) AS date, user_id, any(card_no) AS card_no, any(name) AS name, any(age) AS age, concat(toString(any(position_level)), '0') AS positionLevel, sumMerge(send_apply_times) AS send_apply_times, sumMerge(add_external_num) AS add_external_num, sumMerge(send_message_times) AS send_message_times, sumMerge(chat_total) AS chat_msg_nums, minMerge(start_day) AS start_day, maxMerge(end_day) AS end_day, dateDiff('day', start_day, end_day) + 1 AS num, toDecimal32(sumMerge(first_reply_average_duration) / num, 2) AS first_reply_average_duration, toDecimal32(sumMerge(replied_chat_rate) / num, 2) AS replied_chat_rate, sumMerge(delete_member_customer_num) AS delete_member_customer_num, argMaxMerge(customor_num) AS external_total_num, sumMerge(use_status) AS use_day_num, sumMerge(send_message_single_num) AS send_message_single_num, sumMerge(send_message_group_num) AS send_message_group_num, sumMerge(send_message_moment_num) AS send_message_moment_num, sumMerge(add_chat_num) AS add_chat_num, argMaxMerge(chat_total_num) AS chat_total_num, sumMerge(chat_has_msg_num) AS chat_has_msg_num, sumMerge(add_member_num) AS add_member_num, argMaxMerge(member_total_num) AS member_total_num, sumMerge(member_has_msg_num) AS member_has_msg_num, sumMerge(msg_total_num) AS msg_total_num, sumMerge(start_yx_aide_times) AS start_yx_aide_times, any(market_id) AS market_id, any(area_id) AS area_id, any(company_id) AS company_id, any(market_name) AS market_name, any(area_name) AS area_name, any(company_name) AS company_name, any(card_four) AS card_four, any(card_name_four) AS card_name_four, any(card_seven) AS card_seven, any(card_name_seven) AS card_name_seven FROM micr_scrm.partner_month_detail_statistics_base GROUP BY month_date, user_id) AS temp WHERE month_date BETWEEN 202111 AND 202111
; uncategorized SQLException for SQL []; SQL state [null]; error code [0]; sql injection violation, dbType clickhouse, , druid-version 1.2.8, syntax error: syntax error, expect AS, actual , pos 66, line 1, column 66, token , : SELECT COUNT() FROM (SELECT month AS month_date, cast(month_date, 'String') AS monthStr, concat(substring(monthStr, 1, 4), '-', substring(monthStr, 5, 6)) AS date, user_id, any(card_no) AS card_no, any(name) AS name, any(age) AS age, concat(toString(any(position_level)), '0') AS positionLevel, sumMerge(send_apply_times) AS send_apply_times, sumMerge(add_external_num) AS add_external_num, sumMerge(send_message_times) AS send_message_times, sumMerge(chat_total) AS chat_msg_nums, minMerge(start_day) AS start_day, maxMerge(end_day) AS end_day, dateDiff('day', start_day, end_day) + 1 AS num, toDecimal32(sumMerge(first_reply_average_duration) / num, 2) AS first_reply_average_duration, toDecimal32(sumMerge(replied_chat_rate) / num, 2) AS replied_chat_rate, sumMerge(delete_member_customer_num) AS delete_member_customer_num, argMaxMerge(customor_num) AS external_total_num, sumMerge(use_status) AS use_day_num, sumMerge(send_message_single_num) AS send_message_single_num, sumMerge(send_message_group_num) AS send_message_group_num, sumMerge(send_message_moment_num) AS send_message_moment_num, sumMerge(add_chat_num) AS add_chat_num, argMaxMerge(chat_total_num) AS chat_total_num, sumMerge(chat_has_msg_num) AS chat_has_msg_num, sumMerge(add_member_num) AS add_member_num, argMaxMerge(member_total_num) AS member_total_num, sumMerge(member_has_msg_num) AS member_has_msg_num, sumMerge(msg_total_num) AS msg_total_num, sumMerge(start_yx_aide_times) AS start_yx_aide_times, any(market_id) AS market_id, any(area_id) AS area_id, any(company_id) AS company_id, any(market_name) AS market_name, any(area_name) AS area_name, any(company_name) AS company_name, any(card_four) AS card_four, any(card_name_four) AS card_name_four, any(card_seven) AS card_seven, any(card_name_seven) AS card_name_seven FROM micr_scrm.partner_month_detail_statistics_base GROUP BY month_date, user_id) AS temp WHERE month_date BETWEEN 202111 AND 202111; nested exception is java.sql.SQLException: sql injection violation, dbType clickhouse, , druid-version 1.2.8, syntax error: syntax error, expect AS, actual , pos 66, line 1, column 66, token , : SELECT COUNT(*) FROM (SELECT month AS month_date, cast(month_date, 'String') AS monthStr, concat(substring(monthStr, 1, 4), '-', substring(monthStr, 5, 6)) AS date, user_id, any(card_no) AS card_no, any(name) AS name, any(age) AS age, concat(toString(any(position_level)), '0') AS positionLevel, sumMerge(send_apply_times) AS send_apply_times, sumMerge(add_external_num) AS add_external_num, sumMerge(send_message_times) AS send_message_times, sumMerge(chat_total) AS chat_msg_nums, minMerge(start_day) AS start_day, maxMerge(end_day) AS end_day, dateDiff('day', start_day, end_day) + 1 AS num, toDecimal32(sumMerge(first_reply_average_duration) / num, 2) AS first_reply_average_duration, toDecimal32(sumMerge(replied_chat_rate) / num, 2) AS replied_chat_rate, sumMerge(delete_member_customer_num) AS delete_member_customer_num, argMaxMerge(customor_num) AS external_total_num, sumMerge(use_status) AS use_day_num, sumMerge(send_message_single_num) AS send_message_single_num, sumMerge(send_message_group_num) AS send_message_group_num, sumMerge(send_message_moment_num) AS send_message_moment_num, sumMerge(add_chat_num) AS add_chat_num, argMaxMerge(chat_total_num) AS chat_total_num, sumMerge(chat_has_msg_num) AS chat_has_msg_num, sumMerge(add_member_num) AS add_member_num, argMaxMerge(member_total_num) AS member_total_num, sumMerge(member_has_msg_num) AS member_has_msg_num, sumMerge(msg_total_num) AS msg_total_num, sumMerge(start_yx_aide_times) AS start_yx_aide_times, any(market_id) AS market_id, any(area_id) AS area_id, any(company_id) AS company_id, any(market_name) AS market_name, any(area_name) AS area_name, any(company_name) AS company_name, any(card_four) AS card_four, any(card_name_four) AS card_name_four, any(card_seven) AS card_seven, any(card_name_seven) AS card_name_seven FROM micr_scrm.partner_month_detail_statistics_base GROUP BY month_date, user_id) AS temp WHERE month_date BETWEEN 202111 AND 202111

怎么解决的？


fix EleutherAI/gpt-neox-20b does not work in tgi

@OlivierDehaene OR @Narsil  @danieldk please help review.
found gpt-neox-20b does not work in tgi. there's shape issue in rotary_emb
https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt_neox/modeling_gpt_neox.py#L221-L233
add similar logic in tgi could fix the crash.

@drbh

Hi @sywangyi, thanks for the contribution! Just tested with EleutherAI/gpt-neox-20b locally and it loads and generates as expected 🙏


Sort all imports alphabetically

Fixes: https://github.com/timothycrosley/isort/issues/271
As it is a bit of an outsider from the rest of the methods I decided to use this approach (basically doing all the sort by myself).
I hope it's acceptable. I event wrote a test, which, btw. was easy and fun! Thanks for having a nice test suite easy to extend!

@timothycrosley friendly ping

I think this looks great! Thanks for putting the time into this, and sorry I've been busy / didn't get back sooner, will be including this in the next release.
~Timothy

I would love a release for this, as it's a feature I'm looking to use. I also hope this is a supported config option and not just a command line option.


Evaluate Kuery async changes on TechEmpower at scale

Context & Description
Evaluate the asynchronous changes to Kuery and Kuery-PostgreSQL using the updated TechEmpower benchmarks.
This will allow the /queries and /updates routes to function recursively, rather than the current workaround of using /updatesParallel and /queriesParallel (executing multiple finds in parallel, and using a semaphore to keep track of when all have completed - which blocks a thread).
Background
The issue with the recursive definition for updates and queries is that, because closures are currently invoked synchronously, the connection used for query N is not returned to the pool until the end of that closure - unavailable for query N+1 (invoked from within the closure).
For queries > 1 and many concurrent clients, eventually the benchmark will deadlock, because there are a number of clients waiting for a connection that they can never have because the pool has become empty, and preventing the connections that were just used from returning to the pool.
eg:
djones6@gruffalo:~$ wrk -c256 -t2 -d10s -i1s http://localhost:8080/queries?queries=3
Running 10s test @ http://localhost:8080/queries?queries=3
  2 threads and 256 connections
   1s: 7337 requests in last 1s (   7337.0 req/sec)
   2s: 7501 requests in last 1s (   7501.0 req/sec)
   3s: 7524 requests in last 1s (   7524.0 req/sec)
   4s: 7472 requests in last 1s (   7472.0 req/sec)
   5s: 7434 requests in last 1s (   7434.0 req/sec)
   6s: 7369 requests in last 1s (   7369.0 req/sec)
   7s: 7203 requests in last 1s (   7203.0 req/sec)
   8s: 6991 requests in last 1s (   6991.0 req/sec)
   9s: 80 requests in last 1s (     80.0 req/sec)
  10s: 0 requests in last 1s (      0.0 req/sec)
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    33.53ms    3.87ms 234.91ms   81.76%
    Req/Sec     3.68k   296.36     4.03k    95.65%
  58911 requests in 10.03s, 256 connections, 15.08MB read
Requests/sec:   5873.08
Requests/conn:   230.12
Transfer/sec:      1.50MB

Clear & Actionable Tasks

[ ] Test TechEmpower using https://github.com/IBM-Swift/Swift-Kuery-PostgreSQL/tree/issue_async
[ ] Evaluate queries and updates under stress, up to 20 queries and 256 concurrent clients

Exit Criteria:
This issue is complete once the following criteria are met:

Code...
Tests...

The current issue_async changes for Kuery and KueryPostgres do not behave well under load when there are more clients than available Dispatch threads (64 on Mac).
@kilnerm and I analyzed the behaviour.  In this scenario, all threads eventually become blocked waiting for a connection from the connection pool, because the (offloaded) execute calls do not have a thread available to run on, but are holding connections.
The solution we think is to make ConnectionPool.getConnection() asynchronous, where its completion will execute immediately if a connection is available, otherwise it will be enqueued and executed when a connection returns to the pool.

Yesterday we tested the latest iteration of the issue_async changes, including the redesigned ConnectionPool.getConnection(completion:) with task queueing.
These changes worked very well on macOS under Xcode, with the queue operating as expected, without any serious performance issue (in debug mode under Xcode).
Next step is to repeat the stress test on Linux in release mode, and get some sense of how it performs.

Changes passed testing on Linux and have been merged into their respective next branches.
The changes for the ORM are ready to test, and I will be doing so today.

ORM changes (including async result fetcher changes) all work correctly under stress on Mac and Linux.  Benchmark changes are in this branch: https://github.com/IBM-Swift/Kitura-Benchmarks/tree/issue_async_results
Once Kuery 3.0 is tagged, these changes can be pushed up to the TechEmpower project.


hostPathVolume.hostPath mounts the wrong path?

What is version of KubeSphere DevOps has the issue?
3.3.0
How did you install the Kubernetes? Or what is the Kubernetes distribution?
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.10", GitCommit:"7e54d50d3012cf3389e43b096ba35300f36e0817", GitTreeState:"clean", BuildDate:"2022-08-17T18:26:59Z", GoVersion:"go1.17.13", Compiler:"gc", Platform:"linux/amd64"}
What happened?
My K8s version:
root:containers#kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.4", GitCommit:"e6c093d87ea4cbb530a7b2ae91e54c0842d8308a", GitTreeState:"clean", BuildDate:"2022-02-16T12:38:05Z", GoVersion:"go1.17.7", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.10", GitCommit:"7e54d50d3012cf3389e43b096ba35300f36e0817", GitTreeState:"clean", BuildDate:"2022-08-17T18:26:59Z", GoVersion:"go1.17.13", Compiler:"gc", Platform:"linux/amd64"}

the run time is: containerd:
root@master01:~# ctr version
Client:
  Version:  1.6.8
  Revision: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
  Go version: go1.17.13

Server:
  Version:  1.6.8
  Revision: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
  UUID: 1306ee9c-57fb-4ce8-9339-ee12956adac6

My kubesphere version

in the agent,
according to this document: https://kubesphere.com.cn/forum/d/3811-kubesphere-devops-docker, I think it should be mounted here /var/lib/containers as cache, not /var/run/docker.sock ? It's also possible that I misunderstood something.
jenkins:
  mode: EXCLUSIVE
  numExecutors: 0
  scmCheckoutRetryCount: 2
  disableRememberMe: true

  clouds:
    - kubernetes:
        name: "kubernetes"
        serverUrl: "https://kubernetes.default"
        skipTlsVerify: true
        namespace: "kubesphere-devops-worker"
        credentialsId: "k8s-service-account"
        jenkinsUrl: "http://devops-jenkins.kubesphere-devops-system:80"
        jenkinsTunnel: "devops-jenkins-agent.kubesphere-devops-system:50000"
        containerCapStr: "2"
        connectTimeout: "60"
        readTimeout: "60"
        maxRequestsPerHostStr: "32"
        templates:
          - name: "base"
            namespace: "kubesphere-devops-worker"
            label: "base"
            nodeUsageMode: "NORMAL"
            idleMinutes: 0
            containers:
            - name: "base"
              image: "<IP_ADDRESS>/kubesphere/kubesphere/builder-base:v3.2.0-podman"
              command: "cat"
              args: ""
              ttyEnabled: true
              privileged: true
              resourceRequestCpu: "100m"
              resourceLimitCpu: "4000m"
              resourceRequestMemory: "100Mi"
              resourceLimitMemory: "8192Mi"
            - name: "jnlp"
              image: "<IP_ADDRESS>/kubesphere/jenkins/inbound-agent:4.10-2"
              args: "^${computer.jnlpmac} ^${computer.name}"
              resourceRequestCpu: "50m"
              resourceLimitCpu: "500m"
              resourceRequestMemory: "400Mi"
              resourceLimitMemory: "1536Mi"
            workspaceVolume:
              emptyDirWorkspaceVolume:
                memory: false
            volumes:
            - hostPathVolume:
                hostPath: "/var/run/docker.sock"
                mountPath: "/var/run/docker.sock"
            - hostPathVolume:
                hostPath: "/var/data/jenkins_sonar_cache"
                mountPath: "/root/.sonar/cache"

Relevant log output
No response
Additional information
No response

hi @usernameisnull thanks for contacting us, especially for providing the details. But I still have a couple of questions that hopefully you could help me have a better understanding.

What is the container runtime? docker or containerd ?

it should be mounted here /var/lib/containers as cache, not /var/run/docker.sock

/var/run/docker.sock is not a cache directory. It is the default docker sock file path. Currently, we didn't provide the corresponding config file according to different container runtime. Users have to change it by themself.

/close