Add crystal-expanded & crystal-normalized emit options

From time to time I want to see the generated Crystal code at several specific stages of the Crystal compiler:

all the code after all top level macro expansions
the code of one class (or one file) after all top level macro expansions
the code after semantic pass
the code that will actually be compiled into binary (normalized & with all dead code removed)
...

For a start, I think it would be nice to have

--emit crystal-expanded: dump the crystal code with all macros expanded
--emit crystal-normalized: dump the crystal code after the normalization pass (which transforms all syntactic sugar to basic control structures, like unless -> if, until -> while, etc..).
Also at this stage, the literals would already be expanded (e.g: string interpolation to String.build ...)
This dump would be quite hard to read I think, but would still be Crystal code (raw llvm IR is also nice, but it's another lower level)

(or with crystal => cr to reduce size of flags)
Those options could be used with --no-codegen, and the resulting files be compiled again into binary if needed (as they are valid Crystal code).

It's just ideas on how to inspect what's going on in the compiler, there are many things I didn't talked about that would be cool (e.g: after the type inference pass, or simply the list of symbols/unique string literals, ..)
Also in the long run it could help creating new stdlib (see what is needed), help during ports to new platforms (see what's used?), and in a way, inspecting into the compiler (via emit or tools, or compiler plugins) could allow to make #921 possible? (I think I'm going a bit too far for this issue, but I'd love all those things!!)

This feature would be very useful, use-case here: https://github.com/crystal-lang-tools/vscode-crystal-lang/issues/4

I just got another idea, I think printing typed code would be very useful, by example:
Original code: foo.cr
FOO_METHODS = %w(foo bar)

{% for method in FOO_METHODS %}
def {{method.id}}(baz)
end
{% end %}

foo("foo")
bar(42)

Macros expanded: foo.expanded.cr
def foo(baz)
end

def bar(baz)
end

foo("foo")
bar(42)

Typed code: foo.typed.cr
def foo(baz : String) : Nil
end

def bar(baz : Int32) : Nil
end

foo("foo")
bar(42)

This can be very useful for editor extensions and tools :+1:
Maybe a flag like --emit crystal-typed ? :sweat_smile:

Another use-case is tracking macro errors, by example, I'm implementing new error pages for amber framework, and sometimes I can't see the code snippet for an error because the original file is a macro. If I can save the expanded macro code, somewhere I think I could show better snippets:

The compiler shows macro source on errors...

@asterite Yeah, but would be nice to save expanded macros to a file, so I can read them and do nice things like debugging the expanded macro.
Regarded to saving intermediate code with inferred types  to a file (a.k.a. --emit-typed-crystal), this would be very useful for dev tools and editor extensions, to get faster/better completion and debugging.
By example, currently to get types and do some type debugging, I need to execute crystal tool context on every location I need (-c LOC, --cursor LOC) even if the code is unmodified. With something like --emit-typed-crystal I just run it once to get type info for all the project (until code is changed).
So, WDYT? :sweat_smile:

The compiler shows macro source on errors...

@asterite About the new amber error pages, I guess I need to improve them :wink:
Currently I'm doing this:

Catches error output
Parses error output to get locations /file/path:line:column and some basic error info
Reads file locations to get code snippets (this is the tricky part, because expanded macros aren't saved to a file)
Rendering ECR template with some error data and code snippets

So, I can't see expanded macro for a macro error because the location doesn't exist. :sweat_smile:

If the error is inside a macro, you can show it in a popup. But you usually need the whole trace.
Maybe someone can implement the typed output, but what's the input for that? The problem with Crystal is that to type code you need a main file, and it can be a program, a spec file, many spec files, etc. Do you specify a single main file in your editor? Will you cache the info per main file? If you are viewing some random file, which main file will you use?
I said it lately a couple of times: if you want the feature of every other statically compiled language, Crystal should be compiled modularly, that is, compile a file independently from other files. But that will never happen. So I personally don't think Crystal is a language for which a good IDE will exist. But if you manage to do it, than it'll be quite a feat :-)

@asterite Thank your for your response!

But you usually need the whole trace.

Yeah, that's why I'm doing some refactoring in amber watch to make it pretty configurable, so, people can use --error-trace flag if they want :wink:
Also, I added the Show raw message option, so, developers can always read the full raw error message.

The problem with Crystal is that to type code you need a main file

Hehe, In fact, I have a mainFile config for my editor :sweat_smile:
https://github.com/crystal-lang-tools/vscode-crystal-lang/wiki/Settings#mainfile

So I personally don't think Crystal is a language for which a good IDE will exist. But if you manage to do it, than it'll be quite a feat :-)

No problem :sweat_smile: I still think crystal community can do some nice things to make scry and other crystal tools better and faster :muscle:
For now, I just need something like @bew suggested, during compilation before code generation phase, allow us to emit:

Expanded macro code: --emit-crystal-expanded
Normalized crystal code --emit-crystal-normalized:
Typed crystal code: --emit-crystal-typed

Parse:                             00:00:00.000102665 (   0.19MB)
Semantic (top level):              00:00:00.922344024 (  84.57MB)
Semantic (new):                    00:00:00.004823701 (  84.57MB)
Semantic (type declarations):      00:00:00.122542724 (  84.57MB)
Semantic (abstract def check):     00:00:00.021798212 (  84.57MB)
Semantic (ivars initializers):     00:00:00.461345153 ( 124.63MB)
Semantic (cvars initializers):     00:00:00.020537517 ( 124.63MB)
Semantic (main):                   00:00:03.310385641 ( 301.00MB)
Semantic (cleanup):                00:00:00.016460185 ( 301.00MB)
Semantic (recursive struct check): 00:00:00.002591033 ( 301.00MB)
>>>>>>>>>>>>> Emit code just right here I guess :-) <<<<<<<<<<<<
Codegen (crystal):                 00:00:03.523409110 ( 318.50MB)
Codegen (bc+obj):                  00:00:00.708628038 ( 326.50MB)
Codegen (linking):                 00:00:02.285434906 ( 326.50MB)

We already have emit flag. Perhaps we can add more options to it, like
--emit [asm|llvm-bc|llvm-ir|obj|cr-macros|cr-normalized|cr-typed]
Comma separated list of types of output for the compiler to emit

WDYT?

Could you show what output you expect from each, and how are you going to process it?

Ok, Given a simple file /home/user/Projects/example/foo.cr with the code:
FOO_METHODS = %w(foo bar)

{% for method in FOO_METHODS %}
def {{method.id}}(baz)
  p baz
end
{% end %}

foo("foo")
foo(3.141516)
bar(42)

and compiled with the following command:
crystal build foo.cr --no-codegen --emit cr-expanded --emit cr-normalized --emit cr-typed

The compiler can generate these files:
└── Projects
    └── example
        ├── foo.cr
        ├── foo.expanded.cr
        ├── foo.normalized.cr
        └── foo.typed.cr

NOTE: I know that generated files won't look as beautiful as I think, because they require the prelude and a lot of code from the stdlib, although, they are still pretty useful for debugging and developing other crystal tools. Perhaps, in the future we can do some enhancements. I do this because we need something to start with.

First foo.expanded.cr would looks like:
# ...
# ...
# ...A bunch of code above added by crystal compiler
def foo(baz)
  PrettyPrint.format(baz, STDOUT, 79)
  STDOUT.puts
  baz
end

def bar(baz)
  PrettyPrint.format(baz, STDOUT, 79)
  STDOUT.puts
  baz
end

foo("foo")
foo(3.141516)
bar(42)

# A bunch of code below added by crystal...
# ...

and finally, foo.typed.cr would be generated like:
# ...
# ...
# ...

For a project with many files I think we can merge it in one big file depending on what file is being compiled, for example, for the following project:
└── example
    ├── src
    │   ├── spec_helper.cr
    │   └── example_spec.cr
    └── src
        ├── example
        │   ├── version.cr
        │   ├── buzz.cr
        │   ├── fizz.cr
        │   ├── foo.cr
        │   └── bar.cr
        └── example.cr

we can do something like:
crystal build src/example.cr --emit cr-expanded --emit cr-normalized --emit cr-typed

and get something like:
└── example
    ├── src
    │   └── ...
    └── src
    │   ├── example
    │   │   └── ...
    │   └── example.cr
    ├── example               # => binary file
    ├── example.expanded.cr   # => macros expanded
    ├── example.normalized.cr # => code normalized
    └── example.typed.cr      # => code typed

or If this is a bit noise then I think we can use a tmp directory:
└── example
    ├── src
    │   └── ...
    ├── src
    │   ├── example
    │   │   └── ...
    │   └── example.cr
    ├── tmp
    │   ├── example.expanded.cr   # => macros expanded
    │   ├── example.normalized.cr # => code normalized
    │   └── example.typed.cr      # => code typed
    └── example                   # => binary file

Maybe we can start saving expanded macros to a file using crystal tool expand src/example.cr and generating tmp/example.expanded.cr which can include source code with expanded macros for all required files inside example project.
Another option for generating tmp/example.typed.cr is saving typed source code when using crystal tool context src/example.cr which can include source code with typed stuff for all required files inside example project.
And as I said before. I know this intermediate code can be very dirty/noise, although, IMO this is still very useful for debugging and other nice things, like improving analyzing and auto-completion for a project.
And finally, I know this intermediate "generated" code can change every time I do some code change, no problem with that, because would be still very useful until I edit some file :wink:
So, WDYT? :sweat_smile:

Not sure. As I said many times, the compiler doesn't work file-by-file, it slurps all the files and works on that at once. So generating instantiated code per file maybe could work, but it's not trivial. And then there's macros, which expanded don't belong to any file. And then, not sure why you would need the expansions... I mean, they are not typed, it seems they just have the method argument and return types.
Plus it's a lot of work (今時間がない). I think anyone could grab the compiler's source and do it if they wanted, it's similar to other late-passes the compiler has like the type hierarchy.

Not sure. As I said many times, the compiler doesn't work file-by-file, it slurps all the files and works on that at once. So generating instantiated code per file maybe could work, but it's not trivial. And then there's macros, which expanded don't belong to any file

Perhaps we can output a big file with all the stuff (exanded macros, normalized ad typed code) :smile:

And then, not sure why you would need the expansions... I mean, they are not typed, it seems they just have the method argument and return types.

debugging generated source code is is the main use-case, see: https://github.com/crystal-lang-tools/vscode-crystal-lang/issues/4

Plus it's a lot of work (今時間がない). I think anyone could grab the compiler's source and do it if they wanted, it's similar to other late-passes the compiler has like the type hierarchy.

Don't worry, no problem about that, I'm glad you read my comments and sent us nice responses :sparkles: :tada:
I guess the crystal community (or even myself) could invest some time and understand the compiler enough to implement such features :wink:

Oh, I just found LiteralExpander, so, now I understand the --emit normalized code proposal @bew :+1:

Both the normalizer and the literal expander operate over AST nodes and should not require semantic analysis, so they could be options to crystal tool expand, or simply be exposed by a new tool. (expand was presumably added way after this issue was created.)
My use case is to add this and crystal tool format as Crystal tools for Compiler Explorer, simply because the output will be interesting to look at. (For C/C++ they already have clang-format support, so this makes sense.)


[Experiment] Add North-South UDP Protocol Support on Istio Ingress Gateway

Describe the feature request
Add basic UDP protocol support on Istio Ingress Gateway for north-south traffic management.
Describe alternatives you've considered
For many reasons, istio does not support udp for a long time, but I think we need basic udp traffic management in north-south side. At least, I think we need to support routing udp traffic to target backend svc.
I tried to use EnvoyFilter to dynamically add/update/delete udp listener/proxy in istio-ingressgateway. Adding/Removing udp listener is easy, but there is one big problem is that I think we did not discovery udp services/endpoints, I have tested to create a couple of udp services, but cannot find the endpoints in istio discovery endpoints data. So I need to list/watch svc/endpoints in clusters, and dynamically add/remove endpoints in EnvoyFilter which patches Cluster config of istio-ingressgateway.
Affected product area (please put an X in all that apply)

[ ] Ambient
[ ] Docs
[ ] Installation
[x] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Affected features (please put an X in all that apply)

[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane

Additional context

Given Istio's general limited UDP support, can you help explain why you want to use it compared to other existing UDP ingresses?

Background is that we are one of the platform providers for istio, and we are using istio as the north-sourth/west-east full traffic management solution. Unlike individual team, bringing another ingresses/gateways component though can solve it, like the envoy-gateway I am maintaining, but this will increase much cost of operation, as we need to introduce a new component just for udp traffic and need to operate.
@howardjohn

@Xunzhuo I'm so interested in this idea, but providing more powerful comparisons as a reference for operation.

I am very confused about Istio partial support for udp
@Xunzhuo You could try using ServiceEntry and "fake" the udp or just expose the UDP port directly through another load balancers since i assume you already have a load balancer ready...
I am trying to setup a custom DNS server for use internally but alas the only valid option aside from using envoy filters to redirect the UDP directly to the service is to simply dedicate a UDP load balancer are run the Service itself in LoadBalancer mode...
What confuses me is why Gateway supports UDP but virtual service does not... For the love of me i cant figure out how to route traffic from ingressgateway to the service directly ( even if it means bypassing istio - as they only route it at the moment )
@howardjohn maybe you have some insights?
P.S: here is an envoy that should work in theory
`apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: udp-route
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: LISTENER
patch:
operation: MERGE
value:
name: "<IP_ADDRESS>_53"
udp_listener_config:
udp_listener_name: raw_udp_listener
stat_prefix: udp

applyTo: CLUSTER
patch:
operation: ADD
value:
name: "raw_udp_cluster"
connect_timeout: "10s"
type: STATIC
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: "raw_udp_cluster"
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: dnsmasq.ovpn.svc.cluster.local
port_value: 53`

There is no UDP support in istio at all. This is tracked in https://github.com/istio/istio/issues/1430.

address: dnsmasq.ovpn.svc.cluster.local

I have just followed the [blog](https://www.zhaohuabing.com/istio-guide/docs/tricks/udp-listener/]. However, DNS notation doesn't seem to work.
You need to use the services internal  fix IP.... I can confirm that this is a big problem to not support discovery.
Given that EnvoyGateway supports UDP, I don't know why a project as big as Istio isn't able to provide UDP support. It's really sad since it's a game changer.

address: dnsmasq.ovpn.svc.cluster.local

I have just followed the [blog]([https://www.zhaohuabing.com/istio-guide/docs/tricks/udp-listener/]](https://www.zhaohuabing.com/istio-guide/docs/tricks/udp-listener/]). However, DNS notation doesn't seem to work.
You need to use the services internal fix IP.... I can confirm that this is a big problem to not support discovery.
Given that EnvoyGateway supports UDP, I don't know why a project as big as Istio isn't able to provide UDP support. It's really sad since it's a game changer.

Hi, @lightoyou
Can you give me some usage scenarios for UDP? I also have similar needs and am currently exploring. If you have more usage scenarios, I look forward to your reply.

Hi Samzong,
My scenarios are "limited", I would like to expose game servers/a public DNS server and be able to benefit from the telemetry offered by Istio.
I would like to use Istio as a Mesh service and profit of the Gateway.  I don't want to install another Gateway that supports UDP...
In addition, projects such as Knative, which supports the Istio gateway by default, are being considered for downscaling to zero some service.
Regards.


It's better to simplify the structure of the code

Thank you very much for the code. It really helps me a lot. But the main code has 900+lines, felt it a little tired to read. Anyway, thank you very much, if there is ant pytorch version, that's really great!

Dear @BeamReflection,
I understand that the main code can be a little overwhelming, especially for people who are new to this field.
However, please note that our main code intentionally follows the design decisions of Minnen et al. ICIP 2020 (Google Research), which also has about 700+ loc.
MRIC builds on top of this project and adds some additional complexity. In general, I think it's a good idea to familiarize yourself with tensorflow-compression first before moving on to more advanced variants.

I understand. Thank you very much for your work!

Dear @BeamReflection,
I understand that the main code can be a little overwhelming, especially for people who are new to this field.
However, please note that our main code intentionally follows the design decisions of Minnen et al. ICIP 2020 (Google Research), which also has about 700+ loc.
MRIC builds on top of this project and adds some additional complexity. In general, I think it's a good idea to familiarize yourself with tensorflow-compression first before moving on to more advanced variants.

I understand. Thank you very much for your work!

Hello, in the config.py, there is a lpips_path, and it points to
res/data/model/lpips_weights. But in this project, it seems that there is
not such a lpips_wights file in the directory:
res/data/model/lpips_weights. Because I want to run the project om my own
PC, not in Google Colab. So, where should I I download this file?
Nikolai Körber @.***> 于2024年8月8日周四 16:44写道：

Closed #2 https://github.com/Nikolai10/MRIC/issues/2 as completed.
—
Reply to this email directly, view it on GitHub
https://github.com/Nikolai10/MRIC/issues/2#event-13802572219, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/BJBYNS5VFU4YYVFK5JRC46DZQMOWBAVCNFSM6AAAAABMD2BXW2VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTHAYDENJXGIZDCOI
.
You are receiving this because you were mentioned.Message ID:
@.***>


adding empty measures causes the dashboard to not load properly

This makes a lot of sense that these wouldn't load properly ~ there's no information for the dashboard to use to query for time series.
To reproduce:

add a bunch of measures in the metrics config, but don't do anything with them (no expression)
go to dashboard

Was fixed in one of the PRs


Main README: Coming Soon - Simple Applications (Appliance Mode)

As a prospective user of Zarf
I want to know how Zarf can help me "just run apps"
So that I can understand & convince others of Zarf's applicability

Write new docs/appliance-mode.md page describing what "appliance mode" means & "what you get" by using it
Update "Simple Applications - Coming Soon" link in main README to point to new page

Spun off from #47

Blocked by #237

Going to re-evaluate these stale doc issues, closing for now.


Fixed typos in sortQueryParams docs

Hello, I found a typo at the README.md of your repo. Specifically in the sortQueryParameters example the initial URL is www.sindresorhus.com?b=two&a=one&c=three and the output http://sindresorhus.com/?b=two&a=one&c=three. The query params are not sorted

Nevermind, I misread the docs


[Kafka sink] upgrade from 4.0.15 to 5.0.6: kafka server: Message was too large, with canal protocol

What did you do?

Cluster v4.0.16, broker topics: bin/kafka-topics.sh --create --topic cdc-events --config max.message.bytes=1073741824 replica.fetch.max.bytes=1073741824 fetch.message.max.bytes=1073741824 --bootstrap-server <IP_ADDRESS>:9092
Workload is sysbench
Create changefeed: tiup ctl:v4.0.15 cdc changefeed create --pd=http://<IP_ADDRESS>:2379 --sink-uri="kafka://<IP_ADDRESS>:9092/cdc-events?kafka-version=2.4.0&partition-num=1&protocol=canal&replication-factor=1" --changefeed-id="kafka-task-4"
Upgrade cluster to v5.0.6

What did you expect to see?
No response
What did you see instead?
"id": "kafka-task-4",
"summary": {
"state": "error",
"tso":<PHONE_NUMBER>64353024,
"checkpoint": "2021-12-24 22:07:31.396",
"error": {
"addr": "<IP_ADDRESS>:8300",
"code": "CDC:ErrProcessorUnknown",
"message": "[CDC:ErrKafkaAsyncSendMessage]kafka: Failed to produce message to topic cdc-events: kafka server: Message was too large, server rejected it to avoid allocation error."
}
Versions of the cluster
Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):
(paste TiDB cluster version here)

TiCDC version (execute cdc version):
(paste TiCDC version here)

we have to fix canal protocol.

We has tested open-protocol and canal-json protocol, they works fine. As canal is still in experimental stage, change to severity/minor.

Close it since it is stale.


azure-cosmosdb-spring-boot-starter version 3.0.0 not in maven

As mentioned in https://github.com/MicrosoftDocs/azure-dev-docs/commit/22d75cfe8b6841ce17b8aecdf6f5410b59bf01ae
3.0.0-beta.1 is not in maven.
Latest version in maven is (2.3.3); which downloads azure-cosmos version 3.7.3 and that creates issues alongside azure cosmos v4
If this new verison (3.0.0) have the fix, then please add this to maven central.

@vermacodes thanks for the feedback! We'll look into this.

This has been fixed.


Add 'insert_no_copy' mode

While the copy fallback is very sensible and alleviates the problems in #53, I (and so am assuming is the case for others) am fortunately not affected by it. As such, the copying of the inserted emoji acts only to pollute the state of my clipboard, which is undesirable.
This patch therefore exposes a new mode insert_no_copy that does exactly as expected.
Personally, I'd argue that it should be insert, copy and insert_and_copy, setting the default entry to insert_and_copy, but I can imagine there are some people who don't rely on the default (i.e. they specify -emoji-mode) but are still expecting the copy. This preference is weak enough to not warrant that risk.
Hoping to get this merged as I am currently using rofi -modi emoji -show emoji -emoji-mode stdout | tr -d '\n' | wtype - as a workaround, so looking forward to hearing your thoughts.

This change is live both on the master branch but also on the 3_x_stable branch, and a new 3.4.0 release has just been made.

Great! Thanks for the merge and the project as a whole. I'll let you add to the changelog so I don't have to make another PR if that's alright though 🙂


[Bug] Windows Smart App Control blocked instalation

What did I do
I'm trying to install MQTTX.
What happened
I cannot install the app on Windows 11. I get an error message from Smart App Control.
Environment

OS: Windows 11 Insider
MQTTX version, 1.9.1

More detail
I have a similar problem to issues only with Windows.

Hi.Thanks for your feedback. Sorry, we do have this security issue, we currently don't have security signatures on Windows but it still works, maybe you can start MQTTX as an Administrator or remove the security restrictions.

I can't install it with administrator privileges. I won't turn off Smart App Control because of this, because there is no way to re-enable it without reinstalling.
In order to ensure a more secure experience, we only enable Smart App Control on clean installs of Microsoft Windows 11. We want to be sure that there aren't already untrusted apps running on the device when we turn Smart App Control on.
source: support.microsoft.com


Support to proxy Websocket to Coap server

Added functions to support proxying Websocket connect to Coap broker server. Also, fixed typo of some md files. :)

Thanks for adding the important missing thing. ;-)


Deleted spaces are not refreshed in/removed from the displayed list of recent spaces until a user logs out

Screenshot #1 - After the space is created

Screenshot #2 - After the space is deleted

This has been fixed.


Let HashProbe keep track of memory consumption when listing join results

Hash probe currently has limited memory control when extracting results from the hash table. When a small number of large sized rows from the build side is frequently joined with the left side, the total extracted size will explode, making HashProbe using a large amount of memory. And the process of filling output is not in spillable state, and will often cause OOM.
This PR computes the total size when listing join results in hash probe if there are any variable size columns from the build side that is going to be extracted. It stops listing further when it reaches the maximum size. This can help to control hash probe side memory usage to a confined limit.

Our TPCDS performance regression nearly 5%, do we need to re-consider this feature?
All the small query slows and q24a and q24b slows very much. After revert this one, performance recovered.   TPC benchmarks are important externally and we want to keep it stay SoTA

@jinchengchenghh This is an important fix for Meta internal traffic that prevents certain queries from OOMing. I will patch a fast path for the feature to improve the performance, as @Yuhta mentioned.

Thanks very much. @tanjialiang @Yuhta

@tanjialiang @Yuhta thanks for the quick turnaround. Please also note the benchmark results is done with TPCDS SF3000. Haven't done a larger scale test yet but based on experiences the performance drop may be bigger in that case
thanks, -yuan

@Yuhta @tanjialiang Is the performance regression on track from your side? Is there any following work to fix it?


 Also fixes up the generated package comment error mentioned in https://github.com/helm/helm/pull/6334#issuecomment-531805500.
supersedes #6334

@bacongobbler If we use golang:1.13, we'll always get the latest version in the 1.13 series right? I did that for Helm 3 here but there we use circleci/golang image and they also give latest in 1.13 series. I just checked

If we intentionally don't want the latest always and want to know what version is being used to build, then we need to change it in Helm 3 too. As currently it always gets latest


Cannot see Zabbix query

Hi
My zabbix plugin version is 4.2.5
Grafana version is 8.3.6
Zaabix version is 5.4.5
When i create a panel and choose zabbix as  a datasource i do not see any of the query fields. Its just shows a blank query screen,  If it is an existing panel i can see the data but can see the fields or source details. Please see screenshot of the same.

Any idea how to fix this

is your install dockerized or how is it installed?
is there any files here? /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/partials/

Yes it is dockerized
The following files are present in /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/partials/
annotations.editor.html
query.editor.html

same problem

I'm having the same issue!

@jinucompute try copying those file from /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/partials/ to
/var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/partials/ then restart the container.
note the double datasource-zabbix/datasource-zabbix/

@Dark-ZBX same question to you, linux or windows? and dockerized or standalone?
if linux I would suggest trying the above.

@jinucompute try copying those file from /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/partials/ to /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/partials/ then restart the container. note the double datasource-zabbix/datasource-zabbix/

I did this, but it did not work. When i restart the container the folder /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/  gets deleted

add this to the dockerfile at the end.
then rebuild and bring back up.
RUN mkdir -p /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/
CP /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/* /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/

@Dark-ZBX same question to you, linux or windows? and dockerized or standalone? if linux I would suggest trying the above.

@Sam-H101 it is running on windows server 2016

add this to the dockerfile at the end. then rebuild and bring back up.
RUN mkdir -p /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/ CP /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/* /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/

Apologies but i dont work with the docker file. I use the following command to create the container
docker run -d \
	--name=grafana \
	--restart unless-stopped \
	-p 3000:3000 \
	-e "GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource,alexanderzobnin-zabbix-app,marcusolsson-dynamictext-panel" \
	-e "GF_LOG_MODE=console file" \
	-v grafana-data:/var/lib/grafana \
	-v grafana-config:/etc/grafana \
	-v grafana-def-config:/usr/share/grafana/conf \
	grafana/grafana:latest

Is there another way to achieve what you have said.
Thanks

@jinucompute
you can git pull the master branch
confirm permissions to src/datasource-zabbix/partials is 644 and user/group should be the container user (1010 in my case)
then add
    -v <master_location>/src/datasource-zabbix/partials: /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/partials
I do not think this will invalidate the plugin hash, but if it does you will need to remove the manifest file, and load it unsigned.
Let me know if that works, I am not too up to date without using dockerfiles & docker-compose
for windows let me build an environment.

@Dark-ZBX @wilamu
I just tested on windows
Below steps assume you are using the default plugins directory in data\plugins. if not replace with the directory you are using.

goto <grafana install dir>\data\plugins\alexanderzobnin-zabbix-app\datasource-zabbix
Make new folder datasource-zabbix inside the above directory
Copy the partials folder to inside that directory you just made
final path will look like  <grafana install dir>\data\plugins\alexanderzobnin-zabbix-app\datasource-zabbix\datasource-zabbix\partials
in my case I did not need to bounce grafana, but you may.

RUN mkdir -p /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/
CP /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/* /var/lib/grafana/plugins/alexanderzobnin-zabbix-app/datasource-zabbix/datasource-zabbix/

I spent a lot of time trying to build the container with the RUN and CP command in dockerfile. It does not accept the CP command as a recognizable command and the build fails. I then went back to your original suggestion and created the folder and copied the files there, but i did not bounce the container. This worked.
only thing is everything the container is restarted I will have to do these steps.
Thanks a lot for your help. Things are back to normal. Will there be a permanent fix for this in a future release ?

Thanks for your support, it's working!  but I have another problem with the zabbix problems panel, is not working,
any fix for this?

@Dark-ZBX same question to you, linux or windows? and dockerized or standalone? if linux I would suggest trying the above.

@Sam-H101 it is running on windows server 2016

@Dark-ZBX @wilamu I just tested on windows
Below steps assume you are using the default plugins directory in data\plugins. if not replace with the directory you are using.

goto <grafana install dir>\data\plugins\alexanderzobnin-zabbix-app\datasource-zabbix
Make new folder datasource-zabbix inside the above directory
Copy the partials folder to inside that directory you just made
final path will look like  <grafana install dir>\data\plugins\alexanderzobnin-zabbix-app\datasource-zabbix\datasource-zabbix\partials
in my case I did not need to bounce grafana, but you may.

Hi! Try to upgrade Grafana, this issue is happening in some 8.3.x versions. I haven't noticed it in 8.4.

Thanks to everyone for your support, its Working!
Performed the following steps:
-Install grafana 8.5 on Windows Server 2019
-Install Plugin 2.4.6 via CMD (C:\Program Files\GrafanaLabs\grafana\bin grafana-cli plugins install alexanderzobnin-zabbix-app)
-Go to C:\Program Files\GrafanaLabs\grafana\data\plugins\alexanderzobnin-zabbix-app\datasource-zabbix and create new folder with name , "datasource-zabbix"  by @Sam-H101
-copy the "partials" folder and copy inside the created folder "datasource-zabbix"
-Delete MANIFEST  file on C:\Program Files\GrafanaLabs\grafana\data\plugins\alexanderzobnin-zabbix-app
-add:   allow_loading_unsigned_plugins = alexanderzobnin-zabbix-app, alexanderzobnin-zabbix-datasource, alexanderzobnin-zabbix-triggers-panel in the configuration file (config.ini for Windows Version)
-Restart Grafana Service
Regards!

Hi! Try to upgrade Grafana, this issue is happening in some 8.3.x versions. I haven't noticed it in 8.4.

same problem does not appear in add source
Grafana 8.5.2
Zabbix Plugin 4.2.8


processor(ticdc): replace tableID with Span in sourcemanager and sinkmanager

What problem does this PR solve?

Issue Number: ref #7720
What is changed and how it works?
Replace tableID with Span in sourcemanager and sinkmanager
Check List
Tests

Unit test
Integration test

Questions
Will it cause performance regression or break compatibility?
No.
Do you need to update user documentation, design documentation or monitoring documentation?
No.
Release note
None

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

@overvenus The CI failed. Could you please take a look?

@overvenus The CI failed. Could you please take a look?

Sure, I find following issues so far, and I don't think they are related to this PR.

https://github.com/pingcap/tiflow/issues/7986
https://github.com/pingcap/tiflow/issues/7983

/run-all-tests

/run-all-tests

/run-integration-test

/run-all-tests

/run-all-tests

/run-all-tests

/merge

/run-integration-test

/run-integration-test
/run-kafka-integration-test

/run-kafka-integration-test

/run-all-tests


Where exactly is the settings folder?

I tried adding the install file to "C:\Users\username\AppData\Local\GWToolboxpp". When I start the game, it begins writing in "C:\Users\Documents\GWToolboxpp". Is the document folder the default or can I change where the program saves my settings?

Documents\GWToolboxpp is the toolbox folder, Documents\GWToolboxpp<ComputerName> is the settings folder. These cannot be changed.

Documents\GWToolboxpp is the toolbox folder, Documents\GWToolboxpp is the settings folder. These cannot be changed.

TY.


Fix Candidates method file match

strings.HasSuffix(file.GetName(), loc.FileName) could match the wrong file if there are multiple files with the same name. Instead, set the fileName field to the full file path and use that.

🙏


Support ArrayOfEntity so one can create bound entities like ActivityParties on create

This allows a user to create entities like:
from1 = DynamicsCRM::XML::Entity.new('activityparty')
from1.attributes = DynamicsCRM::XML::Attributes.new({partyid:
DynamicsCRM::XML::EntityReference.new("systemuser", @employee.id)})
to1  = DynamicsCRM::XML::Entity.new('activityparty')
to1.attributes = DynamicsCRM::XML::Attributes.new({partyid:
DynamicsCRM::XML::EntityReference.new("lead", @lead.id) })

body = { subject: "Subject for your phonecall", description: "Description", from: [from1], to: [to1] }
@call = @client.create('phonecall', body)

I haven't added a test for any of this yet, wanted thoughts first :)

like? dislike?

@jheth ping!

Hey @mbhnyc, thanks for the contribution and sorry for the delay.  Yes, I like expanding support for new XML attributes/types.  Looks like you added a spec already. Was there any other work you wanted to do?
I have not used ArrayOfEntity before so I'll rely on you for the support.

Thanks for the contribution!

👍


support query service?

Hi Dear Author,
Thanks for releasing this brilliant KG! I just wonder if there is any API deployment using your KG as the backend data, and supporting users to query this KG? For example, by querying some evidence (entity names) and relations, the API can find corresponding answers from the KG, probably using some machine learning techniques. I'm curious whether you've built such a "query service"?

Hello,
Thanks for reaching out with this wonderful idea! I am a full time PhD student so I have not focused on building an API for this resource -- however, if you are interested, I would love to collaborate on this effort and support as best as I can!
Best,
Payal


#30 Fix XHTML Issue - Backend Option changed from html to xhtml5

Changed Backend-Option to XHTML5 (to be compatible with Confluence API). Add @Ignore to tests breaking on my machine.

pull request build failure due to gpg key issues, investigating ...

Builds of pull requests from forks fail due to security restrictions with secret variables (see https://docs.travis-ci.com/user/pull-requests/#Pull-Requests-and-Security-Restrictions). Please rebase your pull request branch onto the current master in order to get commit ba4f702c which fixes this issue. Thank you for your patience!

See https://github.com/alainsahli/confluence-publisher/issues/62 for an explanation of the failed build. Once https://github.com/alainsahli/confluence-publisher/issues/62 has been fixed, please rebase your pull request branch onto the master and push the changes. Thank you for your patience!

Issue #62 is now fixed, please rebase your pull request branch onto the master of the confluence publisher repo and push the changes. Thank you!

Coverage increased (+3.5%) to 90.16% when pulling 31321a53f6f03f2a3c2bb69fa83b75f65e27241a on klaus31:bugfix/issue-30 into 442f07b7f392a816fc032bb20d9852b61a8a1315 on alainsahli:master.

Trying to merge your pull request, I realized that your commits are on the pull request twice:

Maybe something went wrong while rebasing your changes onto the master? Anyway, may I kindly ask you to create a new branch from an up-to-date master, apply your commits again and create a new pull request (or update the existing one)? Then I can merge that one cleanly into our master.
Alternatively, if you prefer, I can also cherry-pick your changes onto a feature branch in our repo and merge from there (you will not loose any credits). Thank you!

Hi @klaus31, thanks for your contribution. We are going to release 0.2.0 today so I applied your changes in PR #77.


serial: Shouldn't mangle pid when ISIG is changed

Summary
Let's check ISIG flag directly instead since pid is sent by user and the modification will surprise user.

@xiaoxiang781216 - I would need more context here.

@xiaoxiang781216 - I would need more context here.

from https://man7.org/linux/man-pages/man3/termios.3.html:
ISIG   When any of the characters INTR, QUIT, SUSP, or DSUSP are
              received, generate the corresponding signal.

When ISIG is clear, serial driver should skip the signal generation code. The old implementattion achieve this behaviour by changing pid to -1, but it isn't good to modify the value provided by user without caution. So this PR check ISIG directly instead.


s.monitor('top.CLOCK') error

Hello Tianzhu,
Thanks for your great work on making SystemC simulation easy.
While following the instructions, I got the following error during simulation.
s.monitor('top.CLOCK')
Traceback (most recent call last):
File "", line 1, in
File "/apps/hosted/tools/python/anaconda3-2019.10/lib/python3.7/site-packages/bsmedit/bsm/sim.py", line 258, in monitor
obj = self.objects.get(name, None)
AttributeError: 'NoneType' object has no attribute 'get'
My platform is as follows:

OS: Ubuntu 18.04
gcc : gcc-7.4.0
python : anaconda3-2019.10

Can I get your help?
Thanks,
Chaekwan

Thanks for the report, Chaekwan.  It looks like the simulation is not created successfully. How do you create your simulation? Is your path to your simulation valid?  You can call "is_valid" to confirm.
>>> s = simulation(None, './examples/libstart.so')
>>> s.is_valid()
True

And v3.2.1 has some other bugs. Can you try the master branch?
The doc/example is also a little bit old. The "current" way to set the "spin" control style is:
s.monitor('top.CLOCK')
p = s.monitor('top.sig_steps')
p.SetFormatter(fmt.IntFormatter(256, 2**16)) # min=256, max=2**16
p.SetControlStyle('spin')


improve Docs

Hi,
It looks like you have opened a lot of one or two word PRs to projects, possibly for hacktoberfest.
Your change is missing a space, the formatting looks weird and I'm not sure it adds much.
"Improve Docs" is a bit too general as a comment on the commit, a good commit message should be useful to somebody browsing the history in git.
If you are genuinely interested in ark I would spend some time with it and find something meaningful to contribute.
Project owner: I would label this as "invalid", and then it doesn't count towards hacktoberfest, user has been spamming multiple projects.
Best Regards
Stuart


trim-*: discard unmatched

This cutadapt parameter controls if unmatched reads should be discarded - this would be pretty useful to wrap (and straightforward).
This recently came up on the forum.

@thermokarst Would --untrimmed-output untrimmed.fastq help in this case?  It might be easier to add this flag and simply write to another output file. Even if it ends up being empty. As  --discard-untrimmed is only a shorthand for --untrimmed-output /dev/null as per the guide. I do not know much about the QIIME 2 API. But I do use cutadapt often. :-)

Thanks @mikerobeson! I updated my comment above to clarify that I was discussing QIIME 2 types, specifically. Without going into too much detail, QIIME 2 handles a bunch of file validation when reading and saving files. Right now, it is not possible in QIIME 2 to save an artifact of type SampleData[SequencesWithQuality] that is completely empty (read: no data in it). @ebolyen has some clever ideas about how to make the QIIME 2 framework more capable of dealing with these types of cases, but we are still in the planning phases. I know on the surface this problem sounds trivial ("just don't save a file if it doesn't have any data!"), but the reality is a bit more complicated with respect to type theory. Stay tuned!

Thanks for the update @thermokarst! I completely understand these sorts of issues. Thank you for looking into this. Cheers!

This recently came up on the forum.

Stewing on this problem a bit. One thing about this --discard-unmatched flag ---the inclusion of this flag will completely change to resulting reads --- any unmatched reads will be "discarded," rather than being left as-is. I was originally concerned about "empty" output files, but now my concern is that we probably don't want to go changing this functionality underneath people's feet... Options:

New methods: trim_single_and_discard and trim_paired_and_discard
If we supported optional outputs, add a boolean parameter in the original trim_single and trim_paired methods
Just add this behavior into trim_single and trim_paired with no way of disabling or opting out. We could add some new methods to q2-demux (or somewhere else?) --- merge_single/merge_paired (this could have an overlap_method parameter: error_on_overlapping_sample - for merging "datasets", concatenate - for merging "matched samples"), these could allow you to concatenate the matched and unmatched reads.

Thoughts @mikerobeson, @ebolyen, @nbokulich?

now my concern is that we probably don't want to go changing this functionality underneath people's feet...

Why not add the parameter to trim_* and set it to False by default? That way you are not changing the functionality under people's feet.
There still is the concern about having an empty output — just raise and error if the output will be empty (after all, users would be consciously specifying that they want to drop any non-matches, so better to wait and raise an error [== notification] than an empty output that is not useful to them).
I do not like the other options as much for these reasons:

New methods: trim_single_and_discard and trim_paired_and_discard

Method sprawl just confuses people especially if the methods are largely redundant.

If we supported optional outputs, add a boolean parameter in the original trim_single and trim_paired methods

The discarded output would be useless. I could see some cases where it may be useful (e.g., filter out different marker genes that have primers attached, e.g., for those folks mixing ITS + 16S), but by and large this would not be useful. Still, I prefer this out of your three options

Just add this behavior into trim_single and trim_paired with no way of disabling or opting out.

Meh. This breaks the current use case where we use trim_* to remove adapters that may or may not be hanging off of a set of sequences. We don't want to break that functionality for the sake of supported a slightly different but distinct use case.
I really think that we can kill 2 birds with one stone here. Your option 2 is most palatable to me, but why not just discard those reads and raise an error if the output is empty?

Yep, let's just discard outright. When the framework supports optional outputs we can start plopping those reads into their own artifact. :level_slider:

Hello everyone. I think @nbokulich hit all the points quite well and I largely agree with them. I'd also echo his question:

...why not just discard those reads and raise an error if the output is empty?

As this must not only be addressed here but, if I remember correctly, this "empty file" issue has come up before. Didn't this occur with the ITSxpress plugin? Or was that determined to be another issue? If not, I can still imagine several cases where this "empty file" issue can arise in both of these plugins.
Sometimes it can be helpful to have cutadapt to write out the untrimmed sequences, i.e.

--untrimmed-output
--untrimmed-paired-output.

This helps to determine how many off-targets or the types of contamination that can be encountered.  At least my colleagues tend to ask me for the reads that did not make the "cut". Haha, see what I did there? 😆

Oops! Sorry @thermokarst I did not see your response prior to posting mine. :-)


rela.cpython-37m-x86_64-linux-gnu.so: undefined symbol

As I attempted to start training using the command provided in the readme:
python run.py --adhoc --cfg conf/c02_selfplay/liars_sp.yaml \
    env.num_dice=1 \
    env.num_faces=4 \
    env.subgame_params.use_cfr=true \
    selfplay.cpu_gen_threads=0  \
    selfplay.threads_per_gpu=8

I encountered the following error:

Traceback (most recent call last):
File "run.py", line 25, in
import cfvpy.tasks
File "/home/myname/rebel-main/cfvpy/tasks.py", line 15, in
import cfvpy.selfplay
File "/home/myname/rebel-main/cfvpy/selfplay.py", line 27, in
import cfvpy.rela
ImportError: /home/myname/rebel-main/cfvpy/rela.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN8pybind116detail11type_casterIN2at6TensorEvE4loadENS_6handleEb

I am not sure what really went wrong. My guess is this has something to do with the make process. I appreciate it if anyone could shed some light on this issue.
Additional info:
When I first attempted make, the compiler was complaining about not being able to find py::module_ in /opt/conda/envs/rebel/lib/python3.7/site-packages/torch/include/torch/csrc/utils/pybind.h (line 176)
And according to pybind 11 documentation,

py::module has been renamed py::module_, but a backward compatible typedef has been included

So I changed py::module_::import to py::module::import. I completed make without any error, but perhaps this is not the right thing to do.

I described something similar here
Inspired by the subsequent comment by rwgk, I realized that my torch version might be too high (1.13.1 instead of 1.4.0 as in requirements.txt). Downgraded it and the error is gone. Closing this issue now.


flush big streams

I noticed that there is a size limit for a single stream when pushing logs to loki:
Thread-27 (_flush) ERROR:
                        Error while sending logs: 400 Client Error: Bad Request for url: http://localhost:3100/loki/api/v1/push
Thread-27 (_flush) ERROR:
                        Response status code: 400, response text: Max entry size '262144' bytes exceeded for stream '{application="xyz"}' while adding an entry with length '415746' bytes

Maybe we can find an option to flush after a certain threshold of bytes is reached (additional or instead of the timebased flushing)?

Hi, we will check how to flush base on time or buffer size

I am trying to reproduce it but i am  not able to. Can you provide a sample to try to reproduce it?


oss-health complains about missing GITHUB_ACCESS_TOKEN even when it's set

oss-health doesn't seem to be able to read GITHUB_ACCESS_TOKEN from the environment, even when it's set

https://github.com/microsoft/OSSGadget/pull/321

Sorry for the regression.
Thanks for reporting and identifying the fix!
Looks simple enough should have a new version out shortly.

Should be merged now.


Fix/ddw 224 Dark blue theme UI improvements and fix for missing launcher-config.yaml in local development

This PR resolves some minor UI defects in Dark theme such as missing shadows and wrong text hover color in general settings menu.
It also adds missing launcher-config.yaml file as a fix for: https://iohk.myjetbrains.com/youtrack/issue/DEVOPS-802 issue.

@cleverca22 please review this PR - DEVOPS-802 is fixed in commit: https://github.com/input-output-hk/daedalus/pull/867/commits/1722f39e2c06dabaaca1d278b8269c7a60e88a71

@cleverca22 I can confirm that the mainnet installer produced by the BK has the correct production report server URL.


FLUID-5678: Updated docs to reflect changes in HTML and stylus for UI…

Updated docs to reflect changes in Pull request 611

@maozillah for the API changes, could you make sure to note any selectors that have changed (e.g. choiceLabel to tocDesc). The same goes for any other option that has changed.

Updated pull request

Updated, hopefully more clear now.

Updated

Updated

Update

@maozillah while you're making changes in this file, could you please fix the link in the note under the Component Grade Changes section. Because it is in a markup block, it needs to use standard HTML for the link rather than the markdown syntax.

Updated

Closed in favour of https://github.com/fluid-project/infusion-docs/pull/72 which updates the contents of this pull request to address merge conflicts.


Getting Started with Metasploit Framework

Title
Getting started with Metasploit Framework
Introduction Paragraph
Metasploit Framework is a penetration testing platform that enables user to write, test, and execute code.
Metasploit Framework also contains variety of tools that will be discussed in the article. The tools mentioned above helps user to execute attacks or evade detection.
Key Takeaways
By the end of the article you should be able to:

Use and understand the MFs
Access MSF console on Windows and Linux
Relate with modules in Metasploit Framework
Run exploits

References
N/A

sounds like a cool topic approved 👍  @Kimosop-Wakanda


Localization-only mode still mapping

I am currently working on a multi-robot project. One of the issues we are running into is that if two robots are near each other and one robot is still for a while, the other robot will map the still robot into the global map.
To fix this, I tried to change my .lua configuration and run Cartographer in localization-only mode as documented here:
TRAJECTORY_BUILDER.pure_localization_trimmer = {
    max_submaps_to_keep = 3,
}

Even after this modification, Cartographer will still keep mapping and adding to the global map. A change that I did notice is the global map will reset to it's original state if the robot moves to (what I'm assuming is) a different submap from where the modifications occurred, but I am hoping that we don't have to do this.
Is there any way to prevent mapping completely, either by tuning parameters or by changing the code?

https://github.com/cartographer-project/cartographer_ros/blob/c138034db0c47fe0ea5a2abe516acae02190dbf5/cartographer_ros/cartographer_ros/occupancy_grid_node_main.cc#L45C39-L45C43
Maybe you could try making it output only the frozen submaps

The way localization mode works in cartographer is to start a new trajectory.
This means that you will have at least two trajectories in the pose graph: one map trajectory (which is frozen and will not be optimized) and the live robot trajectory.
The latter only keeps the last N (3 in your case) submaps, dropping the old ones.
I think this is the behavior that you are experiencing: once new submaps are created the old one depicting your robot get eliminated.
There is no way to avoid mapping altogether, since this is the fundamental behavior of Cartographer.
Also, I would say that philosophically this behavior is not wrong: the robot standing sill is, temporarily, a useful landmark to consider.
A note on @eithwa 's answer: that is correct if you only care about visualization of the map. The acutual behavior of Cartographer will not be influenced by the setting of the occupancy grid node. It is, after all, a different node.

The way localization mode works in cartographer is to start a new trajectory. This means that you will have at least two trajectories in the pose graph: one map trajectory (which is frozen and will not be optimized) and the live robot trajectory. The latter only keeps the last N (3 in your case) submaps, dropping the old ones. I think this is the behavior that you are experiencing: once new submaps are created the old one depicting your robot get eliminated. There is no way to avoid mapping altogether, since this is the fundamental behavior of Cartographer. Also, I would say that philosophically this behavior is not wrong: the robot standing sill is, temporarily, a useful landmark to consider.
A note on @eithwa 's answer: that is correct if you only care about visualization of the map. The acutual behavior of Cartographer will not be influenced by the setting of the occupancy grid node. It is, after all, a different node.

The way localization mode works in cartographer is to start a new trajectory. This means that you will have at least two trajectories in the pose graph: one map trajectory (which is frozen and will not be optimized) and the live robot trajectory. The latter only keeps the last N (3 in your case) submaps, dropping the old ones. I think this is the behavior that you are experiencing: once new submaps are created the old one depicting your robot get eliminated. There is no way to avoid mapping altogether, since this is the fundamental behavior of Cartographer. Also, I would say that philosophically this behavior is not wrong: the robot standing sill is, temporarily, a useful landmark to consider.
A note on @eithwa 's answer: that is correct if you only care about visualization of the map. The acutual behavior of Cartographer will not be influenced by the setting of the occupancy grid node. It is, after all, a different node.

Hey,
I'm using Cartographer for pure localization and Cartographer is mapping when the pure-localization mode is open. If I use the map_server node to publish the map and I don't use cartographer_grid_node, would it be better? According to what you said, there will be no change. What do you think?

Hey,
I'm using Cartographer for pure localization and Cartographer is mapping when the pure-localization mode is open. If I use the map_server node to publish the map and I don't use cartographer_grid_node, would it be better? According to what you said, there will be no change. What do you think?

I do not understand the goal you want to achieve.
The map_server node is not capable of publishing cartographer maps, unless you create them with the occupancy grid node, save them and republish them.
Could you elaborate on your question?

Sure,
Firstly, I created a map with a cartographer. Then I saved the map as a PNG file using the map_server node. Now I'm using the map_server node instead of cartographer_occupancy_grid_node. I wonder if this has any effect on the Cartographer's background mapping. I can give an example like this:
the old launch file is :

the new launch file is:

Sure,
Firstly, I created a map with a cartographer. Then I saved the map as a PNG file using the map_server node. Now I'm using the map_server node instead of cartographer_occupancy_grid_node. I wonder if this has any effect on the Cartographer's background mapping. I can give an example like this:
the old launch file is :
<node name="cartographer_node" pkg="cartographer_ros"
    type="cartographer_node" args="
        -configuration_directory $(find my_navigation)/configuration_files
        -configuration_basename 2d_localization.lua
        -map_filename $(arg map_filename)"
    output="screen">
</node>
<node name="cartographer_occupancy_grid_node" pkg="cartographer_ros"
    type="cartographer_occupancy_grid_node" args="-resolution 0.05" />

the new launch file is:
<node name="cartographer_node" pkg="cartographer_ros"
    type="cartographer_node" args="
        -configuration_directory $(find my_navigation)/configuration_files
        -configuration_basename 2d_localization.lua
        -map_filename $(arg map_filename)"
    output="screen">
</node>
<node name="map_server" pkg="map_server" type="map_server" args="$(find my_navigation)/map.yaml">
      <param name="frame_id" value="map"/>
</node>

It does not have any effect on Cartographer mapping process.
Cartographer does not use png maps to localize, but rather saves the full pose graph+occupancy grids and more in a pbstream file.
The occupancy_grid_node  is just visualization / export tool.
I also notice that you use the -map_filename flag, which is not a supported flag for the cartographer node. Are you able to run those launch files without crashes?

Hi again,
Of course, I can not run this launch file. I just want to give an example. My launch file is not like this.
Thank you for your information. I got this point. There will be no difference and I want to ask a question. You said, " I would say that philosophically this behavior is not wrong: the robot standing sill is, temporarily, a useful landmark to consider. ". However, my robot has a path where it constantly turns, and since the map constantly updates itself after the robot works for 3-4 hours, the map and therefore the localization shifts by 30-40 degrees. How can I prevent this situation? Do you have any advice?

However, my robot has a path where it constantly turns, and since the map constantly updates itself after the robot works for 3-4 hours, the map and therefore the localization shifts by 30-40 degrees. How can I prevent this situation? Do you have any advice?

I think the parameters you are using may not be allowing the new submap to match well with the old one, which could be causing this issue. You can try adjusting these parameters to make it easier for them to match each other, but of course, this is just one possible guess.
POSE_GRAPH.constraint_builder.global_localization_min_score = 0.5
POSE_GRAPH.constraint_builder.min_score = 0.4

Since this is another possibly lengthy topic i suggest you open another issue :)

Thank you for your advice @eithwa. I'll try to set these parameters.


EPRT doesn't correctly handle IPv6 addresses

Mailed directly to me from a user:
Hi,
We are using pyftpdlib in the test suite of Open vSwitch (see openvswitch.org).
I encountered the problem where the pyftpdlib would reject EPRT commands opening the active data connection when the connection had been mangled by Linux NAT.
Here is the debug log of the failed connection:
[I 15-10-29 17:09:02] >>> starting FTP server on :::21, pid=122735 <<<
[I 15-10-29 17:09:02] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 15-10-29 17:09:02] masquerade (NAT) address: None
[I 15-10-29 17:09:02] passive ports: None
[I 15-10-29 17:09:02] use sendfile(2): True
[I 15-10-29 17:09:06] fc00::240:49216-[] FTP session opened (connect)
[D 15-10-29 17:09:06] fc00::240:49216-[] -> 220 pyftpdlib 1.2.0 ready.
[D 15-10-29 17:09:06] fc00::240:49216-[] <- USER anonymous
[D 15-10-29 17:09:06] fc00::240:49216-[] -> 331 Username ok, send password.
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] <- PASS ******
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] -> 230 Login successful.
[I 15-10-29 17:09:06] fc00::240:49216-[anonymous] USER 'anonymous' logged in.
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] <- SYST
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] -> 215 UNIX Type: L8
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] <- PWD
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] -> 257 "/" is the current directory.
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] <- TYPE I
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] -> 200 Type set to: Binary.
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] <- EPRT |2|fc00:0000:0000:0000:0000:0000:0000:0240|33134|
[W 15-10-29 17:09:06] fc00::240:49216-[anonymous] -> 501 Rejected data connection to foreign address fc00:0000:0000:0000:0000:0000:0000:0240:33134.
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] <- LPRT 6,16,252,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,145,239
[D 15-10-29 17:09:06] fc00::240:49216-[anonymous] -> 500 Command "LPRT" not understood.
[I 15-10-29 17:09:06] fc00::240:49216-[anonymous] FTP session closed (disconnect).

I could get around this by having this setting:
         permit_foreign_addresses = True

which tells me that IPv6 address comparison fails, since the format in the packet is not normalized, as you can see from the “501” line. However, it is the same address and should be considered as the same:

RFC 2428 (https://tools.ietf.org/html/rfc2428) specifies the address format as “IPv6 string representations defined in [RFC 2373]”. RFC 2373 has been obsolete by newer RFCs, and I believe the latest relevant one is RFC 5952 (https://tools.ietf.org/html/rfc5952) “A Recommendation for IPv6 Address Text Representation”. Therein:

“As IPv6 deployment increases, there will be a dramatic increase in
the need to use IPv6 addresses in text.  While the IPv6 address
architecture in Section 2.2 of RFC 4291 describes a flexible model
for text representation of an IPv6 address, this flexibility has been
causing problems for operators, system engineers, and users.  This
document defines a canonical textual representation format.  It does
not define a format for internal storage, such as within an
application or database.  It is expected that the canonical format
will be followed by humans and systems when representing IPv6
addresses as text, but all implementations must accept and be able to
handle any legitimate RFC 4291 format."
The format seen on the EPRT command in the log above is in one of these legitimate RFC 4291 formats.
This problem can be fixed in pyftpdlib by normalizing the address received from the EPRT command as below:
--- /usr/share/pyshared/pyftpdlib/handlers.py.old	2015-10-29 17:01:25.543829322 -0700
+++ /usr/share/pyshared/pyftpdlib/handlers.py	2015-10-29 17:01:40.195437223 -0700
@@ -1935,6 +1935,9 @@
             if self._af == socket.AF_INET:
                 self.respond('522 Network protocol not supported (use 1).')
             else:
+                # Normalize the string representation of the remote IPv6 address
+                internal = socket.inet_pton(socket.AF_INET6, ip)
+                ip = socket.inet_ntop(socket.AF_INET6, internal)
                 self._make_eport(ip, port)
         else:
             if self._af == socket.AF_INET:

I’m not sure if it is just luck that self.remote_ip is in the same normalized format as produced by init_ntop(), but it seems to work. To be sure, it may be better to normalize self.remote_ip in the same way. For now we need to rely on the “permit_foreign_addresses = True” fix, which is unfortunate as the test case in question is to make sure NAT works properly, and one failure mode is the case where the EPRT line is not mangled properly and the data connection is opened to the non-NATted address.
With regards,
Jarno Rajahalme

Closing as duplicate of https://github.com/giampaolo/pyftpdlib/issues/373


WebSocket connection is closed - how to interpret the error code?

Hello - I am trying to interpret these error messages:
WebSocket connection is closed: code = 1005 (no status code [internal), no reason
WebSocket connection is closed: code = 1006 (connection closed abnormally [internal), no reason
which (apparently) occur on my end, based on this: https://tools.ietf.org/html/rfc6455#section-7.4.1
1005
1005 is a reserved value and MUST NOT be set as a status code in a
Close control frame by an endpoint.  It is designated for use in
applications expecting a status code to indicate that no status
code was actually present.
1006
1006 is a reserved value and MUST NOT be set as a status code in a
Close control frame by an endpoint.  It is designated for use in
applications expecting a status code to indicate that the
connection was closed abnormally, e.g., without sending or
receiving a Close control frame.
My code is as follows:
async def connect(self):
	while True:
		log.info("connecting")
		async with websockets.connect(self.url, max_queue=4 * 2**10,) as websocket:
			log.info("connected")
			await websocket.send(json.dumps(self.subscribe))
			log.info("subscribed: {}".format(websocket.remote_address))
			while(True):
				try:
					message_str = await asyncio.wait_for(websocket.recv(), timeout=self._timeout)
					self.queue.put_nowait(message_str)
				except asyncio.TimeoutError:
					log.warn("timeout error - no data in {} seconds, pinging connection".format(self._timeout))
					pong_waiter = await asyncio.wait_for(websocket.ping("keepalive"), timeout=self._timeout)
					await asyncio.wait_for(pong_waiter, timeout=2 * self._timeout)
					log.warn('ping/pong received, keeping connection alive...')
				except Exception as e:
					error_message = "resetting connection: {}".format(e.args)
					log.error(error_message)
					raise Exception(error_message)

How would i figure out what these error codes mean? Typically in a synchronous websocket library, there would be an on_error() handler.  Is that the case here? How do I interpret these errors? Thanks!

Perhaps the remote endpoint closed the connection improperly.
Enable logging to see all messages exchanged on the connection: https://websockets.readthedocs.io/en/stable/cheatsheet.html?highlight=logging#debugging
and you should get a better idea of what's happening.


Show compilation errors in debug-info

I am getting RuntimeError: No completions found; errors in the file?
How can I see the errors?
The server log only shows the python traceback.  The debug info only shows the compile flags.

You're seeing this output in the *ycmd-server* buffer, right? You might try adding "--log debug" to the ycmd command line (i.e. via ycmd-server-command). Perhaps that will give you more info.
@valloric Any suggestions as to how to get more info in this situation?

This error means that ycmd was unable to get any completions out of libclang for that location in the file. Which means that libclang couldn't compile the file. Which means there's probably an unrecoverable compilation error somewhere before that location in the file.
First step is to resolve any diagnostic errors that ycmd is showing, if any.
Second, a solid way to test this out is to take the same flags that ycmd is using for the file (which you can get from ycmd debug info; don't know how emacs-ycmd exposes it) and give that to clang-the-binary. It should spit something out.

OK, I managed to get this to work by switching to a clang completion database.
I still think this would be nice to have, but you can reopen if you want to support that.


Documentation unclear on extension Publish/Unpublish vs. Share/Unshare

For some extensions the menu will show "Publish" and "Unpublish", not "Share" or "Unshare" as depicted in the documentation page. It's not clear in the documentation what determines when these options are shown.  It's likely down to whether the extension is marked public or not but that isn't very clear.
Picture 1: Menu from the management portal

Picture 2: Picture from the documentation page:  note the "Share" vs "Publish" difference

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 14955ee9-29f2-46fe-3a8d-a42666f53ffc
Version Independent ID: 5d417cd0-a671-ee30-8fca-5f8b0addd328
Content: Develop a web extension for Azure DevOps - Azure DevOps
Content Source: docs/extend/get-started/node.md
Product: devops
Technology: devops-ecosystem
GitHub Login: @chcomley
Microsoft Alias: chcomley

Hi @andersforsgren, Thanks for the heads up. I've clarified in the article that it displays differently depending on the extension, but that the functions are the same. Thank you!


Why doesn't the default error handler print in test environment?

i.e. I'm just interested in learning the reasoning behind this conditional?
https://github.com/expressjs/express/blob/c0136d8b48dd3526c58b2ad8666fb4b12b55116c/lib/application.js#L628-L631
Thanks!

This is because during your tests you want to test for errors.


Rancher catalog service is not available for ~40 seconds in no internet mode

Steps to reproduce:

Pre-pull rancher/server:latest image
Turn off wifi
Deploy Rancher docker run -d --restart=always -p 8080:8080 rancher/server
Access catalog tab in the UI immediately. It fails. Retry after ~40 seconds, it starts working. From the logs I am seeing the following:

time="2016-05-24T22:20:29Z" level=info msg="Using catalog community=https://github.com/rancher/community-catalog.git"
...20 sec
time="2016-05-24T22:20:49Z" level=error msg="Failed to pull the catalog from git repo https://github.com/rancher/rancher-catalog.git, error: exit status 1"
time="2016-05-24T22:20:49Z" level=error msg="Git pull for Catalog library failing with error: exit status 1 "
... another 20 secs
time="2016-05-24T22:21:09Z" level=error msg="Failed to pull the catalog from git repo https://github.com/rancher/community-catalog.git, error: exit status 1"
time="2016-05-24T22:21:09Z" level=error msg="Git pull for Catalog community failing with error: exit status 1 "

20 seconds after startup if tries to pull "Catalog library" and fails, then after 20 seconds it tries to pull "Catalog community" and fails. Only after the last failure it pulls the data from cache, and UI can access catalog service. Wonder there is some timeout on library pull, and while the call is going, service is not accessible.

Unable to reproduce this running the following command in an ubuntu VM: docker run -d --restart=always -p 8080:8080 rancher/server:v1.2.0-pre4-rc3. This delay seems to be there in docker for Mac, but not related to the catalog-service code

Tested with rancher server version - v1.2.0-pre4-rc4
In the rancher-server setup, I have blocked access to github by adding the following iptables rules:
iptables -I FORWARD -d <gihub ipaddress>-j DROP

Following are the issues I am seeing:

After fresh install , access catalog from UI gives me the following error:

After about 5mts  -6 mts , I see the catalog being presented successfully to the user.
In my case , github access takes ~2 mts to report failure. And seems like we show "503 Error" until all the 3 catalogs infra/community/library have timed out (sequentially).
time="2016-10-19T23:47:41Z" level=info msg="Starting Rancher Catalog service"

time="2016-10-19T23:47:41Z" level=info msg="Using catalog infra=https://github.com/rancher/infra-catalog.git"

time="2016-10-19T23:47:41Z" level=info msg="Using catalog community=https://github.com/rancher/community-catalog.git"

time="2016-10-19T23:47:41Z" level=info msg="Using catalog library=https://github.com/rancher/rancher-catalog.git”

….

time="2016-10-19T23:49:48Z" level=error msg="Git pull for Catalog infra failing with error: exit status 1 “

time="2016-10-19T23:51:56Z" level=error msg="Git pull for Catalog community failing with error: exit status 1 “
time="2016-10-19T23:54:03Z" level=error msg="Git pull for Catalog library failing with error: exit status 1 “

The very first template from any catalog I try to view from UI takes about 2 mts to load.
After this , every other template in this catalog will load immediately.

Disable and Enable the catalog. Once the catalog is enabled , user shown message "There are no compatible templates" presented to the user for a very long time like 2 mts (seems like this is the time taken to wait for the gihub access to timeout). Every catalog takes additional 2 mts to be shown. When i have 3 catalogs to start with , the last of the catalog takes 6 mts to load since we are doing the catalog fetch sequentially.

Tested with rancher-server version -- v1.2.0-pre4-rc5
All the timing issues tracked in the previous comments work as expected after fresh install and after enabling/disabling all catalogs.
There is still issue with very first template to view any template from any catalog takes a very long time to load which is tracked in https://github.com/rancher/rancher/issues/6388


4 charts: Edits for consistency

I'd like to request some edits to the language inside our charts, for consistency and brevity. Thinking about how they'll look in French.)

Indicators on track

Let's change "nonreporting" to "unavailable".** Non-reporting is often used as a legal compliance term referring to an organization that is not required to report. People may be thrown by the phrase, thinking it means they're not required to report on the indicator.
1.1. Indicator list title: 17 indicators have data unavailable
XX% are > 15% above target -- no change. Indicator list title can use the same phrase.
XX% are on track -- no change. Indicator list title can use the same phrase.
XX% are > 15% below target -- no change. Indicator list title can use the same phrase.

What about the French?
Here's my stab at translating those chart labels to French, for testing purposes. Does everything wrap? Is it still digestible?
- 100% non disponibles
- 100% sont > 15% supérieur au cible
- 100% sont sur les rails
- 100% sont > 15% inférieure au cible
I thought about removing the word "are" from the phrasing -- i.e. 4% > 15% above target. But that could easily be read as "four percent is greater than fifteen percent". Sigh.
Indicators with targets

have no targets --> no targets
1.1. Indicator list title: 17 indicators have missing targets
have targets defined --> have targets

Indicators with results

have no results --> no results
1.1. Indicator list title: 17 indicators have missing results
have reported results --> have results

Results with evidence

are missing evidence --> no evidence
1.1. Indicator list title: 17 indicators have missing evidence
"have evidence" is fine.

All edits verified!


[HeaderMap] Eliminate non-const header access functions follow-up

Action item for CVE-2019-15226
Following up on the patch for CVE-2019-15226 (https://github.com/envoyproxy/envoy/issues/8520), where we added a cached_byte_size_ optional in HeaderMap that is updated with modifications to the HeaderMap, we should take steps to maintain an always-valid internal byte size count:
This would involve eliminating the non-const inline header access functions defined here:
https://github.com/envoyproxy/envoy/blob/48c7c203890cf6c011fa6591a3b233d637c334f5/include/envoy/http/header_map.h#L351-L355
And the non-const get method for any HeaderEntry:
https://github.com/envoyproxy/envoy/blob/48c7c203890cf6c011fa6591a3b233d637c334f5/include/envoy/http/header_map.h#L502
After removing these accessors and changing uses of them to favor other HeaderMap modifiers, we should remove refreshByteSize().
Detail

[x] Add functionality for modifying inline headers directly.

[x] Port over any sets to inline header macros for setReference, setCopy and setInteger

[x] Add functionality for appending and adding Header values to inline headers.

Grepping for insert[A-Z] shows we have 8 references of insert##name left. 4 of these are for using appendToHeader. 2 are const, and 2 are to set GrpcTimeout with Common::toGrpcTimeout.

[x] Remove non-const inline header entry accessor: ##name() and insert##name()
[x] Remove non-const HeaderEntry* HeaderMapImpl::get(const LowerCaseString& key).

Grepping for get([^)] shows we have 2 non-const refs for get(const LowerCaseString& key):
source/extensions/filters/http/lua/wrappers.cc:84
source/extensions/filters/http/ext_authz/ext_authz.cc:169

[ ] Fuzz inline header methods
[ ] Remove unnecessary inline header methods for integer types.
[ ] Wrap up misc TODOs in header_map.h, i.e. string -> string_view conversions

I'm taking care of what I thought were some mechanical changes, but have a performance impact. I'd like to drive by some of the changes that this would require.
Part of the challenge is that these inline header methods allow direct access to an inline HeaderEntry, bypassing the need for a lookup in HeaderMap methods that take a LowerCaseString key. For example, one mechanical change was to switch from using a common pattern of:
headers.insertConnection().value().setReference(value);
to:
headers.setReference(Headers::get().Connection, value);
The former's HeaderMap::insert##name() method will grab the reference to the inline header, and directly modify the value. The latter has to perform the lookup to grab the inline HeaderEntry and then can proceed the same way.
This adds about 100ns in my benchmarks, but it's a constant time lookup. It would ideally be great to figure out a way to uniformly handle inline and non-inline headers through the HeaderMap API. Of course I could add inline header functions that directly reflect the HeaderMap API (like headers.setReferenceConnection(value)) but that would add many redundant methods.
Any ideas? Do you think this is too much of a performance hit?
@jmarantz @htuch @mattklein123

Can we add an API:
headers.insertConnection(value)

that would avoid exposing the non-const ref but still be O(1)?

@asraa how many different cases are there that you need to cover? Per @jmarantz I'm wondering if we should just add the extra methods? It's all done by macro and everything mostly gets inlined so it doesn't seem that bad to me?

Yeah, that would be easiest. I would prefer to do it the macro way so long as it doesn't seem too messy. I would need to handle the cases of adding values that will or will not be copied (addReference/addReferenceKey), and similar for setReference/setReferenceKey.

OK I dug around a little bit and need to edit the comment above:
It would only require set methods. I'd add a set##name##Reference, set##name##Copy, and set##name##Integer.
All of the calls to append a header value to an inline header by comma concatenation go through the addReference API call, with the O(1) added lookup:
https://github.com/envoyproxy/envoy/blob/373af7564f4e943112456bf40084a7f43d5e9d96/source/common/router/router.cc#L395
since there's no other way to do so. There are not many of these calls.

@asraa SGTM

@asraa WDYT we should do about this issue? Any update?

@asraa WDYT we should do about this issue? Any update?

The last part of this is the "Remove unnecessary set##name calls for integer types"
(we generate set##name(absl::string_view value) and set##name(uint64_t value) for all headers, although there are a few integer only headers that don't need the string_view method.)
I would probably do this by splitting headers into integer only type and other type, but now that we have a req/resp split it would split into (req_integer, req), (resp_integer, resp), (req_resp_integer, req_resp). Is it worth it?

The last part of this is the "Remove unnecessary set##name calls for integer types"
(we generate set##name(absl::string_view value) and set##name(uint64_t value) for all headers, although there are a few integer only headers that don't need the string_view method.)
I would probably do this by splitting headers into integer only type and other type, but now that we have a req/resp split it would split into (req_integer, req), (resp_integer, resp), (req_resp_integer, req_resp). Is it worth it?

Maybe not just for the small number of integer types. But are we also generating lots of useless set##name(uint64_t...) for non-integer headers? If so, the cumulative compile overhead of ~100 useless methods in a class that's visible to probably thousands of cc files may be worth a bit of effort.

Maybe not just for the small number of integer types. But are we also generating lots of useless set##name(uint64_t...) for non-integer headers? If so, the cumulative compile overhead of ~100 useless methods in a class that's visible to probably thousands of cc files may be worth a bit of effort.

Did a quick count:
I only found uses for numerics in status, content-length, envoy-attempt-count.
The change is probably really small in that case, and I can give it a stab.

Did a quick count:
I only found uses for numerics in status, content-length, envoy-attempt-count.
The change is probably really small in that case, and I can give it a stab.


fixes for optional key passing, issue #8067 for tile sources

This fixes the same problem as described in #8067. While setting different tile urls setUrl() on a source the animation would always jump to the very first source url that was set. The solution was to set distinguishing optional keys for each source which would select proper tiles form cache for animation. Minor fixes were needed to make sure optional keys can be passes through tile sources.

Thanks for tackling this @dimin.  I hope the comments above make sense.  Let me know if not.

@dimin do you plan to work on this pull request?

Sorry guys, was super busy with some other work. I'll apply all the required fixes soon!

Thanks @dimin.  Any chance you can provide a like to an example that demonstrates the problem and the fix?  I’ll take a look at the example in #8067 when I get a chance.

I'm using something similar to this example:
https://openlayers.org/en/latest/examples/reusable-source.html
I start with one url and change them for different z positions. Although simply setting the source url multiple times does work, smooth transitions would always start with the same tile, i.e. the tile produced from the very first URL I've given my source. The problem was with the key not being set and thus used when finding tiles in the cache.
In order to solve that, I needed to make sure the first URL had a key along with it and I needed to pass the new key along with the new url to the tiles being created and also ensure that the old key is used to fetch the tiles to initialize the transition. I would pass the initial key to the source being created first.
In my case I have my own source derived from TileImage and I've changed the setUrl like this:
setUrl(url, key) {
const urls = expandUrl(url);
const tileUrlFunction = createFromTileUrlFunctions(urls.map(this.myCreateFromTemplate));
this.setTileUrlFunction(tileUrlFunction, key || this.key_);
}
I'm guessing this step is not quite needed as long as the first url has a key associated with it since UrlTile is setting the key from the url itself. Some of that wasn't working for me and I've had to set the keys explicitly.

Thanks @dimin.
I think we may need to revisit this – at least to add documentation and potentially to make it easier to get the behavior people want.

Sure @tschaub, I'll help with what I can and really thank you for creating and maintaining such an amazing and extensible viewer!


Add an "optional" tag to commands arguments' description when generating markdown docs

Current behavior
Although there is an implicit differentiation between required/optional args (<ARG> for required args and [ARG] for optional), but we can't assume every user knows that. This is done automatically when showing the --help in the terminal, but not in the markdown docs (all arguments are shown as <ARG>), which creates an inconsistency.
We need to improve the markdown docs generation to include that information more explicitly.
Desired behavior
Related to https://github.com/build-trust/ockam/issues/5030 (they can be completed in parallel).
We should add a piece of text to optional arguments so the user can easily differentiate between required/optional arguments.
To do so, the markdown command must be updated as follows. In the generate_arg_markdown function, the value_name is always written as "`<{value_name}>`". This should be changed so that:

if arg.is_required_set() is true, print it as "`<{value_name}>`"
else, print it as "`[{value_name}]`, (optional)"

Hi @adrianbenavides, i can take this?

@Alef-gabriel looks like @BradLewis already created a PR for this, sorry! I'll try to create more issues soon 🙏

Ah sorry Alef, I didn't see that you had also commented on this. My bad!

@BradLewis No problem.


Portfolio: Update Deposits tab

Current implementation
Note, it didn't display any deposits for me, despite having made some.

Proposed design
https://www.figma.com/file/K49Nx7XfocShhHcwwQzQkm/Console-V2?node-id=2537%3A32179

Closing as done

I've created a data node ticket because we're missng the "from address" inn thr api: https://github.com/vegaprotocol/data-node/issues/855

Hey @asiaznik @pennyandrews,
As it stands we are not capturing this data (in the datanode DB) therefore cannot expose it to be used as you require.
The core side of this work is scheduled to be done post enable the trading release.

Thanks @gordsport. @mattrussell36 as this isn't going to be unblocked anytime soon I've kicked this ticket to the bottom of the backlog


Question about spring-async-plugin

In spring 5.x, @Async methods are submitted with lambda. com.navercorp.pinpoint.plugin.spring.async.SpringAsyncConfig#DEFAULT_ASYNC_TASK:

And the plugin intercepts it with DefaultClassNameMatcher, which use equals of String. But the actual name of the lambda is like:
org/springframework/aop/interceptor/AsyncExecutionInterceptor$$Lambda$672
where the number 672 in the name is the global counter value of java.lang.invoke.InnerClassLambdaMetafactory#counter .
SO, how can it be matched ?

More generally, lambda is not like anonymous inner class, which the name suffix number is the counter of  the enclosing class. When there are many lambda functions, such as Reactor, how to distinguish lambda functions in the same class?

@hoverwinter
It seems to be a bug.
Thanks for reporting.


Add more favicons to db

Add these 5 favicon values associated with a product name to db.json (in alphabetical order)
Remove the entries from the scripts/favicon-to-add.txt file

The favicons:

-2023266783: https://shodan.io/search?query=http.favicon.hash%3A-2023266783
2023619029: https://shodan.io/search?query=http.favicon.hash%3A2023619029
-2024806776: https://shodan.io/search?query=http.favicon.hash%3A-2024806776
-2024949122: https://shodan.io/search?query=http.favicon.hash%3A-2024949122
2027028811: https://shodan.io/search?query=http.favicon.hash%3A2027028811

Zoomeye too can be used with the iconhash: label https://www.zoomeye.org/searchResult?q=iconhash%3A
Remember to use the devel branch for the PR

Assign this to me

Hey @edoardottt, I would be happy to work on this! Can you please assign me this issue

Done @Nishith-Savla

Is it okay if I can't find the name for a few of them?

yes, no worries. do your best...
in case you want to help pick some favicon in the middle of the list, just check that they aren't the favicon hashes in other issues

Completed, thanks @Nishith-Savla !

Can you please add the hacktoberfest-accpeted label to this PR?

it's already there (anyway, for the competition it's okay to just have the label 'hacktoberfest' in the whole repo)


Gen.elements only returns first or last element.

I ran into a problem with Gen.elements. No matter how often I use it with a Random Generator, it always gives me either the first or the last element of the Foldable1 f.
My suspicion is the internal helper fromIndex:
edge cases seem to work
fromIndex 0 $ 10 :| [20,30,40,50,60] -- 10
fromIndex 10 $ 10 :| [20,30,40,50,60] -- 60

but indices within the bounds don't
fromIndex 5 $ 10 :| [20,30,40,50,60] -- 60
fromIndex 1 $ 10 :| [20,30,40,50,60] -- 60

given a negative index, the function crashes for me
fromIndex -10 $ 10 :| [20,30,40,50,60]
Error found:
in module $PSCI
at :1:12 - 1:14 (line 1, column 12 - line 1, column 14)

  Could not match type

    Int

  with type

    Int -> t0 t1 -> t1

while checking that type Int
  is at least as general as type Int -> t0 t1 -> t1
while checking that expression 10
  has type Int -> t0 t1 -> t1
in value declaration it

where t1 is an unknown type
      t0 is an unknown type

I am still new to Purescript, so I am not sure why a negative Int can cause that.
Luckily, it won't be called with negative numbers as long the provided chooseInt behaves correctly. Nonetheless, above behavior would explain the the uneven distribution of elements.

If I switch the order of elements g and f in the type class instance, the distribution is more even. The index lookup is done from the back (so calling it with 0 gets the last element). Negative numbers still cause the same crash.
instance semigroupAtIndex :: Semigroup (AtIndex a) where
  append (AtIndex g) (AtIndex f) = AtIndex \i -> if i <= 0 then f i else g (i - 1)

You need to use parens around the negative number, -10 is an expression rather than a literal.
What type are you using for your MonadGen instance, maybe the poor distribution is down to that?

Hmm, no, I see it too. That's extremely concerning that it's been like this for a long time!

I think we could do with some distribution tests, maybe in quickcheck if not here. I've definitely seen a few cases (separate from this one) where quickcheck is giving an odd distribution of results.

Yeah, I reproduced this by adding a test case. I thought this library had better test coverage than it does.


Draw persistent SVGs once only

If the user runs a search that selects multiple nodes (beads) in different beadplots, the highlighting will fail once the user navigates from the first beadplot to the next.  In addition, the SVG elements in the beadplots that are not initially displayed on search do not even exist!
I think we have to generate all SVGs for every beadplot on page load so that the user is navigating between persistent SVGs when switching clusters.

Possible approach: populate clusters with all information about their associated variants and samples, run search at cluster level only (to select clusters in time-tree), and then re-run the search as user browsers through clusters to select elements at the bead/variant level.

Resolved in dev branch, clusters are now populated with search text that concatenates all sequence labels


Not getting Jupyter file content based recommendation

Testing #177675

Set up a folder with a file sample.py which contains the following:

print("HI")
#%%
print("cell")
#%%
print("something else")

Open the folder with new --user-data-dir and --extensions-dir
Install the Python extension and uninstall all the Jupyter extensions that get installed with it.
Reload.
Open sample.py
At this point I expect to see a recommendation for the Jupyter extension.

Just tried these steps and it is expected. Because the recommendation prompt at step 3 has both python and jupyter extensions. Since you have accepted the recommendation, it is remembered and we do not want to show the same recommendation again if you manually uninstall any of the recommended extensions. For eg we do not recommend python extension if you uninstall python extension.


Clarify server attributes for riak servers

Clean up some info that's unnecessary to the end user and point out the right values to use for a riak server.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-trafficcontrol-PR/1207/
Test PASSed.


Safari WebGl

Modernizr not recognizing webGL enabled on mac safari.

I don't have this issue. Safari recognizes WebGL, but the surface looks different than it does on Chrome.

Safari 8.0.6, OSX 10.10

Can this be closed? Starting with Safari 8, WebGL should be enabled by default.