Fix multiple Replay extensions' interference

This PR adds an extra check to prevent the final extension in the pipeline from overwriting the Replay ID Mapping when it is not a subscription for the relevant channel.
This is a proposed fix for #814.

@mars could I ask you to take a look at this (small) PR? And thank you for your work on #740!

@mattdrees 😃 Thanks for the fix!
I just pulled this PR locally and verified that Replay IDs still work correctly with my event-driven-functions app ✅
@stomita this would be great to include in the next release.

Thanks, @stomita !

Is this fix available in<EMAIL_ADDRESS>?
I try to add multiple Replay Extensions and I suppose receive same issue (output is below)
What I try to do is
`const extensions = [];
for (const topic of topics) {
let channel = '/topic/' + topic;
let replyId = replayMap[topic] || -1;
extensions.push(new jsforce.StreamingExtension.Replay(channel, replyId));
}
this.streamingClient = (this.conn.streaming as any).createClient(extensions);
for (const topic of topics) {
let subscription = this.streamingClient.subscribe(channel, async (message: StreamingEvent) => {...}).errback(function(error: any) {
console.log(topic, 'ERROR!');
console.log(error);
});
}
`
... F_0045 ERROR! { code: null,   params: [],   message: '400::The replayId for channel {/topic/F_0045} wasn\'t found using the provided replay ID map {{/topic/F_0051=-1}}. Ensure that the channel name you provided in the replay map is valid and matches the channel name used for subscribing.' } F_0047 ERROR! { code: null,   params: [],   message: '400::The replayId for channel {/topic/F_0030} wasn\'t found using the provided replay ID map {{/topic/F_0051=-1}}. Ensure that the channel name you provided in the replay map is valid and matches the channel name used for subscribing.' } F_0048 ERROR! { code: null,   params: [],   message: '400::The replayId for channel {/topic/F_0048} wasn\'t found using the provided replay ID map {{/topic/F_0051=-1}}. Ensure that the channel name you provided in the replay map is valid and matches the channel name used for subscribing.' } F_0049 ERROR! { code: 403,   params: [ 'denied_by_security_policy' ],   message: 'subscribe_denied' } F_0050 ERROR! { code: null,   params: [],   message: '400::The replayId for channel {/topic/F_0050} wasn\'t found using the provided replay ID map {{/topic/F_0051=-1}}. Ensure that the channel name you provided in the replay map is valid and matches the channel name used for subscribing.' } F_0001 ERROR! { code: null,   params: [],   message: '400::The replayId for channel {/topic/F_0001} wasn\'t found using the provided replay ID map {{/topic/F_0051=-1}}. Ensure that the channel name you provided in the replay map is valid and matches the channel name used for subscribing.' }

It's not in 1.9.1, no. That was released before this PR.

@mattdrees could you please suggest anything for my use case?
I have many PushTopics (around 30) and I need to use replayId for each of them.
Maybe you have any workarounds?

I'd suggest installing jsforce from master, or maybe from the commit where this PR was merged, using the npm install <git remote url> syntax described at https://docs.npmjs.com/cli/install.

Hi @stomita ☺ Would it be possible to cut a 1.9.2 release to include this fix?

Cold not get in git hub I blocked I did something wrong in Atom
Mamie
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Mars Hall<EMAIL_ADDRESS>Sent: Thursday, March 7, 2019 1:01:16 PM
To: jsforce/jsforce
Cc: Subscribed
Subject: Re: [jsforce/jsforce] Fix multiple Replay extensions' interference (#815)
Hi @stomitahttps://github.com/stomita ☺ Would it be possible to cut a 1.9.2 release to include this fix?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/jsforce/jsforce/pull/815#issuecomment-470651852, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AoS8OtxhyKZdP-CfBy8x7tV2AokG0xrbks5vUWH8gaJpZM4W9mbv.


Add search capabilities to Peripheral Inspector view

Provide custom SearchOverlay component
Add search overlay to filter tree and tree table
Move custom data into the 'data' for filtering in tree table

Closes #23

@jreineckearm I pushed an update that should address your comments. The only thing I couldn't reproduce was the filtering purely based on visible items. Could you provide an example of where that is happening?

@martin-fleck-at , thanks for the updates! I have provided a reproducer for the "invisible items" issue.

@jreineckearm I investigated the issue and from what I can see the problem occurs when the respective node has never been resolved as part of the children, i.e., the tree does not know about the node since we populate the model lazily on expansion:
https://github.com/eclipse-cdt-cloud/vscode-peripheral-inspector/blob/61550b2e2a83c8a59bd6183ee317ca4922cb2758/src/plugin/peripheral/tree/provider/peripheral-cdt-tree-data-provider.ts#L55-L66
If I resolve all children of all nodes by default, the search works as expected. However, this is where we hit performance issues and everything else becomes quite slow since we do re-generate the model very often. Unfortunately, I'm not sure what the best way forward is. We could introduce a dedicated option in the search to also search "hidden" nodes. However, if we are lucky, maybe switching to Ant Design (https://github.com/eclipse-cdt-cloud/vscode-peripheral-inspector/issues/27) is a solution here as well. Please let me know how you would like to proceed with this PR.

@jreineckearm I had a look at caching the expansion state on the extension side to avoid re-generating the complete UI tree again. However, that only lead to a minor performance improvement when generating the complete tree. So I think we may want to re-consider the complete round-trip that we currently have regarding the expansion state. Is that really something that needs to be part of the extension backend model? Currently, we expand in the webview, notify the extension, the extension re-generates the complete UI model and syncs it back to the webview. It might be much better to simply keep the state in the webview and not sync the expansion state back to the extension, unless we really need it. Overall, I think this should be reworked as part of https://github.com/eclipse-cdt-cloud/vscode-peripheral-inspector/issues/27 when we switch to another UI framework altogether as that might solve our search issue by simply allowing us to have the complete UI tree available. For this PR, I'm not sure how you would like to proceed, i.e., merge it but know that only loaded elements are searched or load all of them and deal with the less than optimal performance for now.

Closing this in favor of https://github.com/eclipse-cdt-cloud/vscode-peripheral-inspector/pull/34 which is based on the new Ant Design UI framework.


aiconsole - Internship for the Junior Programmer - Backend

Implemented database storage of materials on PostgreSQL DB.

Unfortunately, I cannot test if solution works, assets are not loaded. Probably due to change in endpoints

Preinstalled assets are being loaded during FastAPI startup.

Good job!


[Bug]: Input keys are not surpressed properly at times

What happened?
Two reports on Nexus indicate that the weapon sheath/unsheath key (R by default) is not properly suppressed, #39 indicates that the favorites key to search the container is also not suppressed properly.
Version
2.0.0
What Skyrim version are you using?
1.6.1170
Relevant log output
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

In my game (Skyrim GOG v.1.6.1179, but I doubt it does matter here) it's not possible to get ready/stash away weapon(s)/spell(s) while QuickLootIE's menu is active. I.e. the game's default [R] hotkey IS completely suppressed in this case (although it may be inconvenient for some users). Unlike the game's default [Q] hotkey, which is indeed not suppressed at all, as I reported it in #39.

I personally have not been able to encounter any of these issues, which makes it harder to track/fix. It's very inconsistent.
Do you happen to have any mod changing the favorites menu in a way?

I see. Well, except SkyUI (of course) I also use Extended Hotkey System (Anniversary Edition), Persistent Favorites and Favorite Misc Items.  I hope it will help you to find the culprit.

If you have some free time, could you check whether the issue persists if you disable the mentioned mods? Might be a long shot, but if disabling them fixes the issue at least we know what to focus on.

@AtomCrafty
Sure, I should have figured this out myself earlier. Anyway it was super easy and I've found the culprit at once: it was Extended Hotkey System(Anniversary Edition), which is a pure SKSE mod by Vermunds.

@MissCorruption do you think there is a chance that the R issue is also related to other mods? I haven't seen the Nexus reports. Regarding EHS: If it's still maintained we could ask the dev to add a check whether the loot menu is open.

Sorry, but why not just make the mod's hotkeys to be configurable in the MCM or in an *.ini file? Doing so would make the hotkey compatibility issues with other mods to be the problem of users, not yours. Is there any reason not to do so?

@MissCorruption do you think there is a chance that the R issue is also related to other mods? I haven't seen the Nexus reports. Regarding EHS: If it's still maintained we could ask the dev to add a check whether the loot menu is open.

The report on Nexus mentioned an incompatibility with another mod, ye. But it was entirely in Papyrus, it was this one

Closing this in favor of #51


fontspec can't handle fallback fonts provided through markdown.

Description
The issue is that compiling a markdown file fails if fallback fonts have been passed through with similar code as they would go to html.
From an rmarkdown perspective, ideally you want to use the same code to determine fonts, regardless of whether you knit to html or pdf.
For example, you can select a preferred font, and a fallback font, e.g. for the first row of a table (with huxtable package) like this:
font(ht)[1,]   <- paste0("Agenda, sans-serif")

Which if you knit to html results in something like this, which then uses the first available font:
html { font-family: Agenda, sans-serif}

The same code causes an error (in fontspec?) when you knit to pdf.
tlmgr search --file --global "/Agenda, san-serif[.](tfm|afm|mf|otf)"
! Package fontspec Error: The font "Agenda, san-serif" cannot be found.

! kpathsea:make_tex: Invalid filename `Agenda, san-serif', contains ','

Here's an excerpt of what is being passed on as per the markdown file.
`\multicolumn{1}{!{\huxvb{0, 0, 0}{0}}r!{\huxvb{0, 0, 0}{0}}}{\huxtpad{4pt}\raggedleft {\fontspec{Agenda, san-serif} Sepal.Length}\huxbpad{4pt}} &
\multicolumn{1}{r!{\huxvb{0, 0, 0}{0}}}{\huxtpad{4pt}\raggedleft {\fontspec{Agenda, san-serif} Sepal.Width}\huxbpad{4pt}} &`

To be fair it may not be a high priority for fontspec maintainers to handle options like this that rmarkdown wants to throw at it for fallback fonts, but could it be quite easy to handle in terms of just taking the first-named font before the comma? I can't think that rmarkdown users would knit a pdf where they didn't have the first named font very often, or if it happened, they wouldn't think that is a reasonable error that the user can easily fix, e.g by installing the named font.

Surely this should be reported as an error in whatever markdown to tex convertor you are using (you don't say which, perhaps "knit" implies you are using R?) It should generate valid latex (fontspec) syntax.

Thanks @davidcarlisle, yes I am using R (RStudio, Rmarkdown, xelatex).  Where would be the place to raise that issue?

@markbneal  sorry I have no idea where the convertor that R uses is maintained, but naturally any something-to-latex convertor has to be responsible for generating valid latex syntax,  you can not expect latex to change to accept whatever output is generated by any third party software that aims to generate latex.

Thanks @davidcarlisle, I have raised it at the rstudio community here - will see what options are proposed.

Link in my previous comment has a good solution. I’ll close this.


env: split-string implementation

There is a cool feature in in env (GNU coreutils) 8.30:
$ env --help
Usage: /hab/bin/env [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...]
Set each NAME to VALUE in the environment and run COMMAND.

Mandatory arguments to long options are mandatory for short options too.
  -i, --ignore-environment  start with an empty environment
  -0, --null           end each output line with NUL, not newline
  -u, --unset=NAME     remove variable from the environment
  -C, --chdir=DIR      change working directory to DIR
  -S, --split-string=S  process and split S into separate arguments;
                        used to pass multiple arguments on shebang lines
  -v, --debug          print verbose information for each processing step
      --help     display this help and exit
      --version  output version information and exit

A mere - implies -i.  If no COMMAND, print the resulting environment.

GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
Full documentation at: <https://www.gnu.org/software/coreutils/env>
or available locally via: info '(coreutils) env invocation'
[root@stg-jenkins centos]# /hab/bin/env --version
env (GNU coreutils) 8.30
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Richard Mlynarik, David MacKenzie, and Assaf Gordon.

How about to implement this?

Hey, I'd like to take this on if it's still needed? I had a quick look but I don't see a fix in the main branch.

Ah I see, they didn't reference either of the issues connected to this so I missed it! I will find a comfortable place to step in sometime.


Feature: Whole volume validation

It would be very helpful to do full volume validation during training. This would involve running one of the Aggregators instead of the Samplers mid-training and using the new niftynet.engine.signals to trigger the evaluation. The result would serve as a template for future signals to run during training.

on a side note there's a very brief overview of niftynet.engine.signal at https://niftynet.readthedocs.io/en/dev/extending_event_handler.html

I would like to do full volume validation during training. Is that now possible in some way? It would be really helpful!

Hi @tomvars
Any progress on that? Irme is keen to work on it during the hackathon.
Irme and @ericspod - could you have a chat about it and plan what are the required changes?

@Irme and I have discussed. She has a solution which isn't compatible with just making a PR. We will be changing the code a lot during the hackathon which will make that harder. We feel that figuring out what the solution should afterwards is the way forward.


Svg Colors incorrect since SkiaSharp 1.57.1 on some Android devices

I tested two libraries ffimagedloading and XamaRed.Forms.Svg to show svg images in Xamarin Forms.
Both use SkiaSharp and both worked fine up to SkiaSharp 1.56.1. After SkiaSharp 1.57.1 was released colors of svg images on some android devices are messed up.
See:
https://github.com/luberda-molinet/FFImageLoading/issues/605
and
https://github.com/Brainflab/XamaRed.Forms.Svg/issues/3#issuecomment-301759899

I did more tests. I upgraded step bij step and this issue was introduced between 1.56.1 and 156.2.
So 1.56.1 all is fine:

With 1.56.2 colors are messed up:

I added a small example solution
SvgTest (2).zip

Thanks for this issue. I am looking at this now. Not quite sure exactly what is going on, but I think that your observation of the change in v1.56.1 and 1.56.2 helps out a great deal. I was thinking the Google changed something, but it now appears that there may be some issue with the way in which the SVG is drawn.
Closing this one as I believe it is a duplicate of #284  If you want to follow the progress, you can just watch that issue.

Just pushed out a release that should resolve this: https://github.com/mono/SkiaSharp/releases/tag/v1.58.1


Revoke host pattern

Description
Given a host pattern, disable/revoke access for all users.
Acceptance criteria

Given a host pattern, no user should have access to all the hosts matching that host pattern

This will be changed by group, no more host pattern based access, instead group based access. No more relevant.


Adds ZTA and clustering to deployment groups

@josevalim for online deployment we will only use that information to generate the agent docker file?


[coopr-539] info log on CREATE, DELETE events for provisioner added.

According to https://issues.cask.co/browse/COOPR-539

@wolf31o2 Updated.

:+1: LGTM... Thanks!


update doc string for grid parameter

updated comments for grid param

Codecov Report

Merging #2647 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2647      +/-   ##
==========================================
- Coverage   37.64%   37.63%   -0.02%
==========================================
  Files          82       82
  Lines        7254     7254
  Branches      710      710
==========================================
- Hits         2731     2730       -1
- Misses       4385     4386       +1
  Partials      138      138

Impacted Files
Coverage Δ

src/data/sparse_page_raw_format.cc
82.69% <0%> (-1.93%)
:arrow_down:

src/predictor/cpu_predictor.cc
72.25% <0%> (-1.3%)
:arrow_down:

src/objective/regression_obj.cc
87.43% <0%> (+0.54%)
:arrow_up:

src/data/sparse_page_dmatrix.cc
87.43% <0%> (+0.54%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39adba5...da68d4c. Read the comment docs.


IPTV data URL

I think the data URL should be: https://iptv-org.github.io/api/channels.json
Instead of: https://iptv-org.github.io/iptv/index.nsfw.m3u (not available anymore)
The "playlistGenerator.js" script should be modified to parse this file.

It has been some time since I worked on this project. If you would like to see the changes immediately, Please go ahead and create a PR. If you want me to do it, I'll take a look at this by next week.

Don't feel forced to do anything or do it soon. I'm still working on other things related to IPTV (such as creating an IPTV datasource for my Conzept project). I wanted to use your project as a channel "renderer". So take your time. Thanks!!

@waldenn There are inconsistencies between Channels API and Streams API. There are channels in Channels API which do not have an equivalent steam available on Streams API and vise versa. As an interim solution I've updated the scraped playlist to https://iptv-org.github.io/iptv/index.m3u which is currently available till I find a better solution.

Thank you Shiram, works great!


Crash HTC One M8

It's only crashing on this device:
System.ArgumentException: Error setting key TEST with value True ---> System.ArgumentNullException: Value cannot be null.
Parameter name: key
  at System.ThrowHelper.ThrowArgumentNullException (System.ExceptionArgument argument) [0x00000] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/mscorlib/system/throwhelper.cs:80
  at System.Collections.Generic.Dictionary`2[TKey,TValue].Insert (TKey key, TValue value, System.Boolean add) [0x0000b] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/mscorlib/system/collections/generic/dictionary.cs:322
  at System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) [0x00000] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/mscorlib/system/collections/generic/dictionary.cs:192
  at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] keySelector, System.Func`2[T,TResult] elementSelector, System.Collections.Generic.IEqualityComparer`1[T] comparer) [0x0004d] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/System.Core/System/Linq/Enumerable.cs:881
  at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] keySelector, System.Func`2[T,TResult] elementSelector) [0x00000] in /Users/builder/data/lanes/4468/f913a78a/source/mono/mcs/class/referencesource/System.Core/System/Linq/Enumerable.cs:873
  at Acr.Settings.SettingsImpl.NativeValues () [0x00011] in C:\dev\acr\settings\src\Acr.Settings.Android\SettingsImpl.cs:132
  at Acr.Settings.AbstractSettings.OnChanged (Acr.Settings.SettingChangeEventArgs args) [0x00013] in C:\dev\acr\settings\src\Acr.Settings.Interface\AbstractSettings.cs:165
  at Acr.Settings.AbstractSettings.Set[T] (System.String key, T value) [0x0004d] in C:\dev\acr\settings\src\Acr.Settings.Interface\AbstractSettings.cs:119

Android Version 5.0.1
Acr.Settings Version 6.0.0. Previous version has the same problem.

I'm not really sure how to help you with this since I don't have access to this specific device.  The only thing I can suggest is to debug with the unit tests and report back.

Ah, I was wrong, it looks like version 6.0.0 fixed this issue :)
Thank you

I somehow doubt v6 fixed anything.  I did make some slight adjustments to the android implementation, but nothing that would have prevented the error you've sent.


Support TypeScript asserts keyword, syntax error created on format

There is a bug in Deno 0.33 dprint:
export function assert(condition: any, msg?: string): asserts condition {
  if (!condition) {
    throw new Error(msg);
  }
}

when formatted becomes:
export function assert(condition: any, msg?: string): condition {
  if (!condition) {
    throw new Error(msg);
  }
}

which is a syntax error.
'condition' refers to a value, but is being used as a type here.ts(2749)

@David-Else Thanks for reporting! I didn't know it could be used that way (only knew about asserts condition is X). Will fix this soon.


Remove an unused parameter

This PR removes an unused parameter in an auxiliary function.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>8
Details

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 87.854%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1056

Relevant Lines:
1202

💛  - Coveralls


 Instead, some assertions are being called on filter creation.
Repro steps:
Using Istio with docker.io/istio/proxyv2:1.0.3 of the container.
Add this config:
---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ingress-ip-restrictions
  namespace: istio-system
spec:
  workloadLabels:
    istio: ingressgateway
  filters:
    - filterName: envoy.filters.network.rbac
      filterType: NETWORK
      filterConfig:
        rules:
          action: ALLOW
          policies:
            "ip-restrictions":
              permissions:
                - any: true
              principals:
                - source_ip:
                  address_prefix: <IP_ADDRESS>
                  prefix_len: 32
                - source_ip:
                  address_prefix: <IP_ADDRESS>
                  prefix_len: 32

        stat_prefix: ingress-ip-restrictions

(Note that the indentation is incorrect so the source_ip won't be a valid CIDR. Also saw similar crashes when principals was a hash not an array, etc.)
Call Stack:
[2018-12-17 04:16:52.963][47][info][main] external/envoy/source/server/server.cc:401] all clusters initialized. initializing init manager
[2018-12-17 04:16:52.968][47][critical][assert] external/envoy/source/extensions/filters/common/rbac/matchers.cc:53] panic: not reached
[2018-12-17 04:16:52.968][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:125] Caught Aborted, suspect faulting address 0x2f
[2018-12-17 04:16:52.969][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:94] Backtrace thr<0> obj</lib/x86_64-linux-gnu/libc.so.6> (If unsymbolized, use tools/stack_decode.py):
[2018-12-17 04:16:52.969][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<0> #0 0x7f70860ca428 (unknown)
[2018-12-17 04:16:52.969][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<0> #1 0x7f70860cc029 (unknown)
[2018-12-17 04:16:52.969][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<0> obj</usr/local/bin/envoy>
[2018-12-17 04:16:52.973][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #2 0x6a7876 Envoy::Extensions::Filters::Common::RBAC::Matcher::create()
[2018-12-17 04:16:52.976][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #3 0x6a7a5c Envoy::Extensions::Filters::Common::RBAC::OrMatcher::OrMatcher()
[2018-12-17 04:16:52.980][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #4 0x6a5548 Envoy::Extensions::Filters::Common::RBAC::RoleBasedAccessControlEngineImpl::RoleBasedAccessControlEngineImpl()
[2018-12-17 04:16:52.984][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #5 0x6a3d25 Envoy::Extensions::NetworkFilters::RBACFilter::RoleBasedAccessControlFilterConfig::RoleBasedAccessControlFilterConfig()
[2018-12-17 04:16:52.987][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #6 0x6a3586 Envoy::Extensions::NetworkFilters::RBACFilter::RoleBasedAccessControlNetworkFilterConfigFactory::createFilterFactoryFromProtoTyped()
[2018-12-17 04:16:52.991][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #7 0x6a24e8 Envoy::Extensions::NetworkFilters::Common::FactoryBase<>::createFilterFactoryFromProto()
[2018-12-17 04:16:52.995][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #8 0x7a6363 Envoy::Server::ProdListenerComponentFactory::createNetworkFilterFactoryList_()
[2018-12-17 04:16:52.998][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #9 0x7aee76 Envoy::Server::ListenerImpl::ListenerImpl()
[2018-12-17 04:16:53.002][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #10 0x7b049b Envoy::Server::ListenerManagerImpl::addOrUpdateListener()
[2018-12-17 04:16:53.006][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #11 0x7bf6f0 Envoy::Server::LdsApiImpl::onConfigUpdate()
[2018-12-17 04:16:53.010][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #12 0x7c178f Envoy::Config::GrpcMuxSubscriptionImpl<>::onConfigUpdate()
[2018-12-17 04:16:53.013][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #13 0x8fd4bc Envoy::Config::GrpcMuxImpl::onReceiveMessage()
[2018-12-17 04:16:53.015][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #14 0x8fab82 Envoy::Grpc::TypedAsyncStreamCallbacks<>::onReceiveMessageUntyped()
[2018-12-17 04:16:53.018][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #15 0x9250a5 Envoy::Grpc::AsyncStreamImpl::onData()
[2018-12-17 04:16:53.021][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #16 0x929dcb Envoy::Http::AsyncStreamImpl::encodeData()
[2018-12-17 04:16:53.023][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #17 0x966853 Envoy::Http::Http2::ConnectionImpl::onFrameReceived()
[2018-12-17 04:16:53.026][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #18 0x96a716 nghttp2_session_on_data_received
[2018-12-17 04:16:53.029][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #19 0x96e831 nghttp2_session_mem_recv
[2018-12-17 04:16:53.031][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #20 0x96641e Envoy::Http::Http2::ConnectionImpl::dispatch()
[2018-12-17 04:16:53.034][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #21 0x918fbe Envoy::Http::CodecClient::onData()
[2018-12-17 04:16:53.037][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #22 0x91913c Envoy::Http::CodecClient::CodecReadFilter::onData()
[2018-12-17 04:16:53.039][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #23 0x7d11d6 Envoy::Network::FilterManagerImpl::onContinueReading()
[2018-12-17 04:16:53.042][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #24 0x7cf85e Envoy::Network::ConnectionImpl::onReadReady()
[2018-12-17 04:16:53.045][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #25 0x7d004d Envoy::Network::ConnectionImpl::onFileEvent()
[2018-12-17 04:16:53.047][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #26 0x7c94a7 Envoy::Event::FileEventImpl::assignEvents()::{lambda()#1}::_FUN()
[2018-12-17 04:16:53.050][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #27 0xae4354 event_process_active_single_queue.isra.29
[2018-12-17 04:16:53.053][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #28 0xae4a9e event_process_active
[2018-12-17 04:16:53.055][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #29 0xae7827 event_base_loop
[2018-12-17 04:16:53.058][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #30 0x7989a3 Envoy::Server::InstanceImpl::run()
[2018-12-17 04:16:53.061][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #31 0x578600 Envoy::MainCommonBase::run()
[2018-12-17 04:16:53.065][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #32 0x422ea2 main
[2018-12-17 04:16:53.065][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<0> obj</lib/x86_64-linux-gnu/libc.so.6>
[2018-12-17 04:16:53.065][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<0> #33 0x7f70860b582f (unknown)
[2018-12-17 04:16:53.065][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<0> obj</usr/local/bin/envoy>
[2018-12-17 04:16:53.068][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #34 0x47bdd8 _start
[2018-12-17 04:16:53.068][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:121] end backtrace thread 0
2018-12-17T04:16:53.206258Z	warn	Epoch 0 terminated with an error: signal: aborted (core dumped)
2018-12-17T04:16:53.206298Z	warn	Aborted all epochs
2018-12-17T04:16:53.206371Z	info	Epoch 0: set retry delay to 3.2s, budget to 5

/cc @lizan @grosser

/cc @yangminzhu

First, I tried using static config. Evnoy handles this correctly with the following output:
$ envoy/source/exe/envoy-fastbuild  -c ~/Desktop/bad.yaml
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:206] initializing epoch 0 (hot restart version=10.200.16384.127.options=capacity=16384, num_slots=8209 hash=228984379728933363 size=2654312)
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:208] statically linked extensions:
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:210]   access_loggers: envoy.file_access_log,envoy.http_grpc_access_log
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:213]   filters.http: envoy.buffer,envoy.cors,envoy.ext_authz,envoy.fault,envoy.filters.http.header_to_metadata,envoy.filters.http.jwt_authn,envoy.filters.http.rbac,envoy.grpc_http1_bridge,envoy.grpc_json_transcoder,envoy.grpc_web,envoy.gzip,envoy.health_check,envoy.http_dynamo_filter,envoy.ip_tagging,envoy.lua,envoy.rate_limit,envoy.router,envoy.squash
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:216]   filters.listener: envoy.listener.original_dst,envoy.listener.proxy_protocol,envoy.listener.tls_inspector
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:219]   filters.network: envoy.client_ssl_auth,envoy.echo,envoy.ext_authz,envoy.filters.network.dubbo_proxy,envoy.filters.network.rbac,envoy.filters.network.sni_cluster,envoy.filters.network.thrift_proxy,envoy.http_connection_manager,envoy.mongo_proxy,envoy.ratelimit,envoy.redis_proxy,envoy.tcp_proxy
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:221]   stat_sinks: envoy.dog_statsd,envoy.metrics_service,envoy.stat_sinks.hystrix,envoy.statsd
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:223]   tracers: envoy.dynamic.ot,envoy.lightstep,envoy.tracers.datadog,envoy.zipkin
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:226]   transport_sockets.downstream: envoy.transport_sockets.alts,envoy.transport_sockets.capture,raw_buffer,tls
[2018-12-18 22:45:20.151][002729][info][main] [source/server/server.cc:229]   transport_sockets.upstream: envoy.transport_sockets.alts,envoy.transport_sockets.capture,raw_buffer,tls
[2018-12-18 22:45:20.162][002729][info][main] [source/server/server.cc:271] admin address: <IP_ADDRESS>:15000
[2018-12-18 22:45:20.164][002729][info][config] [source/server/configuration_impl.cc:50] loading 0 static secret(s)
[2018-12-18 22:45:20.164][002729][info][config] [source/server/configuration_impl.cc:56] loading 1 cluster(s)
[2018-12-18 22:45:20.167][002729][info][upstream] [source/common/upstream/cluster_manager_impl.cc:136] cm init: all clusters initialized
[2018-12-18 22:45:20.167][002729][info][config] [source/server/configuration_impl.cc:67] loading 1 listener(s)
[2018-12-18 22:45:20.171][002729][critical][main] [source/server/server.cc:86] error initializing configuration '/home/ymzhu/Desktop/bad.yaml': Unable to parse JSON as proto (INVALID_ARGUMENT:(rules.policies[0].value.principals[1]) address_prefix: Cannot find field.): {"rules":{"action":"ALLOW","policies":{"ip-restrictions":{"permissions":[{"any":true}],"principals":[{"prefix_len":32,"source_ip":null,"address_prefix":"<IP_ADDRESS>"},{"prefix_len":32,"source_ip":null,"address_prefix":"<IP_ADDRESS>"}]}}}}
[2018-12-18 22:45:20.171][002729][info][main] [source/server/server.cc:507] exiting

Then I tried using the EnvoyFilter CRD and the Envoy just crashed with the same message. It seems the proto validation is not working correctly in this case.
Last, I tried to correct the indention but introduce a mistake to the field name (Note the prefix_lenbad field):
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ingress-ip-restrictions
  namespace: default
spec:
  workloadLabels:
    app: sleep
  filters:
    - filterName: envoy.filters.network.rbac
      filterType: NETWORK
      filterConfig:
        rules:
          action: ALLOW
          policies:
            "ip-restrictions":
              permissions:
                - any: true
              principals:
                - source_ip:
                    address_prefix: <IP_ADDRESS>
                    prefix_len: 32
                - source_ip:
                    address_prefix: <IP_ADDRESS>
                    prefix_lenbad: 32
        stat_prefix: ingress-ip-restrictions

Surprisingly Envoy didn't crash this time and it actually accepts such config, the config dump show the following (Note the prefix_lenbad field):
"filters": [
         {
          "name": "envoy.filters.network.rbac",
          "config": {
           "stat_prefix": "ingress-ip-restrictions",
           "rules": {
            "action": "ALLOW",
            "policies": {
             "ip-restrictions": {
              "permissions": [
               {
                "any": true
               }
              ],
              "principals": [
               {
                "source_ip": {
                 "prefix_len": 32,
                 "address_prefix": "<IP_ADDRESS>"
                }
               },
               {
                "source_ip": {
                 "prefix_lenbad": 32,
                 "address_prefix": "<IP_ADDRESS>"
                }
               }
              ]
             }
            }
           }
          }
         },

This is not only happening for the RBAC filter because I also tried the following config using the ext_auth filter (Note the grpc_service_bad field which is a mistake):
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: extauth
  namespace: default
spec:
  workloadLabels:
    app: httpbin
  filters:
    - filterName: envoy.ext_authz
      filterType: NETWORK
      filterConfig:
        grpc_service_bad: "what"
        stat_prefix: abc

Envoy accepts the above config without any complaints which I thought the Envoy should have rejected it with a message like  (INVALID_ARGUMENT:grpc_service_bad: Cannot find field.).
As for the crash, it's directly caused by this line. We could remove it to avoid the crash but I don't think it's a good idea to do so. The correct way is to catch the config mistake in the validation stage.
@lizan @liminw @mattklein123
Is the different behavior between the static config and the dynamic config expected? I thought Envoy should handle the both cases in the same way. The mistake in the config should have been caught by the proto validation in Envoy. But it seems the proto validation is not working in the dynamic config case.

No, this is a bug. It should work the same way and we should definitely catch this during config load. Please fix!

@mattklein123 @yangminzhu Not it is not a bug, Istio runs envoy with --allow-unknown-fields for smooth upgrade, that means it will accept such configs with extra unknown fields like the examples above.

As for the crash, it's directly caused by this line. We could remove it to avoid the crash but I don't think it's a good idea to do so. The correct way is to catch the config mistake in the validation stage.

It's not a good idea to remove the line. PGV should prevent that method being invoked when validating the proto, or we missed a PGV rule in the proto definition.

@mattklein123 @yangminzhu Not it is not a bug, Istio runs envoy with --allow-unknown-fields for smooth upgrade, that means it will accept such configs with extra unknown fields like the examples above.

Sure but doesn't this still mean we are missing some constraint somewhere? I didn't read this bug very closely but even if we allow unknown fields shouldn't a constraint be in place to only allow known enum values, etc.? Is that what is missing?

@mattklein123 Right that's what I meant in https://github.com/envoyproxy/envoy/issues/5324#issuecomment-448443837

Got it, +1

@lizan Thanks for the information, super helpful!
The RBAC proto actually has the PGV rule option (validate.required) = true; for the principals field. Isn't this enough or do we need some other PGV rules here?

This seems due to unimplemented bug of PGV, since RBAC is using map for policies, it doesn't validate each element.


Fix neato device info

Breaking Change:
Maybe there are some minor changes since we use other sources to fill in device_info. But I can't imagine a use-case that could be broken by this change.
Description:
An endpoint of neao's API broke some time ago. There is a workaround where the error is caught (#29156). Now we use a different endpoint to fetch device information. We are now using get_general_info() instead of get_robot_info() as the last one is intended for debugging purposes.
Please include this in the next release once it has been accepted.
Related issue (if applicable): fixes #29239
Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.
[x] I have followed the development checklist

If the code communicates with devices, web services, or third-party tools:

[x] The manifest file has all fields filled out correctly. Update and include derived files by running python3 -m script.hassfest.
[x] New or updated dependencies have been added to requirements_all.txt by running python3 -m script.gen_requirements_all.
[x] Untested files have been added to .coveragerc.

I've tagged this for 0.103.0. Please remove this tag again, if this is not allowed.


bogus issue created by my cat

It's remarkable what a cat sitting on a keyboard can accomplish in a short amount of time.  I didn't see any way to outright delete issues, so I'm just closing them.


[rackspace cloud] Add cloudfiles backend for dragonfly, easy config via e

[rackspace cloud] Add cloudfiles backend for dragonfly, easy config via env variables like with s3.
This works (so far, going to do further click-testing.  The dragonfly changes I made pass their specs).  I would propose it not be merged until it can point at real dragonfly again, once mark evans merges my commits to that project.  Still, I'm about to be using it in ~production.

@knewter don't forget about this ;-)


sdist: use setuptools

Unfortunately distutils doesn't support any of install_requires, python_require, etc…

See also, #202.


colors for you!

Hey,
thanks for your quick response on the other issue.
I would like to give something back, so this is no issue per se, but just a new color scheme for you:
It suits really well with the rest of your colors, i think.
// Colors -----------------------------------
@hue-1:   hsl(205, 59%, 76%);  // <- cyan
@hue-2:   hsl(257, 19%, 52%);  // <- blue
@hue-3:   hsl(336, 30%, 42%);  // <- purple
@hue-4:   hsl(137, 26%, 55%);  // <- green
@hue-5:   hsl(318, 15%, 48%); // <- red
@hue-5-2: hsl(354, 89%, 35%);  // <- red 2
@hue-6:   hsl( 25, 59%, 76%);  // <- orange
@hue-6-2: hsl( 85, 59%, 76%); // <- yellow
cheers

Thanks!


[BUG]: wiki command issues

The bot's wiki commands sometimes yields unicode chars(*my terminology might be wrong here) and those look weird in the command. For now, try removing those characters by string manipulation

@Vyvy-vi
From discord. I will take this up. :)

ok fixed will push changes in few secs

um @anubhavcodes  This issue just got fixed in #33
Sorry...
If you want to still work on good issues, maybe you can build the help command


Fixed another bug in expression editor

fixed bug in expression editor causing
[----] F, [2015-03-09T15:10:06.000490 #1402:3ff23ae5fe90] FATAL -- : Error caught: [NoMethodError] undefined method `delete' for nil:NilClass
ManageIQ/manageiq/vmdb/app/controllers/application_controller/filter.rb:1420:in `exp_commit'
ManageIQ/manageiq/vmdb/app/controllers/application_controller/filter.rb:18:in `exp_button'

@romanblanco I'm not sure how to make this happen so I can verify it was fixed.  Can you please give me a recipe and I can then test.  Thx, Dan

@skateman Do you remember, how did you got this exception? I'm not able to reproduce this one.

@romanblanco emm, not really

Generally the extra test does not break anything. The code for the expression editor is pretty fragile and is a candidate for a rewrite. So by merging this we will surely not make the situation worse than now.


Fast sync stalls (due to mis-timing requests? or event processing order?)

trinity Version: trinity-v0.1.0-alpha.20-840-g3b52bdee (current master at commit 3b52bdee).
OS: Arch Linux
Python Version (python --version): 3.7.3
Environment (output of pip freeze): in gist

What is wrong?
This uses the single-preferred-peer set-up described in #501. trinity is in the initial fast-sync stage of getting block headers/bodies and transaction receipts.
Issue highlighted with newlines below:
   DEBUG  04-14 15:25:03   FastChainBodySyncer  Got block bodies for 128/128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>, from <BlockHeader #1774509 117f6be0>..<BlockHeader #1774636 7766dacb>
   DEBUG  04-14 15:25:04   FastChainBodySyncer  Requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 15:25:04        SkeletonSyncer  Skeleton header queue is 128/1536 full
   DEBUG  04-14 15:25:04      HeaderMeatSyncer  Header Skeleton Gaps: active=0 queued=0 max=50
   DEBUG  04-14 15:25:04   FastChainBodySyncer  (in progress, queued, max size) of bodies, receipts: [(128, 317, 2048), (0, 4096, 4096)]. Write capacity? yes
    INFO  04-14 15:25:04   FastChainBodySyncer  blks=0     txs=0      bps=125  tps=428   elapsed=5.0  head=#1766362 7ffe..6ab7  age=1y6m2w
   DEBUG  04-14 15:25:04   FastChainBodySyncer  Got block bodies for 128/128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>, from <BlockHeader #1774637 d89de87f>..<BlockHeader #1774764 3106366a>
   DEBUG  04-14 15:25:04   FastChainBodySyncer  Requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 15:25:04   FastChainBodySyncer  Got block bodies for 128/128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>, from <BlockHeader #1774765 85184232>..<BlockHeader #1774892 f5432c88>
   DEBUG  04-14 15:25:04   FastChainBodySyncer  Requesting block bodies for 61 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 15:25:05   FastChainBodySyncer  Got block bodies for 61/61 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>, from <BlockHeader #1774893 f8cf8278>..<BlockHeader #1774953 721b415b>
   DEBUG  04-14 15:25:09        SkeletonSyncer  Skeleton header queue is 128/1536 full
   DEBUG  04-14 15:25:09      HeaderMeatSyncer  Header Skeleton Gaps: active=0 queued=0 max=50
   DEBUG  04-14 15:25:09   FastChainBodySyncer  (in progress, queued, max size) of bodies, receipts: [(0, 0, 2048), (0, 4096, 4096)]. Write capacity? yes
    INFO  04-14 15:25:09   FastChainBodySyncer  blks=0     txs=0      bps=119  tps=407   elapsed=5.0  head=#1766362 7ffe..6ab7  age=1y6m2w

   DEBUG  04-14 15:25:09   FastChainBodySyncer  Requesting receipts for 245 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 15:25:10   FastChainBodySyncer  Timed out requesting receipts for 245 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 15:25:10  ResponseCandidateStream  Got unexpected Receipts payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>

   DEBUG  04-14 15:25:10   FastChainBodySyncer  Pausing ETHPeer <Node(0x053d@<IP_ADDRESS>)> for 15.0s, for sending 0 receipts
   DEBUG  04-14 15:25:14        SkeletonSyncer  Skeleton header queue is 128/1536 full
   DEBUG  04-14 15:25:14      HeaderMeatSyncer  Header Skeleton Gaps: active=0 queued=0 max=50
   DEBUG  04-14 15:25:14   FastChainBodySyncer  (in progress, queued, max size) of bodies, receipts: [(0, 0, 2048), (0, 4096, 4096)]. Write capacity? yes
    INFO  04-14 15:25:14   FastChainBodySyncer  blks=0     txs=0      bps=113  tps=386   elapsed=5.0  head=#1766362 7ffe..6ab7  age=1y6m2w
   DEBUG  04-14 15:25:19        SkeletonSyncer  Skeleton header queue is 128/1536 full
   DEBUG  04-14 15:25:19      HeaderMeatSyncer  Header Skeleton Gaps: active=0 queued=0 max=50
   DEBUG  04-14 15:25:19   FastChainBodySyncer  (in progress, queued, max size) of bodies, receipts: [(0, 0, 2048), (0, 4096, 4096)]. Write capacity? yes
    INFO  04-14 15:25:19   FastChainBodySyncer  blks=0     txs=0      bps=107  tps=367   elapsed=5.0  head=#1766362 7ffe..6ab7  age=1y6m2w
   DEBUG  04-14 15:25:24        SkeletonSyncer  Skeleton header queue is 128/1536 full
   DEBUG  04-14 15:25:24      HeaderMeatSyncer  Header Skeleton Gaps: active=0 queued=0 max=50
   DEBUG  04-14 15:25:24   FastChainBodySyncer  (in progress, queued, max size) of bodies, receipts: [(0, 0, 2048), (0, 4096, 4096)]. Write capacity? yes
    INFO  04-14 15:25:24   FastChainBodySyncer  blks=0     txs=0      bps=102  tps=349   elapsed=5.0  head=#1766362 7ffe..6ab7  age=1y6m2w

   DEBUG  04-14 15:25:25   FastChainBodySyncer  Requesting receipts for 245 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 15:25:26   FastChainBodySyncer  Timed out requesting receipts for 245 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 15:25:26  ResponseCandidateStream  Got unexpected Receipts payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>

   DEBUG  04-14 15:25:26   FastChainBodySyncer  Pausing ETHPeer <Node(0x053d@<IP_ADDRESS>)> for 15.0s, for sending 0 receipts
   DEBUG  04-14 15:25:29        SkeletonSyncer  Skeleton header queue is 128/1536 full
   DEBUG  04-14 15:25:29      HeaderMeatSyncer  Header Skeleton Gaps: active=0 queued=0 max=50
   DEBUG  04-14 15:25:29   FastChainBodySyncer  (in progress, queued, max size) of bodies, receipts: [(0, 0, 2048), (0, 4096, 4096)]. Write capacity? yes
    INFO  04-14 15:25:29   FastChainBodySyncer  blks=0     txs=0      bps=97   tps=331   elapsed=5.0  head=#1766362 7ffe..6ab7  age=1y6m2w
    INFO  04-14 15:25:32           ETHPeerPool  Connected peers: 1 inbound, 0 outbound
    INFO  04-14 15:25:32           ETHPeerPool  Peer subscribers: 4, longest queue: ETHChainTipMonitor(0)
   DEBUG  04-14 15:25:32           ETHPeerPool  == Peer details ==
   DEBUG  04-14 15:25:32           ETHPeerPool  ETHPeer <Node(0x053d@<IP_ADDRESS>)>: uptime=0:00:05:43, received_msgs=1661, most_received=Transactions (cmd_id=18)(556)
   DEBUG  04-14 15:25:32           ETHPeerPool  client_version_string='Geth/v1.8.26-stable-cdae1c59/linux-amd64/go1.10.4'
   DEBUG  04-14 15:25:32           ETHPeerPool      BlockBodies: msgs=522  items=329907  rtt=0.16/0.37/0.09  ips=6060.98429  timeouts=7  quality=98
   DEBUG  04-14 15:25:32           ETHPeerPool      BlockHeaders: msgs=309  items=58366  rtt=0.05/0.14/0.02  ips=4044.76922  timeouts=5  quality=99
   DEBUG  04-14 15:25:32           ETHPeerPool      NodeData: None
   DEBUG  04-14 15:25:32           ETHPeerPool      Receipts: msgs=223  items=165505  rtt=0.12/0.35/0.08  ips=3999.75970  timeouts=9  quality=63
   DEBUG  04-14 15:25:32           ETHPeerPool  == End peer details ==

Note that it's always the same-size batch. (On one occasion, I've seen batch size increase, as there were more block bodies received in the interim. trinity then requests even more transaction receipts, and that keeps timing out, too.)
Note also that this might be remote-specific. (Haven't tested with a parity node yet.)
How can it be fixed
Not sure.

Split batch/bundle into smaller ones on timeout happening?
Increase timeout duration on timeout happening?

The receipt-timeout pattern keeps repeating, and (as expected) degrades that peer's quality:
   DEBUG  04-14 15:31:32           ETHPeerPool  == Peer details ==
   DEBUG  04-14 15:31:32           ETHPeerPool  ETHPeer <Node(0x053d@<IP_ADDRESS>)>: uptime=0:00:11:43, received_msgs=2413, most_received=Transactions (cmd_id=18)(1236)
   DEBUG  04-14 15:31:32           ETHPeerPool  client_version_string='Geth/v1.8.26-stable-cdae1c59/linux-amd64/go1.10.4'
   DEBUG  04-14 15:31:32           ETHPeerPool      BlockBodies: msgs=522  items=329907  rtt=0.16/0.37/0.09  ips=6060.98429  timeouts=7  quality=98
   DEBUG  04-14 15:31:32           ETHPeerPool      BlockHeaders: msgs=309  items=58366  rtt=0.05/0.14/0.02  ips=4044.76922  timeouts=5  quality=99
   DEBUG  04-14 15:31:32           ETHPeerPool      NodeData: None
   DEBUG  04-14 15:31:32           ETHPeerPool      Receipts: msgs=245  items=165505  rtt=0.12/0.35/0.08  ips=1294.05644  timeouts=31  quality=20
   DEBUG  04-14 15:31:32           ETHPeerPool  == End peer details ==

Another run shows this happening with block body requests instead of tx receipts. There's also some funky fiddling with ResponseCandidateStream on peer disconnect/re-connect.
Doing another run with separate trinity log, and restarting both nodes at the same time; will link when issue encountered.

A later experiment with modified datefmt (return milliseconds) shows this might be due to event ordering or something:
   DEBUG  04-14 19:57:05.547   FastChainBodySyncer  Requesting block bodies for 120 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:57:06.048   FastChainBodySyncer  Timed out requesting block bodies for 120 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:57:06.049  ResponseCandidateStream  Got unexpected BlockBodies payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
--
   DEBUG  04-14 19:57:06.054   FastChainBodySyncer  Requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:57:06.548   FastChainBodySyncer  Timed out requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:57:06.550  ResponseCandidateStream  Got unexpected BlockBodies payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
--
   DEBUG  04-14 19:57:21.560   FastChainBodySyncer  Requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:57:22.027   FastChainBodySyncer  Timed out requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:57:22.029  ResponseCandidateStream  Got unexpected BlockBodies payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
--
   DEBUG  04-14 19:57:39.080   FastChainBodySyncer  Requesting block bodies for 124 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:57:39.560   FastChainBodySyncer  Timed out requesting block bodies for 124 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:57:39.563  ResponseCandidateStream  Got unexpected BlockBodies payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
--
   DEBUG  04-14 19:58:08.715   FastChainBodySyncer  Requesting block bodies for 126 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:58:09.512   FastChainBodySyncer  Timed out requesting block bodies for 126 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:58:09.513  ResponseCandidateStream  Got unexpected BlockBodies payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
--
   DEBUG  04-14 19:58:09.585   FastChainBodySyncer  Requesting receipts for 195 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:58:10.347   FastChainBodySyncer  Timed out requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:58:10.351  ResponseCandidateStream  Got unexpected BlockBodies payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
--
   DEBUG  04-14 19:58:25.358   FastChainBodySyncer  Requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:58:26.240   FastChainBodySyncer  Timed out requesting block bodies for 128 headers from ETHPeer <Node(0x053d@<IP_ADDRESS>)>
   DEBUG  04-14 19:58:26.244  ResponseCandidateStream  Got unexpected BlockBodies payload from ETHPeer <Node(0x053d@<IP_ADDRESS>)>

Modifying trinity/protocol/eth/constants.py to contain:
# Max number of items we can ask for in ETH requests. These are the values used
# in geth and if we ask for more than this the peers will disconnect from us.
MAX_STATE_FETCH = 384
MAX_BODIES_FETCH = 64 # half
MAX_RECEIPTS_FETCH = 128 # half
MAX_HEADERS_FETCH = 192

... seems to alleviate (but not eradicate) this particular issue. I've had continuous sync for ~105 minutes now.

This might be justification for implementing dynamic tuning of request sizes based on historical peer data.  I've experienced something that feels similar though not the same as this, consistently seeing responses for receipts come back with significantly fewer than requested.

Just noting that I applied the same tuning for my isolated process sync
branch. I thought it was related to my changes but looks like there's more
to it.
On Mon, Apr 15, 2019, 18:09 Piper Merriam<EMAIL_ADDRESS>wrote:

This might be justification for implementing dynamic tuning of request
sizes based on historical peer data. I've experienced something that feels
similar though not the same as this, consistently seeing responses for
receipts come back with significantly fewer than requested.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ethereum/trinity/issues/502#issuecomment-483316778,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAfzlYdYK92nyitB9HO2m6-8u9i-jV0yks5vhKQggaJpZM4culrL
.

I swear we had this conversation before, but I can't find it. I think a part of the problem is the way this timeout tuning works:
https://github.com/ethereum/trinity/blob/05a1cdee3322d79a049684e2f1d99f7f06912dc6/trinity/protocol/common/managers.py#L100-L109
If the peer response time shifts (say, their connection gets a bit worse) and we have a tight bound on their timeout based on ^ then we will get a bunch of timeouts until the average time increases enough to get a response again. I am not really convinced that this adaptive timeout is buying us anything, and have it patched off in my local, from:
https://github.com/ethereum/trinity/blob/05a1cdee3322d79a049684e2f1d99f7f06912dc6/trinity/protocol/common/managers.py#L97
to
if True:

which improved things for me.

Hrm,  I think you're probably right and I'm just emotionally attached to that code.  My inclination is to suggest we just put a lower bound on that value so that the timeout never drops below some reasonable minimum which it currently can.  The only argument I have for this is that it ensures we still timeout quests promptly in the event that someone stops responding but I'll admit that argument feels weak.....

My inclination is to suggest we just put a lower bound on that value so that the timeout never drops below some reasonable minimum which it currently can.

Eh, it just adds another constant that we don't really know how to choose, and might be wrong depending on your network.

The only argument I have for this is that it ensures we still timeout quests promptly in the event that someone stops responding but I'll admit that argument feels weak.....

Typically when a peer actually drops off, we get some kind of error right away in the form of a connection lost, so yeah I'm still not really seeing the benefit. Given that it's causing us problems, it seems like reverting to the simpler thing that works is the right first step, and we can add an issue to come back to it later.

I swear we had this conversation before, but I can't find it.

@carver It might've been in the py-evm repo, or on gitter.

Just had same behaviour on regular full sync (also Ropsten). No detailed log, but here's a grepped snippet:
    INFO  04-18 21:19:57.567  RegularChainBodySyncer  Imported block 297441 (0 txs) in 0.02 seconds
(stuck here for an hour as in original report: extremely fast time-out, without recovery; node restarted)
    INFO  04-18 22:36:12.090  RegularChainBodySyncer  Imported block 297442 (63 txs) in 7.18 seconds
    INFO  04-18 22:36:19.462  RegularChainBodySyncer  Imported block 297443 (81 txs) in 7.37 seconds

Hypothesis:

trinity has gone through a streak of blocks light1 in computation/IO - many 0-tx ones;
trinity has reduced the allowed time-out to a very low value for that peer;
when a "heavy"1 block comes, the peer times out;
???
trinity does not adjust time-out.

1 (by its own measure; there might be yet-unknown inefficiencies that make the scale different than, say, for geth or parity)

Blind guess:
There may be a bug that counts local-side processing time as part of transmission time, from which the allowed time-out is determined.
Say, processing of "request answered/lapsed, adjust timer if needed" happens after "block imported". But time-out timer terminates block importing, the request is considered unanswered; then timer adjustment finally "happens", but it doesn't differ too much from the target. The request is repeated with the same time-out; the loop goes on forever.

Also:
https://github.com/ethereum/trinity/blob/05a1cdee3322d79a049684e2f1d99f7f06912dc6/trinity/protocol/common/managers.py#L114-L116
It is easily possible that the timeout is 0:
if time.perf_counter() - start_at > outer_timeout:
    timeout_remaining = 0
    payload_timeout = min(inner_timeout, timeout_remaining) # payload_timeout = 0

I suppose this got addressed by PR #568. Therefore closing.


added git commands for fork-and-pull Git workflow

Added all the git commands used for fork-and-pull Git workflow as mention by @eddiejaoude

btw, you could have pushed to that branch itself(the PR would have gotten updated, without needing the opening of a new one)

btw, you could have pushed to that branch itself(the PR would have gotten updated, without needing the opening of a new one)

Yeah thanks, was just avoiding lotta commits like updated Contributing.md


Cannot clone all branches to local

git clone only have the main branch.

git clone only have the main branch.

How do I clone all remote branches?


How to use plugins when connecting to Chartist via CDN ?

Hi - I’m using the suggested CDN code to link to Chartist.js.
If I want to use a plugin - e.g. Point Label Plugin - can I just do something like this in my header file?

(I downloaded the scripts for the Point Label Plugin from here: https://github.com/gionkunz/chartist-plugin-pointlabels
and then added chartist-plugin-pointlabels.js to my admin/js scripts folder )
I then tried to create a simple line chart like this:
var data2 = {
  // A labels array that can contain any sort of values
  labels: ['Mon', 'Tue', 'Wed', 'Thu', 'Fri'],
  // Our series array that contains series objects or in this case series data arrays
  series: [
    [5, 2, 4, 2, 0]
  ]
};

new Chartist.Line('.ct-chart2', data2, {
  plugins: [
    Chartist.plugins.ctPointLabels({
      textAnchor: 'middle'
    })
  ]
});

however it doesn’t work ..
The chart is blank .. (until I remove the plugins code - then it works) ..
Did I do something wrong when trying to include the plugin script?
Thanks if you can help!

Make sure you're loading the plugin script after Chartist is loaded and before you execute the above code.

Did this do the trick? If you're still experiencing issues please reopen the issue.

Hi - my header.php file includes the following:

I call this file at the beginning of my showcharts.php file (where I then run the JS to create my charts).
Will this work?
Andrew

On 28 Feb 2016, at 10:09, Gion Kunz<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Did this do the trick? If you're still experiencing issues please reopen the issue.
—
Reply to this email directly or view it on GitHub https://github.com/gionkunz/chartist-js/issues/587#issuecomment-189828528.

Do you have any errors? Maybe you can show the code that you're executring to create the chart


Align Configuration File Path with XDG Base Directory Specification

Is your feature request related to a problem?
The configuration file for mac-cleanup-py is currently stored at ~/.mac_cleanup_py, rather than in the directory specified by $XDG_CONFIG_HOME as recommended by the XDG Base Directory Specification. This practice clutters the home directory and does not comply with the common Unix ecosystem standards.
Describe the solution you'd like
To comply with the XDG Base Directory Specification, store the mac-cleanup-py configuration file in $XDG_CONFIG_HOME/mac-clean-up/mac_cleanup_py on macOS. This adjustment will keep the home directory tidy and make managing dotfiles easier.
Describe alternatives you've considered
One alternative is to add an interactive setting that lets users specify a custom configuration file path, enabling them to choose the desired location manually.
Additional Context
Following the XDG Base Directory Specification improves user experience by standardizing where configuration files are stored. This is particularly useful for users managing configurations across multiple systems with a bare dotfiles git repo. Below is the current code snippet from main.py that needs adjustment:
class EntryPoint:
    config_path = Path.home().joinpath(".mac_cleanup_py")

The program should first check if the environment variable $XDG_CONFIG_HOME is set. If it is, store the configuration in $XDG_CONFIG_HOME/mac-cleanup-py/mac_cleanup_py. If not, use~/.config/mac-cleanup-py/mac_cleanup_py since~/.config is the default for $XDG_CONFIG_HOME. Create the directory path if it doesn't exist.

added XDG_CONFIG_HOME support in dceedf3


Latest sqlite-web require extension none

with latest sqlite-web commits, extensions are expected and cannot be null. not a bug in this project, but has definitely an impact..
kms       | + /usr/bin/python3 /home/sqlite_web/sqlite_web.py -rx -e none -H <IP_ADDRESS> /db/pykms_database.db
kms       | Traceback (most recent call last):
kms       |   File "/home/sqlite_web/sqlite_web.py", line 898, in <module>
kms       |     main()
kms       |   File "/home/sqlite_web/sqlite_web.py", line 870, in main
kms       |     options.extensions)
kms       | AttributeError: 'Values' object has no attribute 'extensions'

@simonmicro called it

As suggested in pull request https://github.com/coleifer/sqlite-web/pull/91/files, a temporary fix may be included in Dockerfile to fix the problem.
sed -i '764s/--extension'\''/--extensions'\''/' /home/sqlite_web/sqlite_web.py


Implemented Functionality to search bar

Implemented Functionality to search bar
Issue #155  : A working Search Component solved
Type of PR

[ ] Bug fix
[X] Feature enhancement
[ ] Documentation update
[ ] Other (specify): ___________

Description
This pull request aims to add a working search bar which enables users to search through the items.
Checklist

Add X in the box to specify.
[X] I have performed a self-review of my code.
[X] I have tested the changes thoroughly before submitting this pull request.
[X] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[X] I have commented my code, particularly in hard-to-understand areas.

@swaraj-das Thank you for reviewing my pull request!

@Akshithsaai please give a vedio demo

@AnitSarkar123 ok I will do it rn

@AnitSarkar123
https://github.com/user-attachments/assets/6d302465-0b77-4a57-8246-ad66d0ed8740

Good 👍


Replace class based js/qa hooks with data attribute hooks

Currently we are using js-* and qa-* class based hooks for dom manipulation and testing. We should use data attributes instead and only use classes for styles.
<div class='js-youtube' would become <div data-js-hook='youtube'
<div class='qa-youtube' would become <div data-qa-hook='youtube'

Quick question, can you have multiple values in an attr? I thought you couldn't, making something like data-js-hook="hook-1 hook-2" impossible because it's treated as a single value.

https://css-tricks.com/multiple-attribute-values/

I'm curious what the benefit is.

The benefit is a separation of concerns (i.e. css is only for styling). There is a performance hit with this approach which should be considered.
There is a good article on the subject at http://toddmotto.com/data-js-selectors-enhancing-html5-development-by-separating-css-from-javascript/

I don't feel too strongly about this but I do feel strongly about saying "classes are only for styling". They have some semantic meaning which helps to identify and select certain nodes in the tree.

I wasn't trying to imply that they didn't have some semantic meaning. I just want us to avoid situations like the following (copied from youtube.com) :

Wow that is..... something else! ha.

Update in https://github.com/cfpb/cfgov-refresh/blob/flapjack/src/offices/_single.html, e.g. https://github.com/cfpb/cfgov-refresh/blob/flapjack/src/offices/_single.html#L113

And https://github.com/cfpb/cfgov-refresh/blob/flapjack/src/_includes/macros/activities-block.html#L22

I can't find any js- classes anymore and the last qa- class will be removed in https://github.com/cfpb/cfgov-refresh/pull/2119 ... closing this.

Quick question, can you have multiple values in an attr? I thought you couldn't, making something like data-js-hook="hook-1 hook-2" impossible because it's treated as a single value.

Woot! We have this now in https://github.com/cfpb/cfgov-refresh/blob/flapjack/cfgov/unprocessed/js/modules/util/data-hook.js


Volume name too long (must be no more than 63 characters)

Hi all,
I'm using the spark-operator: chart version: spark-operator-1.1.11 and app version: v1beta2-1.2.3-3.1.1
I'm getting this error with a Scheduled Spark Application.
failed to run spark-submit for SparkApplication bigdata-spark/storage-academy-argos-new-schema-1655202518288923417: WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark/jars/spark-unsafe_2.12-3.1.1.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
22/06/14 10:33:39 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
22/06/14 10:33:39 INFO SparkKubernetesClientFactory: Auto-configuring K8S client using current context from users K8S config file
22/06/14 10:33:40 WARN DriverServiceFeatureStep: Driver's hostname would preferably be storage-academy-argos-new-schema-1655202518288923417-c4b2938161c6af28-driver-svc, but this is too long (must be \u003c= 63 characters). Falling back to use spark-31e8518161c6b089-driver-svc as the driver service's name.
22/06/14 10:33:40 INFO KerberosConfDriverFeatureStep: You have not specified a krb5.conf file locally or via a ConfigMap. Make sure that you have the krb5.conf locally on the driver image.
Exception in thread \"main\" io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://<IP_ADDRESS>/api/v1/namespaces/bigdata-spark/pods. Message: Pod \"storage-academy-argos-new-schema-1655202518288923417-driver\" is invalid: [spec.volumes[5].name: Invalid value: \"storage-academy-argos-new-schema-1655202518288923417-prom-conf-vol\": must be no more than 63 characters, spec.containers[0].volumeMounts[5].name: Not found: \"storage-academy-argos-new-schema-1655202518288923417-prom-conf-vol\"]. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.volumes[5].name, message=Invalid value: \"storage-academy-argos-new-schema-1655202518288923417-prom-conf-vol\": must be no more than 63 characters, reason=FieldValueInvalid, additionalProperties={}), StatusCause(field=spec.containers[0].volumeMounts[5].name, message=Not found: \"storage-academy-argos-new-schema-1655202518288923417-prom-conf-vol\", reason=FieldValueNotFound, additionalProperties={})], group=null, kind=Pod, name=storage-academy-argos-new-schema-1655202518288923417-driver, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Pod \"storage-academy-argos-new-schema-1655202518288923417-driver\" is invalid: [spec.volumes[5].name: Invalid value: \"storage-academy-argos-new-schema-1655202518288923417-prom-conf-vol\": must be no more than 63 characters, spec.containers[0].volumeMounts[5].name: Not found: \"storage-academy-argos-new-schema-1655202518288923417-prom-conf-vol\"], metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
  at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:589)
  at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:528)
  at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:492)
  at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:451)
  at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:252)
  at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:879)
  at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:341)
  at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:84)
  at org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:139)
  at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$3(KubernetesClientApplication.scala:213)
  at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$3$adapted(KubernetesClientApplication.scala:207)
  at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2611)
  at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:207)
  at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:179)
  at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
  at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
  at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
  at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
  at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030)
  at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039)
  at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
22/06/14 10:33:41 INFO ShutdownHookManager: Shutdown hook called
22/06/14 10:33:41 INFO ShutdownHookManager: Deleting directory /tmp/spark-5dedfe5f-637a-47a5-9050-eed3d4f1c96e

I know the name storage-academy-argos-new-schema-1655202518288923417-prom-conf-vol is longer than 63 chars, but there is any way to overwrite this long suffix (UnixNano-prom-conf-vol)?

@robertogyn19 Why and where was the storage created?
Assuming that you are running Spark3.1.1, the issue stems from apache/spark here: https://github.com/apache/spark/blob/v3.1.3/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala#L139
So this is clearly not a spark-operator issue but a apache/spark argocd issue.
But overall I think the issue stems from your declaration file. Can you please share it?

Hi @tahesse, I create a declaration to reproduce this error. I think this only applies to scheduled applications.
Seems like the controller adds a suffix with the schedule name and a time.UnixNano here. Later on the suffix "-prom-conf" is concatenated with the application name and finally the "-vol" is concatenated to the prometheus volume name.
Maybe the same logic applied here can be used with prometheus configuration?
apiVersion: "sparkoperator.k8s.io/v1beta2"
kind: ScheduledSparkApplication
metadata:
  name: spark-pi-prometheus-with-a-long-name
  namespace: spark
spec:
  schedule: "@every 1m"
  concurrencyPolicy: Forbid
  template:
    type: Scala
    mode: cluster
    image: "gcr.io/spark-operator/spark:v3.0.0-gcs-prometheus"
    imagePullPolicy: IfNotPresent
    mainClass: org.apache.spark.examples.SparkPi
    mainApplicationFile: "local:///opt/spark/examples/jars/spark-examples_2.12-3.1.1.jar"
    arguments:
      - "1000"
    sparkVersion: "3.1.1"
    restartPolicy:
      type: Never
    driver:
      cores: 1
      coreLimit: "1200m"
      memory: "512m"
      labels:
        version: 3.1.1
      serviceAccount: spark
    executor:
      cores: 1
      instances: 1
      memory: "512m"
      labels:
        version: 3.1.1
    monitoring:
      exposeDriverMetrics: true
      exposeExecutorMetrics: true
      prometheus:
        jmxExporterJar: "/prometheus/jmx_prometheus_javaagent-0.11.0.jar"
        port: 8090

I'm testing with the v3.0.0-gcs-prometheus image because the v3.1.1-gcs-prometheus doesn't exists in the gcr.io/spark-operator/spark.

@robertogyn19 I haven't had time yet to test, sorry. So why don't you just replace the UnixNano with some unique sha?
Other than that you could truncate the string parts to fulfill the 63 max char constraint while retaining uniqueness.
See https://dev.to/derfenix/comment/1hp14 for string truncation in golang.
The maintainers are slow to respond to PRs, so I'd suggest to do a PR and build the operator on your side as well in the meantime.


Fix bug on publish page with required questions

For this bug on Pivotal.
When the buyer frontend was updated to introduce one or two week
specialist briefs, the question was referring to briefs as being 'live'
and not 'open'. This was changed to 'open' instead of 'live' after the app was released which broke
the code which removed that particular question from the count of
unanswered questions. This update is needed to fix it.

It might be nice to have a test for when this message gets shown or not.

Cheers Kev, have added that test in.

👍  after rebase


Implemented RCL global argument parsing.

Also implemented methods for getting the name and namespace of nodes.

I have since this done a lot more work on my fork:

Made the clock available in .NET including setting the type
Implemented parameter handling (this must be done for each language specific library as per https://github.com/ros2/rcl/blob/1260197bb53353ce91a129a47ba0271eea2033a2/rcl/include/rcl/node.h#L88)
Implemented use_sim_time into Node.cs

All of this is currently merged into my dev branch at: https://github.com/Chootin/ros2_dotnet/tree/dev

@Chootin Thanks for your PR 👍
Currently I'm on vacation and getting things done at home, will look take a look at this when I'm back in office.

@Chootin Thanks for your PR 👍
Currently I'm on vacation and getting things done at home, will look take a look at this when I'm back in office.

No worries mate, enjoy the vacation!

I have implemented the requested changes, along with Node.FullyQualifiedName using some generics to cut down on duplicated code and provide an easy way to implement similar methods should the be required in the future.

Comment by @hoffmann-stefan addressed 👍

Thank you for your contribution 👍


Added PnP PowerShell translation of empty tenant recycle bin

Hi
I've just added a new translation of the existing empty tenant recycle bin using PnP PowerShell - just been need to do this at work so thought I'd add this quickly.
Thanks
Leon

Hey @LeonArmston , thank you, nice one!! 👏😊 This should be live shortly 😊


Ability to remove all notes from trash at once.

It would be nice to be able to empty trash clicking on one button. Deleting every note separately can be annoying.

I'm looking at this 👍

@siliconeidolon Are you already working on this. I am looking at taking it up.

I am working on it.

Closed in #146

@all-contributors please add @siliconeidolon for code, bug, tests


Add seeding base roles and actions

Create 3 roles: guest, registered, admin
Generate actions (based on names of the api routes)

Create tables: auth_actions, auth_groups, auth_group_action, auth_group_user
2.Run Make Models, Controllers, Transformers... commands