Media Foundation leaks .WEBM, .MKV, and .FLAC file handles

Windows 10 version 2004 Build 19041.572.
All WEBM, MKV, and FLAC file handles opened with MFCreateSourceReaderFromURL() remain permanently locked by the process even after all interfaces are properly released. This doesn't happen to other file types. This has remained a consistent bug for the past several Windows 10 releases.

Seeing this also in 1909 18363.1256

Seeing this also in 1909 18363.1256

Any update on this or suggested workaround?

I'm beginning to wonder if this was the most effective forum to report this but it doesn't seem like anyone is watching. I don't know where else to go with it, the bug is pretty obvious.

Yeah I was a little shocked to have to google my way into this random ticket to find anything about it. It was completely ruining a simple app I made that just transcodes files.
Perhaps report it in developercommunity.visualstudio.com under c++. That seems to be the bin where someone from Microsoft will pick it up and (hopefully) route it to the correct team.

Chrome /Chromium does / did it too when you play mp4 (or mkv renamed into mp4 as it cannot play mkv, but plays renamed file 😪🤣🤣). Does it use MFCreateSourceReaderFromURL()? We do use MFF in our Mojo video decoder.

@ValZapod Do you have more info to add to your comment? If we can track down this issue in Chromium then we could possibly work around it?

Do you have more info to add to your comment?

It was fixed in 86 Milestone. So no need ;)

@ValZapod .... do you have a link to the patch or? What I'm asking is how did they fix it, because for the rest of us out here running into this windows bug, since Microsoft seems uninterested in fixing it, it would be nice to be able to work around it like others have.

you have a link to the patch

Nope, sorry. You can try to search trough https://chromium-review.googlesource.com/ of course, but it may be very hard.

@bitcrazed can you shed any light on this issue please?

Ok, maybe @mattwojo or @zooba ?

I've sent a mail to find someone who owns this area. Stay tuned. We appreciate your patience.

Hey @TechnikEmpire & @ClosetBugSlayer
I've spoken with some of our engineers re. this issue. They've tried but failed to repro the issue you describe using the following code. Could you try this on your end and see if it works for you?
Alternatively, please share your repro case so our devs can take a look and see if there's anything they can suggest.
    auto startingHandleList = PrintAndReturnProcessHandleList(); // 👈 Base line open handles, no file
    {
        auto coInit = wil::CoInitializeEx(COINIT_MULTITHREADED);
        RETURN_IF_FAILED(MFStartup(MF_VERSION));
        unique_call<decltype(&::MFShutdown), ::MFShutdown> mfshutdown;

        {
            ComPtr<IMFSourceReader> sourceReader;
            RETURN_IF_FAILED(MFCreateSourceReaderFromURL(L"Dichterliebe Op. 48- Im wunderschÃ¶nen Monat Mai.flac", nullptr, &sourceReader));
            DWORD streamIndex = 0;
            DWORD streamFlags = 0;
            LONGLONG timestamp = 0;
            ComPtr<IMFSample> sample;
            sourceReader->ReadSample(MF_SOURCE_READER_ANY_STREAM, 0, &streamIndex, &streamFlags, &timestamp, &sample);
            auto middleList = PrintAndReturnProcessHandleList(); // 👈 This is while we are reading I see the file handle
            auto midDifference = startingHandleList.Diff(middleList);
            midDifference.PrintHandles();
        }
    }
    auto endingHandleList = PrintAndReturnProcessHandleList(); // 👈 Final list of handles, the file handle is no longer there.
    auto differenceHandleList = startingHandleList.Diff(endingHandleList);
    differenceHandleList.PrintHandles();

    // Note: `PrintAndReturnProcessHandleList()` queries the handles with NtQuerySystemInformationFunction and so should list all of them.

Thanks for the update. A puzzle indeed, I'll try to get a repro. However from your given example, I suspect that fully shutting down MF before checking the handles is hiding the issue, but I may be wrong.

In my code, every interface is released when closing the source reader but the handle to the file is still active in the process, I can see it in Process Explorer.  I am unable to delete the file from Explorer until my process is terminated. The control for this is MP3 and MP4 files which close correctly using the same code.

@ClosetBugSlayer It's really difficult to help diagnose a problem without repro steps. Please create and share a minimal repro and share so we can take a look.

@ClosetBugSlayer Do you have a specific file you were able to repro on? I'm unable so far to reproduce but where I originally observed this was a sort of drag-net, scooping up thousands of files. I'll try to hunt one of those streams down but thought I'd ask in the meantime.

OK I had it slightly wrong. After testing, MFCreateSourceReaderFromURL() isn't leaking FLAC handles. IMFSourceResolver.CreateObjectFromURL() is where my leak is happening, and I get the interface from MFCreateSourceResolver().
This C# interop chunk from my project doesn't directly compile but it should be very easy to follow:
`
Win32Api.MFCreateSourceResolver(out IMFSourceResolver sourceResolver).TryThrowError();
sourceResolver.CreateObjectFromURL(
_strFileSystemPath,
MediaFoundationSourceResolverFlags.MF_RESOLUTION_MEDIASOURCE
| MediaFoundationSourceResolverFlags.MF_RESOLUTION_CONTENT_DOES_NOT_HAVE_TO_MATCH_EXTENSION_OR_MIME_TYPE
| MediaFoundationSourceResolverFlags.MF_RESOLUTION_READ,
null,
out MF_OBJECT_TYPE type,
out IUnknown objMediaSource)
objMediaSource.Release();
`

@bitcrazed I'm not sure what happened, but I'm no longer able to reproduce this even in the original software where it was first observed. I also created a standalone example with full HW support etc, no repro. Sorry to take up your time for nothing!

Should be noted though that  I observed this issue with MFCreateSourceReaderFromURL, not MFCreateSourceResolver.

Should be noted though that I observed this issue with MFCreateSourceReaderFromURL, not MFCreateSourceResolver.

I could have sworn it was MFCreateSourceReaderFromURL but last night changed things. We're in the twilight zone now.

@ClosetBugSlayer & @TechnikEmpire - Thanks for investigating at your end.
Since this may have become a heisenbug with no repro, I'll close this issue for now to get it off our radar. If you ever do manage to create a repro, please feel free to ping this thread, share your repro case, and request that the bug be reopened.

@ClosetBugSlayer & @TechnikEmpire - Thanks for investigating at your end.
Since this may have become a heisenbug with no repro, I'll close this issue for now to get it off our radar. If you ever do manage to create a repro, please feel free to ping this thread, share your repro case, and request that the bug be reopened.

MediaFoundationLeakBug.zip
TADAAAAAA I attached a repro. I hope the comments in the .CPP file guide you through the repro successfully. Please reopen this bug and crush it mercilessly!

@ClosetBugSlayer nice, will run this. @bitcrazed

Thanks for sharing the repro case @ClosetBugSlayer  - re-opening while we take a look. I and a bunch of people are OOF on vacation at the moment, so there may be a delay in replying, but we are looking. Stay tuned.

Hey @ClosetBugSlayer & others on this thread: Summarizing some feedback from the team:
The handle leak is due to a subtle API usage bug. In both samples Shutdown() is not being called on the media source created by the source resolver.
The documentation clearly states that it’s the application’s responsibility to shutdown the media source and not doing so can lead to resource leaks:

“If the application creates the media source, either directly or through the source resolver, the application is responsible for calling Shutdown to avoid memory or resource leaks.”

Hope this helps resolve the issues reported.

Closing issue as per last comment


postive regulation of cAMP biosynthetic process

http://www.ebi.ac.uk/QuickGO/GTerm?id=GO:0030819#term=annotation
Lots of things related to adenylate cyclase regulation via GPCR signalling are annotated to "cAMP biosynthetic process"
I wonder if they should not be (this process/pathway doesn't synthesise  cAMP as such, its just an isomerization of ATP.
Should these terms exist?

@pgaudet  would this fit your def of metabolism?

ignore this one

Hi @ValWood
Why ignore this ? I agree that adenylate cyclase regulation via GPCR signalling should not be lined to cAMP biosynthetic process. It just doesn't sound correct. Is this being discussed in the signaling work @pfey03 ?
Thanks, Pascale

@pgaudet, @ValWood
adenylate cyclase activity GO:0004016
part_of cAMP biosynthetic process GO:0006171
AC catalyses the conversion of adenosine triphosphate (ATP) to 3',5'-cyclic AMP (cAMP) and pyrophosphate. In that sense YES, it is involved in the biosynthesis of cAMP

Yes I discussed with Antonia. Came to the conclusion that "cAMP metabolism" was OK, but "biosynthesis" wasn't  (because its just an interconversion)...yes I'd still like to discuss this then...
It will be a very full day!

Hmm, it does get rid of the pyrophosphate, so a true reaction not just interconversion in my mind

Hi @pfey03
If you consider this reaction metabolism, then the logical conclusion would be that any enzymatic activity should be metabolism (which if I understand correctly is more or less how GO is organized). I would prefer it if we stuck to primary and secondary metabolism for 'metabolism' but I that would probably involve a lot of changes in the ontology.
@ukemi @hdrabkin anyone else have input about this ?
Thanks, Pascale

I too think that this is NOT an interconversion. It's a biosynthesis of cAMP.

Yes it is. This is different from the other examples we discussed about metabolism.


Nested highlighting

Codecov Report

Merging #6 into master will decrease coverage by 0.26%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master       #6      +/-   ##
==========================================
- Coverage   94.72%   94.45%   -0.27%
==========================================
  Files          27       27
  Lines         379      379
  Branches       44       44
==========================================
- Hits          359      358       -1
- Misses          5        6       +1
  Partials       15       15

Impacted Files
Coverage Δ

...-solr-app/src/components/hit-views/DetailsView.tsx
89.79% <0%> (-2.05%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 179f2f9...054ad12. Read the comment docs.


feat(hacbs-1112): add unit testing for pipeline controller

Signed-off-by: srivickynesh<EMAIL_ADDRESS>

/retest


Crafting Recipe for making more of a type of block

Crafter Item + Requested type, and then the other materials 1-2 dyes, and the cobble / glass / glowstone. required.
Its a little unusual but would allow crafting using a crafting table.

Added in v4.3


Copy and Paste issue with closing brace

Version: 1.30.1 (system setup)
Commit: dea8705087adb1b5e5ae1d9123278e178656186a
Datum: 2018-12-18T18:12:07.165Z
Electron: 2.0.12
Chrome: 61.0.3163.100
Node.js: 8.9.3
V8: 6.1.534.41
Betriebssystem: Windows_NT x64 10.0.17763
Windows 10
See GIF

The closing brace go to wrong place after paste code, in second example.

Here is much more issues with autoindent. Should I create a new ticket or paste it here? Example

Here another variant with 1.33.0
Structure looks fine, but after copy&paste format is mismatch. After paste I press shortcut STRG+Z (german) for "undo" and its not remove the block, its formatted the block correctly!? A second "undo" remove the pasted block. Create formating new with starting from line beginning the copy&paste error is gone. See

The difference of indentation in a pasted block vs. receiving code gets applied to a line of code directly after paste location. Meaning: if you copy 8-spaces indented code and paste it into 4-spaces indented code, one of the original code lines incorrectly gets deindented by four spaces. If the copied code was indented less than the paste location, extra indent gets added to the next line. My testcases https://paste.zi.fi/p/vscodebug.py/view

This is due to the limited architecture (regex based indentation rules and only support limited code styles), we may want to consider using formatters if they exist for indenation adjustment (if interested, we can discuss in https://github.com/microsoft/vscode/issues/19847).
For this particular issue, if the indentation rules are leading to more trouble than having no auto indent, you can set editor.autoIndent to advanced to disable the regex based indentation adjustment or use auto formatting (editor.formatOnType, editor.formatOnPaste). As we don't have an easy good-enough fix for it, we close it for now and explore better solutions through issues like (https://github.com/microsoft/vscode/issues/19847 and https://github.com/microsoft/vscode/issues/34621).


Read more not visible in dark mode

Describe the bug/feature
The read more link is black in colour, when the dark mode is toggled, it is not clearly visible.
To Reproduce
Steps to reproduce the behavior:

Changing the colour of read more link when dark mode is toggled.

Expected behavior
A clear and concise description of what you expected to happen.
Logs
If applicable, add logs to help explain your problem (e.g. the error message and/or exception traceback).
Environment (please complete the following information):

OS: Windows
Browser: Chrome

Additional context
Add any other context about the problem here.

I would like to work on it under LGMSOC.

Hii @supzi-del we are holding to work on dark mode for now. When the frontend work will be almost done then we will update dark mode.


feat(aria): add aria labeling to the trigger

Alaska Airlines Pull Request
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes: #77
Summary:
This change is meant to get a valid aria labeling rule on the trigger element.
If a custom aria-label is applied at the root that will get applied to the trigger elem.
Else if a custom aria-labeledby is applied to the root that will get applied.
Else if the label slot has content that will get applied.
After reviewing the implementation of this with @geoffrich it was understood that all instances of auro-dropdown need to have aria-label or aria-labelledby on the trigger or the voiceover experience will not meet expectations. An update has been made to the demo page explaining this and all apiExamples have been updated to be compliant with this expectations.
Type of change:
Please delete options that are not relevant.

[x] New capability
[ ] Revision of an existing capability
[ ] Infrastructure change (automation, etc.)
[ ] Other (please elaborate)

Checklist:

[x] My update follows the CONTRIBUTING guidelines of this project
[x] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

@blackfalcon I am done making updates to this PR based on the conversation with @geoffrich. I think this is ready for a final review from you.

@geoffrich @jason-capsule42
I am curious as to this implementation. When I made this issue and linked to the aria-labeledby recommendation, the intent that I had was that we would use this to bind the content from the string in the label slot to this aria rule.
The error that I encountered was that since focusing on the trigger totally bypasses the label, it is never read back to the user.
The following is the update I had in mind.
  render() {
    return html`
      <div id="popover" class="popover" aria-live="polite" style=${`min-width: ${this.dropdownWidth}px;`}>
        <slot role="tooltip"></slot>

            <slot name="label"></slot>

            <slot name="trigger"></slot>

        ${this.chevron ? html`

            <auro-icon
              category="interface"
              name="chevron-down"
              customColor>
            </auro-icon>

        ` : undefined}

        <slot name="helpText"></slot>

    `;
  }
}

With this update, the content in the label slot will take priority over the use of aria-label at the root. But, using aria-label at the root address the use case for when the label slot is not used.
Where this does fall short is using aria-labeledby to content outside the scope of the element. But I am not sure that is a use case we need to support. Either the label slot is used or aria-label on the root is used instead of label content.
What is the advantage of the current solution versus what I am proposing?

@blackfalcon The update you made has two notable impacts:

The aria-labelledby attribute is left on the trigger content even when it's not going to be in use. This doesn't seem to cause a problem for VoiceOver on my mac. @geoffrich would you be able to check using NVDA?
It's no longer possible for a custom aria-labelledby. I'm not sure why we would choose to support a custom aria-label but not aria-labelledby.

Otherwise, the change seems fine.

What is the scenario for a custom labeledby?
The only thing I can think of is attaching this to something else in the DOM outside the scope of the element. If that is the case, that has to happen on the parent container of the element, aria does not pierce the shadow DOM.

:tada: This PR is included in version 2.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Type guard doesn't work on nested fields when wrapped in a function

Bug Report
Type guards are applied for some expressions referenced in a nested arrow function. This, however, isn't the case if the expression is a property access expression.
In particular, if I have the following code:
type NestedType = {
  field: number | string;
}

type NestedType = {
  field: number | string;
}
type NotNestedType = number | null;

const doNothingFn = <T>({
  fn,
} : {
  fn: () => T;
}) : T => {
  return fn();
}

the following type checks just fine:
const needsToReturnNum = (arg: NestedType):number => {
  return typeof arg.field === "number" ? arg.field : 0;
}

const needsToReturnNumNotNested = (arg: NotNestedType):number => {
  return typeof arg === "number" ? arg : 0;
}

const needsToReturnNumNotNested2 = (arg: NotNestedType):number => {
  return typeof arg === "number" ? doNothingFn({
    fn: () => arg,
  }) : 0;
}

This code, however, doesn't type check:
const needsToReturnNum2 = (arg: NestedType):number => {
  return typeof arg.field === "number" ? doNothingFn<number>({
    fn: () => arg.field,
  }) : 0;
}

with the following error:
Type 'string | number' is not assignable to type 'number'.
  Type 'string' is not assignable to type 'number'.

🔎 Search Terms
I used the search expression. is:issue is:open label:Bug type guard narrowing property access expression
🕗 Version & Regression Information
This fails in the typescript all versions available:

5.1.0-beta
5.0.4
4.9.5
4.8.4
4.7.4
4.6.4
4.5.5
4.4.4
4.3.5
4.2.3
4.1.5
4.0.5
3.9.7
3.8.3
3.7.5
3.6.3
3.5.1
3.3.3

⏯ Playground Link
Playground link with relevant code
note that JSX needs to be turned off/set to "None".
💻 Code
type NestedType = {
  field: number | string;
}
type NotNestedType = number | null;

const doNothingFn = <T>({
  fn,
} : {
  fn: () => T;
}) : T => {
  return fn();
}

const needsToReturnNum = (arg: NestedType):number => {
  return typeof arg.field === "number" ? arg.field : 0;
}

const needsToReturnNum2 = (arg: NestedType):number => {
  return typeof arg.field === "number" ? doNothingFn<number>({
    fn: () => arg.field,
  }) : 0;
}

const needsToReturnNumNotNested = (arg: NotNestedType):number => {
  return typeof arg === "number" ? arg : 0;
}

const needsToReturnNumNotNested2 = (arg: NotNestedType):number => {
  return typeof arg === "number" ? doNothingFn({
    fn: () => arg,
  }) : 0;
}

🙁 Actual behavior
This fails to type check.
🙂 Expected behavior
The code presented in the playground should type check.

Duplicate of #9998 (and therefore many others).   I think #52055 probably explains what's going on here more or less.


[Testing GH action] - Feeding dashboard with reassure data

Details
It is testing PR in which we want to test the GH action script which parses the reassure output.json and makes call to graphite server. The destination is not main but graphite-testing-script branch
After test are done we will proceed with the main PR -> https://github.com/Expensify/App/pull/36480
Flow for testing:

Create PR to graphite-testing-script
Merge PR
Observe reassurePerformanceTests GH action
After reassure test are completed:

it should call action: "Get and save graphite string"
It should make the call with graphiteString to the Graphite Server

Fixed Issues

$ https://github.com/Expensify/App/issues/29274
PROPOSAL: https://github.com/Expensify/App/issues/29274
Tests

[ ] Verify that no errors appear in the JS console

Offline tests

QA Steps

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

We're merging not into main but another branch so skipping tests


reconcile service resource

Signed-off-by: Ashley Schuett<EMAIL_ADDRESS>What this PR does / why we need it:
Changed reconciling of labels and annotations to be more general like the other resources. Mainly refactored the function to be more readable and also removed duplicate unit tests since they where in two separate files.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Reconcile service resource

/retest

Why did you decide to look for resource changes in this way? you deleted the snippet

// First check if there's anything to do.
if objectMatchesVersion(&cachedService.ObjectMeta, version, imageRegistry, id, r.kv.GetGeneration()) {
// spec and annotations are already up to date. Nothing to do
return patchOps, false, nil
}

which would first check the generation before doing any processing, by getting rid of that you have the reconciling loop do a bit more than what it would do before in the case no reconciliation is needed.

I'm not saying you should have used the expected generations approach we use for other resources as clearly both approaches work and it's a tradeoff between CPU and memory (generations store more info), but maybe we should keep the snippet above to skip processing this function if the version has not changed?

@acardace APIService does not have a generation field which is why I am doing a compare of the fields we expect to not change. The deleted snipit is actually just checking that labels/annotations are as we would expect which is now done in resourcemerge.EnsureObjectMeta(modified, existingCopy, desired)

Why did you decide to look for resource changes in this way? you deleted the snippet

// First check if there's anything to do.
if objectMatchesVersion(&cachedService.ObjectMeta, version, imageRegistry, id, r.kv.GetGeneration()) {
// spec and annotations are already up to date. Nothing to do
return patchOps, false, nil
}

which would first check the generation before doing any processing, by getting rid of that you have the reconciling loop do a bit more than what it would do before in the case no reconciliation is needed.

I'm not saying you should have used the expected generations approach we use for other resources as clearly both approaches work and it's a tradeoff between CPU and memory (generations store more info), but maybe we should keep the snippet above to skip processing this function if the version has not changed?

@acardace APIService does not have a generation field which is why I am doing a compare of the fields we expect to not change. The deleted snipit is actually just checking that labels/annotations are as we would expect which is now done in resourcemerge.EnsureObjectMeta(modified, existingCopy, desired)

Gotcha, now I see :+1:

/lgtm
But maybe https://github.com/kubevirt/kubevirt/pull/5459/commits/d02040e58913a530e03de8229fc598257a435ba6 and https://github.com/kubevirt/kubevirt/pull/5459/commits/a4071b5174e4ad0e4efb804943b70bc5c806ff5d belong to a separate PR? Why have they been introduced in this one?

/hold
https://github.com/kubevirt/kubevirt/pull/5459/files#diff-971364d0a5491acdd4a676a6d6e98c316e574b9479eebcc4773a282ac1c785e9R67-R68
should this trigger a delete and replace? If the "desired" service type and current service type are the same even if not of type ClusterIP that should be okay. I think this test should be removed

@ashleyschuett: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/lgtm

https://github.com/kubevirt/kubevirt/pull/5459/files#diff-971364d0a5491acdd4a676a6d6e98c316e574b9479eebcc4773a282ac1c785e9R67-R68

should this trigger a delete and replace? If the "desired" service type and current service type are the same even if not of type ClusterIP that should be okay. I think this test should be removed

@davidvossel I see you wrote the original test for this https://github.com/kubevirt/kubevirt/blame/d575b867afd043173ba385001dc85c846623c60d/pkg/virt-operator/install-strategy/create_test.go#L361-L383
I think it makes sense that we only reconcile ClusterIP service. however by patching if the types are equal but the desired type if not of type ClusterIP we end up in an infinite reconciliation loop. Is it okay to change this behavior and change this test so that deleteAndReplace should be false?

should this trigger a delete and replace? If the "desired" service type and current service type are the same even if not of type ClusterIP that should be okay. I think this test should be removed

I only tested that applying updates works with ClusterIP type, and that required me to understand the relationship between the dynamically set clusterIP and our cached resource.
I don't know how the other service types work. If they have defaults being dynamically set then we run the risk that our cached service won't match the one in the install strategy due to the defaulting values. That's why i took the conservative approach of delete/replace

@davidvossel thanks for the explanation. sounds good, I have removed my comment.
/unhold

/lgtm

/test pull-kubevirt-e2e-k8s-1.17-rook-ceph

/test pull-kubevirt-e2e-k8s-1.17-rook-ceph

/test pull-kubevirt-e2e-k8s-1.17-rook-ceph


[TW#27857] aws iot watchdog triggered during connect to aws

@mahavirj my code is based on aws iot example:
E (10340) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (10340) task_wdt:  - IDLE1 (CPU 1)
E (10340) task_wdt: Tasks currently running:
E (10340) task_wdt: CPU 0: IDLE0
E (10340) task_wdt: CPU 1: aws_iot_task

this issue is always in somewhere in this code:
    do {
        rc = aws_iot_mqtt_connect(&client, &connectParams);
        if(SUCCESS != rc) {
            ESP_LOGE(TAG, "Error(%d) connecting to %s:%d", rc, mqttInitParams.pHostURL, mqttInitParams.port);
            vTaskDelay(1000 / portTICK_RATE_MS);
        }
    } while(SUCCESS != rc);

    /*
     * Enable Auto Reconnect functionality. Minimum and Maximum time of Exponential backoff are set in aws_iot_config.h
     *  #AWS_IOT_MQTT_MIN_RECONNECT_WAIT_INTERVAL
     *  #AWS_IOT_MQTT_MAX_RECONNECT_WAIT_INTERVAL
     */
    rc = aws_iot_mqtt_autoreconnect_set_status(&client, true);
    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "Unable to set Auto Reconnect to true - %d", rc);
        abort();
    }

To not open new issue, this is related to aws iot, here are logs. Could someone explain this:
I (772892) wifi: bcn_timout,ap_probe_send_start
I (775397) wifi: ap_probe_send over, resett wifi status to disassoc
I (775397) wifi: state: run -> init (100)
I (775399) wifi: pm stop, total sleep time: 707105600 us / 771459313 us

I (775403) wifi: n:6 0, o:6 0, ap:255 255, sta:6 0, prof:1
I (775410) esp32_ap_eventHandler: wifi event no: 5
E (775949) aws_iot: failed! mbedtls_ssl_write returned -0x4e
W (775953) subpub: MQTT Disconnect
I (775953) subpub: Auto Reconnect is enabled, Reconnecting attempt will start now
I (777830) esp32_ap_eventHandler: wifi event no: 5
I (777952) wifi: n:6 0, o:6 0, ap:255 255, sta:6 0, prof:1
I (777953) wifi: state: init -> auth (b0)
I (777957) wifi: state: auth -> assoc (0)
I (777963) wifi: state: assoc -> run (10)
I (777979) wifi: pm start, type: 1

I (777982) esp32_ap_eventHandler: wifi event no: 4
I (778695) event: sta ip: <IP_ADDRESS>, mask: <IP_ADDRESS>, gw: <IP_ADDRESS>
I (778696) esp32_ap_eventHandler: wifi event no: 7
I (778698) subpub: AWS IoT SDK Version 3.0.1-
I (778703) subpub: Connecting to AWS...
Heap summary for capabilities 0x00000800:
  At 0x3ffca2d4 len 32767 free 12200 allocated 20492 min_free 0
    largest_free_block 9356 alloc_blocks 7 free_blocks 3 total_blocks 10
  At 0x3ffae6e0 len 6432 free 0 allocated 6300 min_free 0
    largest_free_block 0 alloc_blocks 25 free_blocks 0 total_blocks 25
  At 0x3ffc69a0 len 104032 free 200 allocated 103152 min_free 0
    largest_free_block 76 alloc_blocks 157 free_blocks 5 total_blocks 162
  At 0x3ffe0440 len 15072 free 76 allocated 14876 min_free 0
    largest_free_block 40 alloc_blocks 19 free_blocks 3 total_blocks 22
  At 0x3ffe4350 len 113840 free 28644 allocated 75920 min_free 872
    largest_free_block 21464 alloc_blocks 2265 free_blocks 46 total_blocks 2311
  At 0x40097cdc len 33572 free 33536 allocated 0 min_free 33528
    largest_free_block 33536 alloc_blocks 0 free_blocks 1 total_blocks 1
  Totals:
    free 74656 allocated 220740 min_free 34400 largest_free_block 33536
E (780918) aws_iot: failed! mbedtls_ssl_setup returned -0x7f00
E (780917) aws_iot: failed! mbedtls_ssl_setup returned -0x7f00
W (781931) subpub: MQTT Disconnect
W (781932) subpub: Auto Reconnect not enabled. Starting manual reconnect...

Basically i know whats going on, i have memory leak, then mbedTLS failed with -0x7f00, because lack of heap. What is strange to me its this:
W (775953) subpub: MQTT Disconnect
I (775953) subpub: Auto Reconnect is enabled, Reconnecting attempt will start now

W (781931) subpub: MQTT Disconnect
W (781932) subpub: Auto Reconnect not enabled. Starting manual reconnect...

@chegewara  (At first look) it looks like some form of memory corruption, how about putting watchpoint around pClient->clientStatus.isAutoReconnectEnabled?

@mahavirj could you help me with this?
I am subscribing to few topics:
I (14309) subscribe_topic: Subscribing... => /test/FNT/4
I (14403) subscribe_topic: Subscribing... => /test/ota
I (14504) subscribe_topic: Subscribing... => /all/ota
I (14607) subscribe_topic: Subscribing... => /test/reset

test/ota and all/ota are using the same callback function:
    memset(TOPIC, 0, 100);
    sprintf(TOPIC, "/%s/%s", name, "ota");
    rc = subscribe_topic(&client, TOPIC, ota_subscribe_callback_handler);
    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "Error subscribing : %d ", rc);
        abort();
    }

    rc = subscribe_topic(&client, "/all/ota", ota_subscribe_callback_handler);
    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "Error subscribing : %d ", rc);
        abort();
    }

The issue is that all topics works but /test/ota. I cant find the problem, the is no error during subscription, even if i stop using /all/ota i dont receive messages from /test/ota. I could assume that maybe /test topic is somehow filtered by aws, but /tst/reset works.

@chegewara Hi, could you help share if any updates for this issue? Thanks.

Yes, i can close it now. Most likely it was problem with my app (aws iot example code) or sdkconfig settings. Thanks.


Use molecule docker for default scenario

Use molecule docker driver for default scenario and provide addtional kvm and podman scenarios

added molecule docker as default scenario


StreamNotificationService | An error occurred while sending the request.

This is still spammed incessantly into the console even on v2.17.4.
If nothing else, at least the message might as well be hidden.

That gets a bit old real fast. 😜 I'd be more interested in a proper implementation that doesn't hit rate-limits constantly...but again, if nothing else, the message should at least be suppressed.
It is interesting to note that this seems to happen every ~2 hours or so.

Twitch? Are you using your own api key?

Yes Twitch only and yes I'm using my own API key.

From the next patch, stream notification will post a full error. Just post that when you see it.

waits patiently for next Windows release 😄(one of these days I'll spin up a Linux VM for this...)

Grabbed 2.18.1. Now we just play the waiting game... 👍

Where ya host? Local?
Verstuurd vanaf mijn iPad

Op 9 apr. 2018 om 21:15 heeft xnaas<EMAIL_ADDRESS>het volgende geschreven:
Grabbed 2.18.1. Now we just play the waiting game... 👍
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Ah, so I guess not much that can be done on the Nadeko side until it's sorted out on the dotnet side, eh? 😛

I have the same problem but its crashing when multiple (9+) errors generate within the same timestamp.

Since ya mention it, it's gotten worse recently. The bot doesn't even post offline notifications for streams anymore. 😂 Someday™

I have 9 twitch streams and 1 smashcast stream followed.
Many times I will generate 4 to 9 of these with the same exact timestamp. In a few instances, these errors have seemingly caused the bot to stop responding to commands, but will still generate errors. restart required to return to normal operation.
Sometimes causes bot to no longer be able to autodelete completed commands when set to do so when producing 60+ errors a minute. restart required to return to normal operation.

As an update on this, I've just stopped following streams with Nadeko entirely. It's just plagued with too many issues. Hopefully it can be sorted out in the future with updates to dotnet. 👍

I have done the same, as the crashes, lockups, and such are to much of a problem to smooth operations.

@xnaas issue resolved 🤔 ?

Per https://github.com/dotnet/corefx/issues/20066#issuecomment-386910537 this should be resolved in .NET Core 2.1. IIRC, Nadeko is still on 1.something, right?

I think nadeko is on 2.0.0, assuming of these two files:
search usedhttps://github.com/Kwoth/NadekoBot/search?q=dotnet&unscoped_q=dotnet
https://github.com/Kwoth/NadekoBot/blob/ebf114328c18353bb4af72e4ea69bba6acf32fd7/NadekoBot.Core/NadekoBot.Core.csproj#L42
https://github.com/Kwoth/NadekoBot/blob/41555aa629111b043fa8c1fbc26b23246f703c5b/src/NadekoBot/NadekoBot.csproj#L43
Suggest: Updating netcore to 2.1.x?

https://github.com/Kwoth/NadekoBot-BashScript/pull/38 This is addressed here by @samdivaio
I have also been using Nadeko (Docker) with .NET 2.1.4 for quite a while - months - no issues.

Soon™ / Someday™

@xnaas the pull https://github.com/Kwoth/NadekoBot-BashScript/pull/38 is merged!
Can you try to re-create the issue again and see if it persists?

The Windows version had the SDK updated/upgraded as well? I'll give it a try.

0 19:44:25 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 19:44:25 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 19:44:25 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 19:44:25 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 19:44:25 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()

I'm not getting the original message from this Issue at the moment, but still getting the above and no notifications in Discord ever.

A note on the above: this is with all entries in the database related to Twitch and streams being completely cleared out/reset.

No, sorry.  All of the streams were cleared from the database, reboot, add stream, start getting those.

@xnaas can you try it with the newest patch, I changed how httpclients are created.

Upgraded to 2.25.2. I'll let you know what happens once a stream notification tries to send.

Nope!
0 15:10:40 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 15:10:40 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 15:10:40 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 15:10:40 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 15:10:40 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()
0 15:10:40 StreamNotificationService | System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Searches.Services.StreamNotificationService.<GetStreamStatus>d__14.MoveNext()

No messages in Discord. Just the above errors.

Well, on the plus side, one notification made it through for the first time in forever. Still 99% errors tho. 😛

One more stream announcement made it through, but still just dozens and dozens of errors like the above.

Another update: It seems to be mildly successful at posting messages now. It usually posts it (sometimes will miss one or two that IFTTT actually got), but the logs are still full of errors as well.

Maybe there was a caching issue as it appears to be getting better about posting stream notifications. However, the logs are still full of errors. There doesn't appear to be any huge issues, so maybe they can just be suppressed, but idk.

Getting these too, but I don't know if it's related.
0 11:09:20 RemindService | Discord.Net.HttpException: The server responded with error 50007: Cannot send messages to this user
   at Discord.Net.Queue.RequestBucket.<SendAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.Net.Queue.RequestQueue.<SendAsync>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.API.DiscordRestApiClient.<SendInternalAsync>d__62.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.API.DiscordRestApiClient.<SendJsonAsync>d__59`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.API.DiscordRestApiClient.<CreateMessageAsync>d__79.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.Rest.ChannelHelper.<SendMessageAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.Rest.RestDMChannel.<Discord-IMessageChannel-SendMessageAsync>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NadekoBot.Modules.Utility.Services.RemindService.<ReminderTimerAction>d__15.MoveNext()

The main difference:
0 11:09:20 RemindService | Discord.Net.HttpException: The server responded with error 50007: Cannot send messages to this user

And now it's back to double-posting notifications. This is one hell of a rollercoaster lol

Whats the status now? Back to square 1? A new error persists? Is it maybe some specific streams that are causing errors or are other modules/processes bugging the console?

https://github.com/Kwoth/NadekoBot/issues/2212#issuecomment-405648198 is probably the most accurate still.

idk if @Kwoth shortened the error messages again, but it's back down to just this, at least, I guess:

It still messes a stream here or there, but it's better than before with double-posting and missing like 90% of the notifications. 😛

Linux hosting on GCP: I get 4 posts at once about the same stream, so it's spamming the channel.


Add Jack in Showcase

Hi, I fixed the indentation.
Regarding the infoLink tag, isn't my article on medium following your guidelines ? We are a startup, and I'm talking about our switch to your technology<EMAIL_ADDRESS>Thanks and regards

Do you have a source that shows this is a funded startup?

Well, Jack Media was founded in January 2015 and raised a total of EUR 1,790,000 to date (equivalent to ~ 1,820,000 USD).
We have not published public press releases about this, but this has been formalised through authenticated government and notarised documents. If needed, we can send you a copy of such a document (in french). Please note also the recent press coverage on Jack in Forbes (http://www.forbes.com/sites/montymunford/2017/01/12/850/#27511b894eac)
Alternatively, I can also ask my CEO to contact you and provide a formal/signed statement confirming the above.


installing on windows fails with LNK1107 (lua 5.2)

Can't install using luarocks on windows 10 with lua 5.2
c:\Users\soapdog\prog\lua\luarocks-2.4.0-win32>luarocks install luafilesystem
Installing https://luarocks.org/luafilesystem-1.6.3-2.src.rock
cl /nologo /MD /O2 -c -Fosrc/lfs.obj -IC:/lua/include src/lfs.c
lfs.c
link -dll -def:lfs.def -out:lfs.dll C:/lua/lua52.dll src/lfs.obj
Microsoft (R) Incremental Linker Version 14.00.24215.1
Copyright (C) Microsoft Corporation.  All rights reserved.

C:\lua\lua52.dll : fatal error LNK1107: invalid or corrupt file: cannot read at 0x330

Error: Build error: Failed compiling module lfs.dll

luabinaries has a folder for Static windows libraries:
https://sourceforge.net/projects/luabinaries/files/5.3.4/Windows Libraries/Static/
Those zip files contain lua53.lib. As described on #22, edit your config to point to that lib file.
You need the right one for your version of Visual Studio. See the README.windows.static.md file for more info.

How did you build your lua binaries? Seems the microsoft toolchain is used while building lfs, whereas the Lua binaries seem to have been build with MinGW or other gcc toolchains.

whoops, only now seeing this is a very old issue, got triggered by the previous comment.
@soapdog did you fix your issue? Can you post your resolution and close this issue if so?

@Tieske, I couldn't solve it myself. Ended up using Lua For Windows which has it already https://github.com/rjpcomputing/luaforwindows

I came across this error on Windows when, in the Luarocks configuration, both CC and LD were set to cl. I fixed it by keeping CC as cl, and changing LD to link.


⚠ RPC server - xDai has degraded performance

In 3dc273a, RPC server - xDai ($XDAI_RPC_URL) experienced degraded performance:

HTTP code: 200
Response time: 415 ms

Resolved: RPC server - xDai performance has improved in 8fa8b27 after 10 minutes.


Add node 4.0.0 to node image

Shippable/pm#4678
Add install script for node 4.0.0 script here - https://github.com/dry-dock/u14nod/tree/master/version
Update Dockerfile to call the script\
This should automatically kick off a Shippable build and update the image at https://hub.docker.com/r/drydock/u14nod/

done. closing.


"Use of closed network connection" error when using concurrently

Description
This is more a PSA than an actual bug report.
When sending messages using goroutines, sending randomly fails with an error like this:
send failed: sending SMTP MAIL FROM command: use of closed network connection, use of closed network connection, affected message ID: xxxx

To Reproduce
This is a simplified version:
client, err := gomail.NewClient(...)

for range 100 {
	go func() {
		m := gomail.NewMsg()
		m.FromFormat("name"<EMAIL_ADDRESS>		m.AddTo(toAddress)
		m.SetBodyString(gomail.TypeTextPlain, "Hello")
        	err := client.DialAndSend(m) // err here is not nil about 15-20% of the times
	}
}

Expected behaviour
No errors (or maybe one random error every now and then if the network or server is down).
Screenshots
No response
Attempted Fixes
Instantiate a NewClient for every message sent. This works.
Additional context
I have the feeling this library intends to be thread-safe but it's not (I guessed that because I see some mutexes). So maybe this is a bug.
I suggest making DialAndSend thread-safe, or clearly documenting that a client should not be used from multiple goroutines (or locking should be used).

Thanks for reporting @jgimenez. Threat-safety has been introduced in a recent release with the intention that the Client can be reused within the go-routines, so that you don't have to use DialAndSend but can just use Send on an open connection similar to what we do in this test:
                wg := sync.WaitGroup{}
                for id, curMessage := range messages {
                        wg.Add(1)
                        go func(curMsg *Msg, curID int) {
                                defer wg.Done()
                                if goroutineErr := client.Send(curMsg); err != nil {
                                        t.Errorf("failed to send message with ID %d: %s", curID, goroutineErr)
                                }
                        }(curMessage, id)
                }
                wg.Wait()

But I agree that DialAndSend should also be thread-safe to fullfil the claim. I'll have a look at this and hopefully introduce a fix soon.


HIVE-14815: Support vectorization for Parquet

This patch includes the following changes:

Implement a vectorized Page reader which support dictionary and RLE encoding.
Enable vectorization for Parquet input format.
Support several data types

If a lot of these class are borrowed from Spark (https://github.com/apache/spark/tree/master/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet), then could we simply just use the Spark classes as they are to implement the VectorizedParquetRecordReader. It would be better than forking the code because any bug fixes Spark makes to this code can automatically be picked up in Hive. If we need to make any modifications to the code for Hive, we can just extend the existing classes.

Hi @sahilTakiar , I think we may not be able to use the code in Spark directly since they're lazy materialized and we just keep the hierarchy: record -> column -> page.


Update Planbox urls

Updated URLs for endpoint and documentation

@mateomurphy Thanks for the fix! This has been deployed to production.


Additional Convenience methods

Adding more convenience methods for #207

Looks like this also needs a cargo fmt - you can use the included makefile to handle that and all of our other lints for you

Addressed the above :) Thanks a ton for the review, any suggestions on which issue to tackle next are appreciated

Addressed the above :) Thanks a ton for the review, any suggestions on which issue to tackle next are appreciated

Feel free to pick anything from the open issues, but #191 is closely related here


Confirming cli app only

Hello!
Thanks for a great tool.  I have an mp3 library I'm trying to wrangle into consistency and rnr has smoothed that path quite nicely.  I especially appreciate undo file/dry-run by default.
Anyway, I think it's not possible, but can rnr be incorporated into another rust app via a crate dependency, i.e. is there an api implementation for rnr?  Or is the only option to use rnr from within a rust program to call rnr via std process Command?
Thanks!

I'm glad that this tool is useful to you. ;)
Currently, this tool is only working as a CLI. However, I was planning to separate the core so it could be integrated in other interfaces (p.e. TUI, GTK UI, ...).
What functionality would be useful to be exposed for  your use case?

I'll let you know if I think of anything.  I've found it intuitive to use when I have.  Thanks!


Code compiling with 0.29.0 on LLVM 6.0.1 but not 3.9.1

When compiling a particular crystal program using crystal 0.29.0 available from Homebrew, there are no issues. That build uses LLVM 6.0.1. Using the tarball provided in the GitHub release for 0.29.0, which was compiled with LLVM 3.9.1, I get an error. Here's a sequence of commands to illustrate the problem:
╭─taylor@cintra ~/Code/thicket Crystal system ‹develop*›
╰─$ crystal -v
Crystal 0.29.0 (2019-06-06)

LLVM: 6.0.1
Default target: x86_64-apple-macosx

╭─taylor@cintra ~/Code/thicket Crystal system ‹develop*›
╰─$ crystal build --release src/runner.cr -o bin/thicket

╭─taylor@cintra ~/Code/thicket Crystal system ‹develop*›
╰─$ crenv local 0.29.0

╭─taylor@cintra ~/Code/thicket Crystal 0.29.0 ‹develop*›
╰─$ crystal -v
Crystal 0.29.0 (2019-06-05)

LLVM: 3.9.1
Default target: x86_64-apple-macosx

╭─taylor@cintra ~/Code/thicket Crystal 0.29.0 ‹develop*›
╰─$ crystal build --release src/runner.cr -o bin/thicket
Assertion failed: (cast<DISubprogram>(Scope)->describes(MF->getFunction())), function getOrCreateRegularScope, file /var/cache/omnibus/src/llvm/llvm-3.9.1.src/lib/CodeGen/LexicalScopes.cpp, line 159.
/Users/taylor/.anyenv/envs/crenv/versions/0.29.0/bin/crystal: line 103: 52413 Abort trap: 6           "$INSTALL_DIR/embedded/bin/crystal" "$@"

Is this something I'm doing wrong or not understanding?

We shouldn't still be using such an old llvm version.
@bcardiff maintains the OSX tarballs

Should new deployments of Crystal remove support or older llvm compilers?

I installed crystal through both brew and asdf, and it locked me in to LLVM 3.9.1.
Here's a small example showing this output:
command = "touch app.cr && crystal build --release app.cr"
process = Process.run(command, shell: true, output: IO::Memory.new, error: STDERR)

Assertion failed: (cast(Scope)->describes(MF->getFunction())), function getOrCreateRegularScope, file /var/cache/omnibus/src/llvm/llvm-3.9.1.src/lib/CodeGen/LexicalScopes.cpp, line 159.
--: line 1: 68570 Abort trap: 6           crystal build --release app.cr

I agree with the remove support for older ones. I seem to also have 4, 5, and 6 installed locally, but I'm not sure how to tell crystal to use one of those versions.

Did we update LLVM yet, I think so, no?

Darwin tar.gz tarballs ships llvm 6.0.1 since Crystal 0.30.0. Hombrew currently uses llvm 10
The compiler codebase still has reference and supposed support for older llvm but they are not checked by any CI.
For the record, you can't tell to a shipped crystal compiler which llvm to use, that is only available when building the compiler itself.


Fix: groups and permissions filter redirects

Steps to reproduce:

Create an admin group (if you don't already have one)
Register a new user and add to the group
Create two views that can only be accessible by this group, let's call them A and B
Filter these views (via the route or filter)
Login and visit page A.
Change the group of the logged in user, preferrably from the database directly
Try to visit page B.

What you'll notice is that the browser dies of too many redirects error.  This happens because the filter is redirecting the user back to the previous page, which at this point, the user no longer has enough privileges to access it.
A real-life scenario:
A forum website with multiple groups and permissions, an admin might decide to remove a certain permission from a user or remove the user from a certain group WHILE that user might be logged in. If this user visits another page that requires the same permission or group that s/he has been stripped of, then it causes the browser to die due to too many redirects.

Yes, you are correct @datamweb, I had it in mind to state it as part of my observation, but it completely skipped my mind.
I'll push a commit as per you suggestion.

On a second thought, I think the remainder of the code covers for the removed or deleted group.
This covers for an authenticated user: https://github.com/codeigniter4/shield/blob/49887d9eaa83b52449e9fc5218272b591747cc3c/src/Filters/AbstractAuthFilter.php#L32-L34
This covers for both valid groups and permissions:
https://github.com/codeigniter4/shield/blob/49887d9eaa83b52449e9fc5218272b591747cc3c/src/Filters/AbstractAuthFilter.php#L36-L38
And this will present a descriptive information for the user without forcing logout:
https://github.com/codeigniter4/shield/blob/49887d9eaa83b52449e9fc5218272b591747cc3c/src/Filters/AbstractAuthFilter.php#L47
Please check again and let me know what you think.

There were 2 failures:

1) Tests\Authentication\Filters\GroupFilterTest::testFilterIncorrectGroupWithPrevious
Redirect URL `https://example.com/index.php/open-route` does not match `https://example.com/index.php/`
Failed asserting that false is true.

/home/runner/work/shield/shield/vendor/codeigniter4/framework/system/Test/TestResponse.php:212
/home/runner/work/shield/shield/tests/Authentication/Filters/GroupFilterTest.php:79

2) Tests\Authentication\Filters\PermissionFilterTest::testFilterIncorrectGroupWithPrevious
Redirect URL `https://example.com/index.php/open-route` does not match `https://example.com/index.php/`
Failed asserting that false is true.

/home/runner/work/shield/shield/vendor/codeigniter4/framework/system/Test/TestResponse.php:212
/home/runner/work/shield/shield/tests/Authentication/Filters/PermissionFilterTest.php:79

https://github.com/codeigniter4/shield/actions/runs/3867406237/jobs/6592163575

I am not sure if there is a normal use case of accessing a page without permission.
So I don't see a problem with redirecting to /.

I'm just thinking out loud. Maybe this is not related to PR.
https://github.com/codeigniter4/shield/blob/49887d9eaa83b52449e9fc5218272b591747cc3c/src/Config/Auth.php#L48-L52
Why don't we have these two?
    public array $redirects = [
        'register' => '/',
        'login'    => '/',
        'logout'   => 'login',
        'afterPermissionDenied' => '/',
        'afterGroupDenied' => '/',
    ];

Why don't we have these two?

Ah @datamweb , I think that will be great. Developers can be able to set a custom URL (path, named route or absolute URL) for each of them.
I also agree with you that it should be another PR. I'll do that when #577 is merged.

@sammyskills It looks like you are using tag -S to sign each commit. This makes you forget to sign your commits in some cases.
To check further, run the following command in the terminal
git config --list

user.name=Samuel Asor
user.email=your emaillllllllll
user.signingkey=9079CC1BED930E60
commit.gpgsign=false
tag.forcesignannotated=true

If you see commit.gpgsign=false, please run the following command. From now on, all your commits will be signed automatically. And there is no need to use -S.
git config --global commit.gpgsign true

But In this PR, you have two commits that you must use gpg-signing-old-commits (all commit) given below to sign them.

@kenjis, please clarify, is this implementation what you had in mind?

I'm okay with this implementation.

Hi @datamweb ,
I've tried to follow the instruction on signing all commits, but each time I run the command, a new file opens up in my text editor and the terminal just stops. I've waited for hours, but it is still at the same spot, as if it is expecting an input.
I've checked online but didn't get any headway. Do you have a clearer documentation on the steps to take?

I run the command

Which command?

a new file opens up in my text editor and the terminal just stops. I've waited for hours,

I saw that it took a while for you to answer. You wait for the terminal, the terminal waits for you :blush:.
This is normal, because you have to save the changes in the editor, you can close the editor by pressing the q button and then look at the terminal.

Do you have a clearer documentation on the steps to take?

The easiest way is the following, considering your lack of familiarity with git rebase.
git fetch upstream
git switch develop
git merge upstream/develop
git switch fix/update-groups-permissions-redirects
git branch -m fix/update-groups-permissions-redirects fix/update-groups-permissions-redirects.bk1
git switch -c fix/update-groups-permissions-redirects develop
code .

After that, change src/Filters/AbstractAuthFilter.php and tests/Authentication/Filters/GroupFilterTest.php and tests/Authentication/Filters/PermissionFilterTest.php.
git add .
git commit -m "fix: groups and permissions filter redirects"
git push -f origin fix/update-groups-permissions-redirects

Thank you @datamweb.

@sammyskills thanks!
merge!


How to implement onStatsHandler in Java

Hi There,
akka-stream-mon looks promising and I'm excited to try it out. Sorry, there was no other way to get in touch with you. Two quick questions.
I'm wondering how I can use your library in java. Specifically, I wonder what I need to implement for the onStatsHandler? A small example in the read.me would be highly appreciated.
Thank you for putting this under open source.

In my case the onStatsHandler does a very simple thing which is it sends an akka message to a healthcheck actor which is the central actor that keeps all monitoring state, we then expose that state via REST and JMX separately.
Alternatively you could probably directly interact with some sort of an API to put the values inside the Stats object into the metrics API of your choice (ex. in Finagle it's StatsReceiver, Dropwizard has MetricRegistry etc.)
A very basic example how that could look like is
private BoxedUnit onStatsHandler(Throughput.Stats stats) {
  healthCheckActor.tell(stats, getSelf());
  return BoxedUnit.UNIT;
}

with call site (the flow)  which can be whatever your Flow implementation is
Flow.fromSinkAndSource(sink, src)
   .via(ThroughputMonitor.apply(FiniteDuration.create(15, TimeUnit.SECONDS), this::onStatsHandler);

You might want to have a think about what's the size of the time window for which to measure the throughput depending on how often your stats object will be polled or how often it pushes the data upstream.


Updated Emmiter Definition makeParticles() and particleClass

fixes #2257

Thanks. I can't merge because wrong branch, but have added the changes locally.


Support deploy to cloud platforms

[ ] Heroku

I'm actually researching this myself right now. Did any plugins come out of this, or do you know any that are already available?


eth_abi.exceptions.NoEntriesFound: No matching entries for 'tuple' in decoder registry

Seems to be like https://github.com/ethereum/web3.py/pull/2211
Sorry if this is not proper form.  I have literally started coding 2 months ago.
Replicate this with the boilerplate tutorial, and tx : 0xa139231454fd021dd227a94fff6a1b6260890bb95e5f5bf8517af36e228575e6

It's eth_abi.decode's problem.

Oh, I think I can fix it

It's been released in 0.1.5.
If you encounter any further problems on decoding tuple, feel free to reopen this issue :)

Thank you! =)

I just started getting this error using v0.1.5.
eth_abi.exceptions.NoEntriesFound: No matching entries for 'tuple' in decoder registry
Here is code:
`
from web3_input_decoder import InputDecoder
abi = [{'inputs': [{'internalType': 'uint256', 'name': 'amountIn', 'type': 'uint256'},
{'internalType': 'uint256', 'name': 'amountOutMin', 'type': 'uint256'}, {
'components': [{'internalType': 'address', 'name': 'from', 'type': 'address'},
{'internalType': 'address', 'name': 'to', 'type': 'address'},
{'internalType': 'bool', 'name': 'stable', 'type': 'bool'}],
'internalType': 'struct BaseV1Router01.route[]', 'name': 'routes', 'type': 'tuple[]'},
{'internalType': 'address', 'name': 'to', 'type': 'address'},
{'internalType': 'uint256', 'name': 'deadline', 'type': 'uint256'}],
'name': 'swapExactTokensForTokens',
'outputs': [{'internalType': 'uint256[]', 'name': 'amounts', 'type': 'uint256[]'}],
'stateMutability': 'nonpayable', 'type': 'function'}]
input_data = '0xf41766d80000000000000000000000000000000000000000000000000000000002dc6c000000000000000000000000000000000000000000000000001bbc2e22c5c1c2cb00000000000000000000000000000000000000000000000000000000000000a00000000000000000000000007c77b132a0cd0ad1c694ab8645affa26c2787d6600000000000000000000000000000000000000000000000000000000636aaa2f0000000000000000000000000000000000000000000000000000000000000001000000000000000000000000ea32a96608495e54156ae48931a7c20f0dcc1a21000000000000000000000000deaddeaddeaddeaddeaddeaddeaddeaddead00000000000000000000000000000000000000000000000000000000000000000000'
decoder = InputDecoder(abi)
decoded = decoder.decode_function(input_data)
print(decoded)
`

@mmm3bbb
Thanks for reporting. Can you post your transaction ID?

0.1.6 is released now. Tuple array is now supported.


Course vs Lecture naming

Current state: both naming is present which is unfortunate.
Which will we use? And refactor rest?
I think We should pick option will be more easy to refactor.

In *.swift

Course (whole word) - 25 matches in 8 files
Lecture (whole word) - 35 matches in 8 files
Lecture - 124
Course - 120

Hah, just made the same statistic. We should definitely pick one and keep it as convention. I used Course instead of Lecture because this convention is used on Edux.
If we move to Lecture, we should also rename parser objects and save short names (as Lecture) for model objects instead.
I also asked my friend who is native speaker about his opinion. So lets wait we will decide later..

Decided to use Course instead of Lecture. I will do refactor.

Changes in Feature/lecture-rename-course  branch


Fix clean-css version 4 compatibility

This fix add support for both clean-css v3 and v4. In clean-css v4 the
--root option was removed and is now automatically taken from --output.
See https://github.com/jakubpawlowicz/clean-css-cli
Closes #485.

Thanks!


BUG 2132892: controller: fix csiaddonsnodes object deletion

resolveEndpoint get the pod object to resolve the ip address as the pod object is deleted resolveEndpoint will fail to get the pod and it blocks the csiaddonsnodes object from getting deleted because of this logs will get flooded and other csiaddons operation might also fail.
To fix this problem call resolveEndpoint only if the csiaddonsnodes object is not deleted.
Signed-off-by: Madhu Rajanna<EMAIL_ADDRESS>(cherry picked from commit 08d7a490dbfa176ce0971cc869df1b4074e92a61)

/assign @nixpanic

/lgtm
/approve


feat: move API to typescript code generation

Can I get some advice of synth.py now it has too many post-processing for removing files or replacing file content, is there a faster way to do that? @alexander-fenster Thanks!

OK so removal of grafeas files can be done with a loop as I wrote in comments above. As for the "surgery", I'm thinking of maybe trying a mixin here? Let's talk tomorrow and see if we can make it in a more gentle way.


Can't find the .app file in the release

Is the release itself the .app file? I tried renaming the extracted zip, but no dice.

Only the source code is available. We're in the process of releasing the binaries on the Mac App Store.

So the current instructions from the README will be true once it's published to the app store?

I'll remove that as soon as I have the App Store link. Thanks.

On Apr 3, 2016, at 6:20 PM, Josh Teneycke<EMAIL_ADDRESS>wrote:
So the current instructions from the README will be true once it's published to the app store?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub


OSX installer - suboptimal user experience (and not working?)

Had a guest to the offices today excited to try out KA Lite, who downloaded the OSX installer and installed. There were numerous issues, ranging from "annoying" to "confusing" to "it didn't work at all". We were never successful in getting it up and running. I'll list a few things here, but it seems like what we need is a serious UX (and testing) sprint on the OSX installer/monitor, as it's currently in pretty a bad state, and likely losing us users (as well as potential supporters).

Still called "KA-Lite-Monitor" rather than "KA Lite"
After launching the app, there was no way to know that it was accessible via the task tray icon, or what the next steps should be.
The preferences window came up at some point, but it was very very confusing (even for me, and I know how it's supposed to work).

The "Admin name/password" form looked like it was for putting in the system password. No indication that it was for creating a superuser for use within KA Lite. Also, we don't need that now, since creating the superuser happens in the web interface.
Pressing "Apply" did nothing. It was unclear whether it had created a user, or done anything at all -- nothing changed. Window stayed open, and only options were still "Apply" and "Cancel".
 "Setup: this may delete all your data", etc. Pressing "setup" didn't seem to do anything, and didn't have any progress indicator or other feedback.
The KA Lite logo in the bottom left of the window had a spinner/progress icon on it, but it was frozen/not spinning.

We could not figure out how to get the server started. The options under the task tray dropdown were all grayed out, except Preferences. No apparent way to start the server. We eventually gave up.

OSX version was not El Capitan. Will request more details.
@jtamiace, perhaps you or an intern could help with trying this process out and making UX recommendations? A lot of low-hanging fruit, so it doesn't need to be an expert UX analyst.
(CC @aronasorman @cpauya)
Thanks!

Hi @jamalex,
So sorry to hear that the OSX app did not work with your guest, I assume it was v0.15.x?
With those issues, it's most likely that an error has occurred in the .app (or ka-lite) thus it did not work as expected, these may be visible in the Logs tab.
Some answers to your queries:

We have already removed the "monitor" on the develop branch (for 0.16.x release).

The .app have notifications to notify the user of things like: "KA-Lite is starting...", "Setup is finished!  You can now start KA-Lite.", etc.  Perhaps the system was on "Do Not Disturb" mode?

Regarding the Preferences window:

We have already removed the form for creating the admin username/password on the develop branch (for 0.16.x release).
When the Apply button was clicked, were there any messages on the Logs tab?
We have already removed the setup, extract assessment, and reset app in our feature/osx-0.16.x branch (due for 0.16.x).
We will remove the confusing spinner icon.

For tracing any issues about the .app:

On the Terminal, try typing kalite to see if it was installed.  If not, then something went wrong during the setup of ka-lite which happens on initial launch of the .app.
We can check output messages of the .app at the Logs tab.
We can also use the Console application at the /Applications/Utilities of the OSX machine, and filtering by "KA-Lite".

Yes, more details would be very helpful like which KA-Lite installer and OSX version were used.
Again, I apologise if the installer/.app did not work for you, something must have happened that we didn't catch on our tests.  I admit that the .app needs a lot of UX attention and would really appreciate if someone could jump in to help out.

@cpauya -- I could identify this as a possibly fixed / possibly remaining issue?

We will remove the confusing spinner icon.

Otherwise, this could be closed? :)

Closing this because of missing feedback - if we have to work out these issues, we would need to chunk them up into separate workable smaller issues.
CC: @mrpau-eduard @mrpau-richard @cpauya

A lot has happened to the Application UI and functionality since this issue was reported.
So yes @benjaoming, let's close this and open specific issues as they emerge.


Fix inferral of dynamic batch size from the config

Fixes #189
As per title. Need to add a test on reloading models compiled with dynamic batch size, will be in a coming PR.

Could you review @samir-souza , thanks!

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


gdpr compliance

Is your feature request related to a problem? Please describe
Right now we don't have a cookie warning and consent for the trackable data. This does not follow EU regulations.
Describe the solution you'd like
Please add a cookie consent solution, to be compliant, for example https://www.npmjs.com/package/vanilla-cookieconsent
Additional context
We are currently using tracking with matomo and functional local storage for light/dark mode.

Hello! In which file are cookies set and how (i have not found it)? Should this cookie warning also have a separate "Manage Settings" page?  I am also interested in functional local storage for light/dark mode, I switched the theme but the local storage is always empty.

Hello! In which file are cookies set and how (i have not found it)? Should this cookie warning also have a separate "Manage Settings" page? I am also interested in functional local storage for light/dark mode, I switched the theme but the local storage is always empty.

Feel free to implement that page! You don't have to use the library if it doesn't make sense from your perspective. Please open a separate issue for the light/dark mode switch.
Matomo is set via external script that is loaded in tracking.ts - that part needs to be adjusted

Hello! I added the en translation to the modal translation, but the locales files are in the gitignore file, should I leave the text instead of the 'Lang' component?

Hello! I added the en translation to the cookie consent modal, but the locales files are in the gitignore file, should I leave the text instead of the 'Lang' component?

the translations go innto the filess sin /language - the other files are generated form that via the npm run language command


Fix redirect to app after login

Your PR Server URL is https://near-wallet-pr-102.onrender.com.
Follow its progress at https://dashboard.render.com/web/srv-bko1r9efgkco0i8njmb0.
Cheers,
The Render Team


Support sorting by a child object's property value

Is your feature request related to a problem?
Let's say I have a Team Object that has a List of Member objects, each member containing FirstName, LastName, and a bool IsCaptain. If I return a list of Teams, I can generally display all data without issue. I may be displaying this data in a table and want to be able to sort by the Team captains name, however; Members would not be part of the TeamSortInput object as its an array. I'm not sure it being the array is the exact cause for it not showing in the schema definitions, but its been my experience so far.
The solution you'd like
I would like to be able to sort a List (usually IQueryable) items by any property including children's properties.
Product
Hot Chocolate

@h5aaimtron  sorting on nested object values is supported
Sorting by a member of a nested list is not because there is no order


Fix native memory leak

This PR address issue #108

@petreeftime I would appreciate your feedback on this if possible. It fixed the leak I've reported in issue #108  But I'm not sure I fully understand the dbus object allocation

This looked ok. Any reason why you closed it?

After a second look it seemed the leak was not fixed. And I didn't want to provide a patch that did not fully addresses the problem.
I'm still looking into it, using the small program provided in the issue description, and would like to reopen the PR once I can verify there is no memory changes
I think there is also a problem with the BluetoothAdapter constructor / destructor.
the given Adapter1 *object was provided by a g_dbus method. so calling g_object_ref(object) just increase the reference count by one more, leading to the destructor call to g_object_unref not releasing the memory.
Do you think it's safe to remove the g_object_ref call in BluetoothAdapter constructor?

No, g_object_ref needs to be there to keep the Adapter1 *object alive until the BluetoothAdapter goes away, and then, when the destructor gets called, the reference count should drop to zero and release the memory. The call to  g_list_free_full(objects, g_object_unref); should be at the end of the methods in which you added it, since that would definitely cause a leak.

To address the issue I had to fix 3 different issues. profiling the example code with this patch result with a steady memory usage that does not grow.

Release g_list references
Call g_object_unref in specific places where it was missing and the reference counter was not zero after a BluetoothObject destructor ran
Provide method to release memory by java client that does not wait for the finalizer thread

The java code used for tests is
public static void main(String[] args) throws InterruptedException {
	BluetoothAdapter adapter = BluetoothManager.getBluetoothManager().getAdapters().get(0);
	while (true) {
		List<BluetoothDevice> devices = adapter.getDevices();
		if (devices == null) continue;
		for (BluetoothDevice dev : devices)
			dev.close();
	}
}

Thanks for the review and approval


Baseline ignoring MaxLineLength: on 1.0.1

Expected Behavior
I upgraded from 1.0.0-RC14 to 1.0.1 and my baseline file stopped working.
Observed Behavior
I had a baseline file like so:
<?xml version="1.0" ?>
<SmellBaseline>
  <Whitelist>
    <ID>MaximumLineLength:</ID>
    <!-- other rules with specific signatures -->
  </Whitelist>
</SmellBaseline>

After upgrading I get over 200 issues failed with MaxLineLength. Even if I add <ID>MaxLineLength:</ID> to the baseline file it fails.
Steps to Reproduce
Relevant build.gradle
plugins {
id "io.gitlab.arturbosch.detekt" version "1.0.1"
}

dependencies {
detektPlugins "io.gitlab.arturbosch.detekt:detekt-formatting:1.0.1"
}

detekt {
    toolVersion = "1.0.1"
    input = files("src/main/kotlin", "src/test/kotlin")
    filters = ".*/resources/.*,.*/build/.*"
    baseline = file(".config/detektBaseline.xml")
}

Context
If I regenerate the baseline it adds each of the occurrences of MaxLineLength with it's own signature and at this point it works but the baseline file becomes unmaintainable.
Your Environment

Version of detekt used: 1.0.1
Version of Gradle used (if applicable): 5.2.1
Operating System and version: macOS 10.14.5
Link to your project (if it's a public repository):

If you want to ignore all instances of MaximumLineLength you could disable the rule in the YAML config file instead.
I don't know the history of baseline file support so I'm not sure whether adding items to the baseline like that was ever actually supported, but I might be wrong. The docs only describe adding the entire signature to the baseline.

@nicodn Please be aware that there are two rules which basically do the same.
MaxLineLength is the detekt native rule. This one provides more configuration options.
MaximumLineLength is the wrapped ktlint rule.
So you need to choose between one of the two.
In your code only MaximumLineLength is included in your baseline file. MaxLineLength is missing.
HTH

Hi @schalkms , thanks for your reply.
I tried adding MaxLineLength: to the whitelist in the baseline file but it still gives the 200+ errors.
It's strange because on 1.0.0-RC14 it never reported any issues with MaxLineLength.

Hi, the baseline does not support the formatting ruleset (yet).
Also an ID entry looks like this NewLineAtEndOfFile:PluginVersion.kt$io.gitlab.arturbosch.detekt.PluginVersion.kt</ID>.
It is not only the rule id but the signature of a code element. In this case it is the whole file itself.
Try running detektBaseline and see what happens.
Additionally we could indeed support suppressing formatting findings via baseline like:
MaximumLineLengthPluginVersion.kt$io.gitlab.arturbosch.detekt.PluginVersion.kt#[line].[column]</ID>.

@arturbosch What do you mean by that?
Currently, the baseline feature doesn't support rules in the formatting rule set.
We would need to adapt detekt's codebase in order to support this feature.
Am I right?

@arturbosch Some additional info.  It looks like the formatting of the MaxLineLength rule changed for certain elements.  I recently upgraded to 1.0.1 from RC14 and saw a lot of errors that were already in the baseline.  I reran the baseline and saw that rules previously formatted like this:
<ID>MaxLineLength:FileName.kt$ClassName$private val fieldName = {code}</ID>
<ID>MaxLineLength:FileName.kt$ClassName$override fun method(...) {</ID>

became this after the baseline was regenerated:
<ID>MaxLineLength:FileName.kt$ClassName$private</ID>
<ID>MaxLineLength:FileName.kt$ClassName$override</ID>

They are no longer uniquely identified and apparently no longer match older baselines.

I could not reproduce this issue, my tested MaxLineLength findings have a correct signature.
Could you test it with 1.1.1 again?
I've implemented a way to suppress formatting issues in the link PR.

I could reproduce this on the sonar-kotlin repository.
The signature is right and contains the whole line but is not considered when filtering findings.

@arturbosch Is this reproducible after 6b21237 in your scenario?

@schalkms baseline suppressing works as expected in current master ^^